A stateless broadcasting method and system based on multi-modal time series prediction

Abstract

The application discloses a stateless broadcasting method and system based on multi-modal time series prediction, comprising the following steps: acquiring state data of a broadcasting device in real time, preprocessing the state data of the broadcasting device to obtain broadcasting state data; constructing a joint prediction decision model, inputting the broadcasting state data into the joint prediction decision model, and generating a control instruction according to the broadcasting state data by the joint prediction decision model; and controlling the broadcasting device to broadcast according to the control instruction. The application acquires the broadcasting device data, generates the control instruction according to the broadcasting device data by the joint decision model, thereby controlling the broadcasting, and solves the problems of the complex and poor expansibility of the existing broadcasting system.

Description

Technical Field

[0001] This invention relates to the field of broadcasting technology, and in particular to a stateless broadcasting method and system based on multimodal timing prediction. Background Technology

[0002] With the increasing application of broadcast systems, device status management is a crucial component in ensuring smooth playback of broadcast content. Traditional broadcast systems typically rely on device status data for management, such as CPU utilization, memory usage, audio quality, and network connection status of terminal broadcast devices. The system makes decisions based on this device status information, including playing, pausing, or adjusting broadcast content. However, as the scale and complexity of devices increase, its drawbacks become increasingly apparent.

[0003] However, this method has the following drawbacks:

[0004] The strong coupling between broadcasting equipment requires the system to transmit and synchronize status information between multiple devices, which makes the system complex and difficult to maintain, leading to increased costs for troubleshooting.

[0005] Secondly, the management of each broadcast equipment status information requires the system to process a large amount of equipment information, which results in poor system scalability and difficulty in making flexible adjustments. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a stateless broadcasting method and system based on multimodal time-series prediction. By acquiring broadcasting device data and generating control commands based on the broadcasting device data through a joint decision model, the broadcasting can be controlled, thus solving the problems of complexity and poor scalability in existing broadcasting systems.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A stateless broadcasting method based on multimodal temporal prediction includes the following steps:

[0009] S1. Acquire the status data of the broadcasting equipment in real time, preprocess the status data of the broadcasting equipment, and obtain the broadcast status data;

[0010] S2. Construct a joint prediction and decision model, input broadcast status data into the joint prediction and decision model, and generate control commands based on the broadcast status data;

[0011] S3. Control the broadcasting equipment to broadcast according to the control instructions.

[0012] Furthermore, the status data of the broadcasting device includes: device data, audio playback data, and network connection data.

[0013] Furthermore, in step S1, the real-time acquisition of the broadcast device's status data is specifically implemented as follows:

[0014] The broadcasting device includes a performance monitoring interface, through which device data of the broadcasting device is obtained. The device data includes CPU usage status, memory usage status, and timestamp.

[0015] Audio playback data is collected from the audio stream of the broadcasting equipment. The audio playback data includes playback parameters, audio parameters, and playback status.

[0016] The broadcasting device includes a network interface, through which network connection data of the broadcasting device is obtained. The network connection data includes bandwidth, latency, packet loss rate, network connection status, and timestamp.

[0017] Furthermore, the preprocessing of the status data of the broadcasting equipment specifically includes: cleaning the data, removing empty data, and deduplicating.

[0018] Furthermore, in step S2, the specific implementation of inputting the broadcast status data into the joint prediction decision model is as follows: the broadcast status data is integrated into JSON format data to obtain JSON broadcast data, and the JSON broadcast data is input into the joint prediction decision model.

[0019] Furthermore, in step S2, the specific structure of the joint prediction decision model includes a time series prediction module, a multimodal fusion module, and a joint prediction decision module.

[0020] Furthermore, the time-series prediction module is a long short-term memory network, and the time-series prediction module includes a forget gate, an input gate, an output gate, and a memory gate. The loss function of the time-series prediction module is the mean squared error function.

[0021] The specific working method of the time-series prediction module is as follows: each time broadcast status data is collected, it is a time step. At each time step, the broadcast status data is converted into an input vector. At each time step, the input vector is input to the time-series prediction module. The time-series prediction module predicts the prediction indicators for several future time steps. The prediction indicators include device indicators, audio indicators, and network indicators.

[0022] Furthermore, the multimodal fusion module is a recurrent neural network. For each type of data in the broadcast state data, the multimodal fusion module maps each type of data to a feature vector space of fixed dimension, aligns them on the same output dimension, and transforms them into feature vectors. The feature vectors of each type of data are concatenated to obtain a feature sequence.

[0023] Furthermore, the joint prediction decision module makes decisions based on prediction indicators and feature sequences. Specifically, it performs weighted fusion of prediction indicators and feature sequences to obtain an evaluation vector, and generates control commands based on the evaluation vector.

[0024] Through the above technical solution, the present invention has the following beneficial effects: by collecting and analyzing the status of terminal devices in real time, and based on the joint decision-making model and multimodal device data, the management and playback of broadcast content is transformed from traditional device status management to intelligent data-driven stateless management through intelligent management of broadcast devices. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of a stateless broadcasting method based on multimodal temporal prediction according to the present invention.

[0026] Figure 2 This is a schematic diagram of a stateless broadcast system based on multimodal temporal prediction in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Example 1

[0030] See Figure 1 A stateless broadcasting method based on multimodal temporal prediction includes the following steps:

[0031] S1. Acquire the status data of the broadcasting equipment in real time, preprocess the status data of the broadcasting equipment, and obtain the broadcast status data;

[0032] S2. Construct a joint prediction and decision model, input broadcast status data into the joint prediction and decision model, and generate control commands based on the broadcast status data;

[0033] S3. Control the broadcasting equipment to broadcast according to the control instructions.

[0034] In an optional embodiment, the status data of the broadcasting device includes: device data, audio playback data, and network connection data.

[0035] In an optional embodiment, the real-time acquisition of the broadcast device's status data in step S1 is specifically implemented as follows:

[0036] The broadcasting device includes a performance monitoring interface, through which device data of the broadcasting device is obtained. The device data includes CPU usage status, memory usage status, and timestamp.

[0037] Audio playback data is collected from the audio stream of the broadcasting equipment. The audio playback data includes playback parameters, audio parameters, and playback status.

[0038] The broadcasting device includes a network interface, through which network connection data of the broadcasting device is obtained. The network connection data includes bandwidth, latency, packet loss rate, network connection status, and timestamp.

[0039] Specifically, the CPU and memory usage is obtained through the performance monitoring interface of the broadcast device's operating system. The performance monitoring will periodically record the current CPU usage value, including the system CPU usage and the user CPU usage, and record the total memory and used memory of the device. The timestamp is the time when the device data is obtained.

[0040] In an optional embodiment, the preprocessing of the status data of the broadcasting device specifically includes: cleaning the data, removing empty data, and deduplicating.

[0041] Specifically, for device data, missing CPU or memory values ​​can be filled with the historical average or historical median of the CPU or memory data of the broadcasting device, or they can be marked as "data missing". For outliers, especially those caused by failures or communication interruptions during acquisition, the outliers can be cleaned up or corrected. For duplicate data, duplicate data can be removed, and the device data with the most recent acquisition time can be retained.

[0042] For audio playback data, playback parameters include volume, audio parameters include audio quality and playback duration, and playback status includes buffering and pause. Each record is checked for missing data; if playback duration is missing, it is set to 0 by default; if volume is missing, it is set to the median of the maximum volume. The audio playback data is cleaned, including:

[0043] Volume error: The volume should be between 0 and 100. If it exceeds this range, correct it to 0 or 100. Playback duration error: The playback duration should be a non-negative integer. If it is negative or invalid, correct it to 0. Buffering status: The audio is buffering. Mark this status and add a label to the data.

[0044] Deduplicat audio playback data; for data with the same timestamp, check for duplicate records and retain valid data.

[0045] For abnormal data, if the playback duration is abnormal, the playback duration will be estimated or filled in using other relevant fields. For abnormal volume and playback duration, the same normal frames will be used to complete and correct the data.

[0046] For network connection data, check for missing or unavailable data fields. Missing data can be filled with default values. For outlier values, the bandwidth value should be within a reasonable range (e.g., 0-1000Mbps). If the value is too large or too small, it is considered outlier. The latency value should generally be less than 1000ms. If it exceeds this value, it is considered outlier. The packet loss rate should be between 0% and 100%. If it exceeds this range or is negative, it is considered invalid data. The network connection status is comprehensively evaluated based on bandwidth, latency, and packet loss rate. If the packet loss rate is high or the latency is large, the network connection status is marked as poor.

[0047] Deduplicat network connection data to ensure that each record represents a unique timestamp and remove duplicate records.

[0048] In an optional embodiment, in step S2, the specific implementation of inputting the broadcast state data into the joint prediction decision model is as follows: the broadcast state data is integrated into JSON format data to obtain JSON broadcast data, and the JSON broadcast data is input into the joint prediction decision model.

[0049] The JSON data of the broadcast device status, audio playback status, and network connection status obtained above are combined to form a comprehensive JSON broadcast data input into the joint prediction and decision model. The joint prediction and decision model will intelligently analyze the device status and generate corresponding control commands, enabling the broadcast device to play stateless broadcasts without relying on the server system, and the server system to play stateless broadcasts without relying on the device status.

[0050] In an optional embodiment, in step S2, the specific structure of the joint prediction decision model includes a time series prediction module, a multimodal fusion module, and a joint prediction decision module.

[0051] The Joint Predictive Decision Model (GPDM) is adopted to realize stateless management and intelligent control of the broadcast system. The model consists of three core sub-modules: the Time Series Prediction Module (TSPM), the Multimodal Fusion Module (MMFM), and the Joint Predictive Decision Module (JDM), forming an adaptive control system model of "prediction-fusion-optimization-decision".

[0052] In an optional embodiment, the time-series prediction module is a long short-term memory network, which includes a forget gate, an input gate, an output gate, and a memory gate. The loss function of the time-series prediction module is a mean squared error function.

[0053] The specific working method of the time-series prediction module is as follows: each time broadcast status data is collected, it is a time step. At each time step, the broadcast status data is converted into an input vector. At each time step, the input vector is input to the time-series prediction module. The time-series prediction module predicts the prediction indicators for several future time steps. The prediction indicators include device indicators, audio indicators, and network indicators.

[0054] The time series prediction module models the time series patterns of historical data based on the Long Short-Term Memory (LSTM) network, captures long-term dependencies, and predicts the trends of key indicators in the future time period. When constructing the time series prediction module, a large amount of historical data is acquired to train the time series prediction module. During each training, the weight of the prediction module is adjusted by using the mean squared error as a loss function until the loss is less than a preset threshold, and then the training ends.

[0055] In an optional embodiment, the multimodal fusion module is a recurrent neural network. For each type of data in the broadcast state data, the multimodal fusion module maps each type of data to a feature vector space of fixed dimension, aligns them on the same output dimension, and transforms them into feature vectors. The feature vectors of each type of data are concatenated to obtain a feature sequence.

[0056] By utilizing a recurrent neural network and a multimodal feature extraction mechanism, each data category is encoded separately to generate low-dimensional, uniform-dimensional feature vectors. This results in globally unified fused features, which serve as feature indicators for subsequent decision-making.

[0057] Specifically, device status data, audio playback data, and network status data are each converted into feature matrices via neural networks. The feature vectors of different modalities of the status data are concatenated on the time axis or feature axis to form a longer feature sequence. This unified feature representation comprehensively reflects the status of device status data, audio playback status data, and network status data, serving as a data feature indicator.

[0058] In an optional embodiment, the joint prediction decision module makes decisions based on prediction indicators and feature sequences. Specifically, the prediction indicators and feature sequences are weighted and fused to obtain an evaluation vector, and control instructions are generated based on the evaluation vector.

[0059] Based on the predicted state data obtained from the time-series prediction module and the multimodal fusion module, the objective function of the joint prediction decision module is F = α * β, and its physical meaning is:

[0060] α represents the future temporal evolution trend, β represents the global state characteristics of the current multimodal data, and the result of F represents the global optimal control quantity combining temporal prediction and multimodal analysis.

[0061] Through the joint prediction and decision-making module, the system can adjust and optimize its objectives in real time based on the current actual operating conditions (such as sudden network changes and equipment load fluctuations), forming an adaptive joint prediction and control capability. It outputs optimal control strategy parameters (such as playback control, resource scheduling, and anomaly warnings) to control the operation of broadcast equipment.

[0062] Specifically, the broadcasting equipment is located in the broadcasting system, which includes a broadcasting server and several broadcasting devices. The broadcasting server obtains the broadcasting status of different broadcasting devices through the control commands and sends the status to the broadcasting server. The broadcasting server then uses the obtained command status to play the broadcast.

[0063] Example 2

[0064] See Figure 2 A stateless broadcast system based on multimodal time-series prediction, comprising:

[0065] The data acquisition module is used to acquire the status data of the broadcasting equipment in real time, preprocess the status data of the broadcasting equipment, and obtain the broadcast status data.

[0066] The instruction generation module is used to build a joint prediction and decision model. Broadcast status data is input into the joint prediction and decision model, and the joint prediction and decision model generates control instructions based on the broadcast status data.

[0067] The broadcast control module is used to control the broadcasting equipment to broadcast according to control commands.

[0068] The embodiments disclosed in this specification are merely illustrative of one aspect of the invention, and the scope of protection of the invention is not limited to these embodiments. Any other functionally equivalent embodiments fall within the scope of protection of the invention. Those skilled in the art can make various other corresponding changes and modifications based on the technical solutions and concepts described above, and all such changes and modifications should fall within the scope of protection of the claims of this invention.

Claims

1. A stateless broadcasting method based on multimodal temporal prediction, characterized in that, Includes the following steps: S1. Acquire the status data of the broadcasting equipment in real time, preprocess the status data of the broadcasting equipment, and obtain the broadcast status data. The status data of the broadcasting equipment includes: device data, audio playback data, and network connection data; S2. Construct a joint prediction and decision-making model. Input the broadcast state data into the joint prediction and decision-making model, and the joint prediction and decision-making model generates control commands based on the broadcast state data. The specific structure of the joint prediction decision model includes a time series prediction module, a multimodal fusion module, and a joint prediction decision module. The multimodal fusion module is a recurrent neural network. For each type of data in the broadcast state data, the multimodal fusion module maps each type of data to a fixed-dimensional feature vector space, aligns them on the same output dimension, and transforms them into feature vectors. The feature vectors of each type of data are then concatenated to obtain a feature sequence. The joint prediction decision module makes decisions based on prediction indicators and feature sequences. Specifically, it is implemented by weighting and fusing the prediction indicators and feature sequences to obtain an evaluation vector, and then generating control commands based on the evaluation vector. S3. Control the broadcasting equipment to broadcast according to the control instructions.

2. The stateless broadcasting method based on multimodal temporal prediction according to claim 1, characterized in that, In step S1, the real-time acquisition of the broadcast device's status data is specifically implemented as follows: The broadcasting device includes a performance monitoring interface, through which device data of the broadcasting device is obtained. The device data includes CPU usage status, memory usage status, and timestamp. Audio playback data is collected from the audio stream of the broadcasting equipment. The audio playback data includes playback parameters, audio parameters, and playback status. The broadcasting device includes a network interface, through which network connection data of the broadcasting device is obtained. The network connection data includes bandwidth, latency, packet loss rate, network connection status, and timestamp.

3. The stateless broadcasting method based on multimodal temporal prediction according to claim 2, characterized in that, The preprocessing of the status data of the broadcasting equipment specifically includes: cleaning the data, removing empty data, and deduplicating.

4. The stateless broadcasting method based on multimodal temporal prediction according to claim 1, characterized in that, In step S2, the specific implementation of inputting the broadcast status data into the joint prediction decision model is as follows: the broadcast status data is integrated into JSON format data to obtain JSON broadcast data, and the JSON broadcast data is input into the joint prediction decision model.

5. The stateless broadcasting method based on multimodal temporal prediction according to claim 1, characterized in that, The time-series prediction module is a long short-term memory network, which includes a forget gate, an input gate, an output gate, and a memory gate. The loss function of the time-series prediction module is the mean squared error function. The specific working method of the timing prediction module is as follows: each time broadcast status data is collected, it is a time step. At each time step, the broadcast status data is converted into an input vector. At each time step, the input vector is input to the timing prediction module. The timing prediction module predicts the prediction indicators for several future time steps. The prediction indicators include device indicators, audio indicators, and network indicators.

6. A stateless broadcasting system based on multimodal time-series prediction, characterized in that, include: The data acquisition module is used to acquire the status data of the broadcasting equipment in real time, preprocess the status data of the broadcasting equipment to obtain broadcast status data, which includes: device data, audio playback data and network connection data. The instruction generation module is used to construct a joint prediction and decision model. Broadcast state data is input into the joint prediction and decision model, which then generates control instructions based on the broadcast state data. The specific structure of the joint prediction and decision model includes a time-series prediction module, a multimodal fusion module, and a joint prediction and decision module. The multimodal fusion module is a recurrent neural network. For each type of data in the broadcast state data, the multimodal fusion module maps each type of data to a fixed-dimensional feature vector space, aligns them on the same output dimension, and transforms them into feature vectors. The feature vectors of each type of data are concatenated to obtain a feature sequence. The joint prediction and decision module makes decisions based on prediction indicators and the feature sequence. Specifically, it performs weighted fusion of the prediction indicators and the feature sequence to obtain an evaluation vector, and then generates control instructions based on the evaluation vector. The broadcast control module is used to control the broadcasting equipment to broadcast according to control commands.

Images