Data video stream adaptive processing system and method based on stream processing technology
Through the data video stream adaptive processing system of stream processing technology, the network status and video content are monitored in real time, and the compression code rate and frame rate of the video stream are dynamically adjusted, which solves the traditional system's instability in network environment and diversified terminal performance. Video playback issues achieve a smoother and higher-quality video experience.
Patent Information
- Application Number
- CN202411155021.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-22
AI Technical Summary
In the case of unstable network environment or diversified user terminal performance, traditional video stream processing systems cannot effectively adjust the quality and resolution of video streams, resulting in a decrease in the continuity and clarity of video playback, affecting the user experience.
Adaptive processing system for data video streams based on stream processing technology is adopted, and the network quality prediction module, video complexity evaluation module and real-time optimization and adjustment module are combined with long and short-term memory network and recurrent neural network to monitor network status and video content in real time. Dynamically adjust the compression code rate and frame rate of the video, match network and device conditions, and optimize the stability of the video stream.
In the dynamically changing network environment, the smoothness and picture quality of video playback are improved, the stability and quality of video streams are enhanced, and the user's viewing experience is improved.
Smart Images

Figure CN118694945B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video stream processing, and in particular to a data video stream adaptive processing system and method based on stream processing technology. Background Art
[0002] The field of video stream processing technology involves the processing of all aspects of video signal capture, encoding, transmission, storage and display. The core technologies in the field of video stream processing include video compression and encoding and decoding, which ensure that data is effectively transmitted under limited bandwidth while maintaining the highest possible image quality. In addition, video stream processing also involves adaptive transmission of content, that is, adjusting the quality and resolution of the video stream according to network conditions and the capabilities of the terminal device. This includes the use of various algorithms and protocols to manage buffering, error correction and synchronization issues of video data to improve service reliability and user experience.
[0003] Among them, the data video stream adaptive processing system is a system specially designed to optimize the transmission of video content, so that it can automatically adjust the quality of the video data stream according to different network environments and user device performance. The main purpose of this system is to provide the best viewing experience under different network bandwidths and device processing capabilities, and avoid video freezes or quality degradation caused by network fluctuations. By monitoring the network status and the performance of the endpoint device in real time, the system can dynamically adjust the video resolution, bit rate and compression level to achieve efficient and smooth video playback.
[0004] Traditional video streaming processing systems focus on the application of fixed algorithms and standard protocols, and are insufficiently responsive to the variability of network environments and the personalized needs of users. In particular, when network quality is unstable or user terminal performance is diverse, traditional systems output videos at a fixed compression rate. In the case of limited bandwidth or low device processing power, the continuity and clarity of video playback cannot be guaranteed. For example, when users use video streaming services in a mobile environment, such as on high-speed vehicles, fixed compression strategies cannot adapt to the rapid changes in network quality, often resulting in frequent video loading and buffering, affecting the user's viewing experience. Summary of the invention
[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a data video stream adaptive processing system and method based on stream processing technology.
[0006] In order to achieve the above object, the present invention adopts the following technical solution: a data video stream adaptive processing system based on stream processing technology, the system comprising:
[0007] The network quality prediction module is based on the video streaming environment, monitors the network status during the video streaming process, evaluates the streaming network quality, and analyzes the streaming network change trend based on the streaming network quality data over a period of time through the long short-term memory network combined with the recurrent neural network, predicts the streaming network quality in the future period, and obtains the predicted network quality data;
[0008] The video complexity evaluation module evaluates the complexity of the video stream based on the video data to be streamed, by analyzing the number of differentiated pixels between video frames and combining the changes in the static and dynamic states of the video screen, and obtains the video stream complexity information;
[0009] The compression bit rate adjustment module adjusts the compression bit rate and frame rate of the video based on the predicted network quality data and the video stream complexity information, combined with the screen size and screen pixel density of the video streaming terminal, to match the network and device conditions, and obtain the adjusted video streaming information;
[0010] The real-time optimization and adjustment module monitors the video streaming process based on the adjusted video streaming information, evaluates the stability of the video stream by analyzing the video performance, and optimizes the video compression bit rate and frame rate according to the stability of the video stream to obtain real-time optimized streaming information.
[0011] The present invention is improved in that the method for evaluating the quality of the streaming network is:
[0012] Based on the video streaming environment, monitor the network status during the video streaming process and collect network status data during the video streaming process. The network status data includes bandwidth, delay and packet loss rate.
[0013] Based on the network status data, by the formula:
[0014]
[0015] Calculating network quality score ,in, is the network bandwidth, For network delay, is the packet loss rate, , and is the weight coefficient, Rate the network quality, is the base of natural logarithms;
[0016] Based on the network quality score ,according to The value is used to evaluate the network quality and obtain the streaming network quality information.
[0017] The present invention is improved in that the step of obtaining the predicted network quality data is:
[0018] Based on the streaming network quality information, collecting streaming network quality data within a period of time to obtain a network quality time data set;
[0019] Based on the network quality time dataset, the data is processed by long short-term memory units to obtain the network quality data at each time point. The value is analyzed using the formula:
[0020]
[0021] Calculate network quality score for future time periods , get the predicted network quality data, where, Rate the current network quality. is the average of historical network quality scores. is the standard deviation of historical network quality scores, is the time difference factor, Score the network quality for future time periods, , and is the weight coefficient.
[0022] The present invention is improved in that the step of obtaining the video stream complexity information is:
[0023] Based on the video data to be streamed, by comparing the video data frame by frame, identifying the change of pixel values between frames, calculating the number of differential pixels between video frames, and obtaining differential pixel number information;
[0024] Based on the differential pixel quantity information, static and dynamic state changes in the video screen are analyzed, and the number of dynamic frames in the video is counted. The dynamic frame is judged by comparing the differential pixel quantity with a preset threshold to obtain dynamic frame quantity data;
[0025] Based on the differentiated pixel quantity information and dynamic frame quantity data, the formula is:
[0026]
[0027] Calculating video stream complexity score , get the video stream complexity information, where, represents the number of differentiated pixels, is the number of dynamic frames, is the total number of frames in the video, , and is the weight coefficient, Score the video stream complexity.
[0028] The present invention is improved in that the step of obtaining the adjusted video stream information is:
[0029] Based on the predicted network quality data and video stream complexity information, the screen size and screen pixel density data of the video streaming terminal are obtained to obtain the stream association data;
[0030] Based on the stream association data, by formula:
[0031]
[0032] Calculate the adjusted compression bitrate , and by the formula:
[0033]
[0034] Calculate the adjusted video frame rate , get the adjusted video stream information, where, Score the complexity of the video stream, Score the network quality for future time periods, , , and is the adjustment factor, Indicates the adjusted compression bit rate, is the basic compression bit rate, is the adjusted video frame rate, is the basic video frame rate, The network bandwidth.
[0035] The present invention is improved in that the method for evaluating the stability of the video stream is:
[0036] Based on the adjusted video stream information, monitor key performance indicators of the video stream in real time, where the key performance indicators include the number of buffering events, frame loss rate, and delay time;
[0037] Based on the key performance indicators, the formula:
[0038]
[0039] Calculating the stability score of a video stream ,in, Indicates the stability score of the video stream. is the frequency of buffering events, is the frame loss rate, is the average delay time, , and is the adjustment factor, is the attenuation coefficient, is the base of natural logarithms;
[0040] Stability score based on the video stream ,according to The value is used to evaluate the stability of the video stream and obtain the video stability evaluation result.
[0041] The present invention is improved in that the step of obtaining the real-time optimized streaming information is as follows:
[0042] Stability score based on the video stream , set the stability threshold, through the formula:
[0043]
[0044] Calculate the optimized compression bitrate , and by the formula:
[0045]
[0046] Calculate optimized video frame rate ,in, is the current compression bit rate, is the current video frame rate, Rate the video stream stability, is the stability threshold, and is the adjustment factor, and is the power coefficient;
[0047] Based on the optimized compression bit rate and optimized video frame rate , adjust the compression bit rate and frame rate during video streaming to obtain real-time optimized streaming information.
[0048] The data video stream adaptive processing method based on the streaming technology is executed based on the data video stream adaptive processing system based on the streaming technology, and comprises the following steps:
[0049] S1: Based on the video streaming environment, monitor and record the network status parameters during the streaming process, evaluate the streaming network quality, analyze and predict the network quality in the future period through the long short-term memory network and recurrent neural network, and obtain the predicted network quality data;
[0050] S2: Based on the video data to be streamed, the pixel difference between the frames is calculated, and the complexity of the video is evaluated by combining the static and dynamic changes of the picture to obtain the complexity information of the video stream;
[0051] S3: Based on the predicted network quality data and the video stream complexity information, combined with the screen size and pixel density of the video streaming terminal, adjusting the video compression bit rate and frame rate to obtain adjusted video streaming information;
[0052] S4: Based on the adjusted video streaming information, monitor the video streaming process, analyze the video performance, evaluate the stability of the video stream, and obtain a video stability evaluation result;
[0053] S5: Based on the video stability evaluation result, the compression bit rate and frame rate of the video stream are optimized to obtain real-time optimized stream information.
[0054] Compared with the prior art, the advantages and positive effects of the present invention are:
[0055] In the present invention, by real-time monitoring and analysis of the network status during the video streaming process, the future trend of the network quality is predicted, and the transmission quality of the video data can be adjusted in a dynamically changing network environment. By evaluating the complexity of the video data stream, the compression bit rate and frame rate can be adjusted in time according to the real-time dynamic and static changes of the video content, thereby optimizing the smoothness and image quality of the video playback. By continuously monitoring the video performance during the adjustment process, the compression bit rate and frame rate are optimized according to the stability of the video stream, thereby enhancing the stability and quality of the video stream and improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 is a system flow chart of the present invention;
[0057] Figure 2 A flow chart of evaluating streaming network quality according to the present invention;
[0058] Figure 3 A flow chart for obtaining predicted network quality data according to the present invention;
[0059] Figure 4 A flowchart of obtaining video stream complexity information according to the present invention;
[0060] Figure 5 A flowchart of the present invention for obtaining adjusted video stream information;
[0061] Figure 6 A flow chart for evaluating the stability of a video stream according to the present invention;
[0062] Figure 7 The present invention is a flow chart of obtaining real-time optimized streaming information. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0064] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0065] See also Figure 1 The present invention provides a technical solution: a data video stream adaptive processing system based on stream processing technology, the system comprising:
[0066] The network quality prediction module is based on the video streaming environment, monitors the network status during the video streaming process, and uses the bandwidth, delay index and packet loss rate of the streaming network to evaluate the streaming network quality. Based on the streaming network quality data over a period of time, the long short-term memory network combined with the recurrent neural network analyzes the streaming network change trend, predicts the streaming network quality in the future period, and obtains the predicted network quality data;
[0067] The video complexity evaluation module evaluates the complexity of the video stream based on the video data to be streamed, by analyzing the number of differentiated pixels between video frames and combining the changes in the static and dynamic states of the video screen, and obtains the video stream complexity information;
[0068] The compression bit rate adjustment module adjusts the compression bit rate and frame rate of the video based on the predicted network quality data and video stream complexity information, combined with the screen size and screen pixel density of the video streaming terminal, matches the network and device conditions, and obtains the adjusted video streaming information;
[0069] The real-time optimization and adjustment module monitors the video streaming process based on the adjusted video streaming information, evaluates the stability of the video stream by analyzing the video performance, and optimizes the video compression bit rate and frame rate according to the stability of the video stream to obtain real-time optimized streaming information.
[0070] The predicted network quality data includes the predicted network bandwidth range, the predicted average delay time and the predicted packet loss rate percentage. The video stream complexity information includes the number of pixel changes between video frames, the percentage of total video pixel changes and the frequency of dynamic scene changes. The adjusted video streaming information includes the adjusted video bit rate, the adjusted resolution setting and the adjusted frame rate. The real-time optimized streaming information includes the real-time adjusted compression parameters, the real-time evaluated smoothness score and the real-time adjusted buffer size.
[0071] See also Figure 2 , the method to evaluate the streaming network quality is:
[0072] Based on the video streaming environment, monitor the network status during the video streaming process and collect network status data during the video streaming process. The network status data includes bandwidth, delay and packet loss rate.
[0073] Based on the network status data, through the formula:
[0074]
[0075] Calculating network quality score ,in, is the network bandwidth, For network delay, is the packet loss rate, , and is the weight coefficient, Rate the network quality, is the base of natural logarithms;
[0076] Based on network quality rating ,according to The value is used to evaluate the network quality and obtain the streaming network quality information.
[0077] formula:
[0078]
[0079] Parameter details and how to obtain them:
[0080] bandwidth : Obtained in real time through network performance monitoring tools, it reflects the data transmission capacity of the network, usually measured in Mbps (megabits per second). For example, the average bandwidth is recorded and calculated regularly through network monitoring software such as Wireshark or Cisco's network management tools.
[0081] Network latency : Measured by a network delay test tool (such as the ping command), it indicates the time required for data to travel from the sending point to the receiving point, usually in milliseconds (ms). This parameter usually reflects the response time and efficiency of the network.
[0082] Packet loss rate : An important indicator of network quality, calculated by network monitoring tools and usually expressed as a percentage. Packet loss rate is calculated by dividing the number of packets that were not successfully transmitted by the total number of packets sent within a certain period of time.
[0083] Weight coefficient , and :The weight coefficient is set according to the actual application requirements of the network, reflecting the influence of different network parameters on the overall network quality assessment. The setting of the coefficient usually requires statistical analysis based on historical data and network performance indicators.
[0084] Calculation example:
[0085] Assume the following data:
[0086] bandwidth Mbps, Latency ms, packet loss rate , the weight coefficient is assumed to be .
[0087] Calculating network quality score :
[0088]
[0089] Calculated The value is 50.19, which reflects the quality of video streaming under the current network status.
[0090] See also Figure 3 ,The steps to obtain the predicted network quality data are:
[0091] Based on the streaming network quality information, the streaming network quality data within a period of time is collected to obtain a network quality time data set;
[0092] Based on the network quality time dataset, the data is processed by long short-term memory units to obtain the quality of each time point. The value is analyzed using the formula:
[0093]
[0094] Calculate network quality score for future time periods , get the predicted network quality data, where, Rate the current network quality. is the average of historical network quality scores. is the standard deviation of historical network quality scores, is the time difference factor, Score the network quality for future time periods, , and is the weight coefficient.
[0095] formula:
[0096]
[0097] Parameter details and how to obtain them:
[0098] Weight coefficient , and : Used to adjust the influence of different data sources. Usually set through statistical analysis or based on expert experience. For example, if historical data shows that latency changes have a greater impact on network quality, the weights related to latency may be increased.
[0099] Current network quality score : It is the network quality score measured in the most recent evaluation cycle, which can be calculated by the previous steps.
[0100] The average value of the historical rating dataset : It is obtained by averaging the network quality scores over a certain period of time in the past, usually by calculating the average value from historical data stored in a database.
[0101] Standard deviation of historical data : Indicates the degree of fluctuation of the historical network quality score, which is obtained by calculating the standard deviation of the score data set, which is also calculated by querying the database for historical data and applying the standard formula.
[0102] Time difference factor : Adjust the parameters to reflect the distance between the future prediction time point and the current time. This can be set based on the needs of the prediction model. For example, if the network status is predicted within the next hour, is set to 1, while the state predicted one day from now might be set to 24.
[0103] Calculation example:
[0104] Assume the following parameter values:
[0105] , , , (Current network quality score), (average of historical network quality scores), (standard deviation of historical data), (The prediction time interval is 2 hours).
[0106] calculate :
[0107]
[0108] Calculated The value is approximately 89.87, which is higher than the current and historical averages, indicating that the forecast model predicts that the network quality will improve in the next two hours.
[0109] See also Figure 4 , the steps to obtain the video stream complexity information are:
[0110] Based on the video data to be streamed, by comparing the video data frame by frame, identifying the change of pixel values between frames, calculating the number of differential pixels between video frames, and obtaining differential pixel number information;
[0111] Based on the differential pixel number information, the static and dynamic state changes in the video screen are analyzed, and the number of dynamic frames in the video is counted. The dynamic frame is judged by comparing the differential pixel number with the preset threshold to obtain the dynamic frame number data;
[0112] Based on the differentiated pixel quantity information and dynamic frame quantity data, the formula is:
[0113]
[0114] Calculating video stream complexity score , get the video stream complexity information, where, represents the number of differentiated pixels, is the number of dynamic frames, is the total number of frames in the video, , and is the weight coefficient, Score the video stream complexity.
[0115] formula:
[0116]
[0117] Parameter details and how to obtain them:
[0118] , and : The weight coefficient is used to balance the contribution of different parts in the formula and is usually set based on historical data and expert experience.
[0119] : The number of differentiated pixels, calculated by comparing the video data frame by frame, and the pixel difference between each frame is obtained by comparing the pixel values of the current frame and the previous frame. The comparison of each pixel can be achieved by calculating the difference in its RGB value.
[0120] : The number of dynamic frames, which refers to the number of frames in a video sequence that are classified as dynamic due to large pixel changes. Dynamic frames are usually defined by setting a pixel change threshold, and the number of pixel changes in a frame that exceeds this threshold will be counted as a dynamic frame.
[0121] : The total number of frames in the video. This is a basic attribute of the video and is usually obtained directly from the video metadata. It indicates the total number of frames in the video.
[0122] Calculation example:
[0123] Assume the parameters are set as follows:
[0124] , , , Assuming that a total of 5000 differential pixels are detected in consecutive frame comparisons, 300 frames in the video are judged as dynamic frames. The total number of video frames is 1000.
[0125] calculate :
[0126]
[0127] The calculation process uses natural logarithm, and the obtained The value is about 85.07, which reflects the complexity of the visual content changes in the video data. A higher value indicates that the proportion of dynamic scenes in the video is large, or the changes between frames are frequent, which means that a higher bit rate or more complex encoding strategy is required when encoding the video to maintain quality.
[0128] See also Figure 5 , the steps for obtaining the adjusted video stream information are:
[0129] Based on the predicted network quality data and video stream complexity information, the screen size and screen pixel density data of the video streaming terminal are obtained to obtain the stream association data;
[0130] Based on the stream association data, through the formula:
[0131]
[0132] Calculate the adjusted compression bitrate , and by the formula:
[0133]
[0134] Calculate the adjusted video frame rate , get the adjusted video stream information, where, Score the complexity of the video stream, Score the network quality for future time periods, , , and is the adjustment factor, Indicates the adjusted compression bit rate, is the basic compression bit rate, is the adjusted video frame rate, is the basic video frame rate, is the network bandwidth.
[0135] formula:
[0136]
[0137] and
[0138]
[0139] Parameter details and how to obtain them:
[0140] , , and : Adjustment coefficients, based on experiments or previous data analysis, are used to balance the impact of different factors on video compression bitrate and frame rate. Each coefficient is carefully selected according to the needs of specific network and device conditions to ensure the best video streaming experience.
[0141] : Basic compression bitrate, the initial setting bitrate of the video, usually set by the video content provider or platform standard, reflecting the video compression level before any adjustment of network conditions.
[0142] : Network bandwidth, obtained through network monitoring tools.
[0143] : The network quality score for the future period, calculated by the previous steps.
[0144] : The video stream complexity score, calculated by the previous step.
[0145] : Basic video frame rate is the initial frame rate of video playback or recording, which is usually determined by the video production standard or source file properties.
[0146] Calculation example:
[0147] , , , , kbps (basic compression bit rate), Mbps (network bandwidth), (Network Quality Rating), (Video Complexity), fps (base frame rate).
[0148] Compression bit rate adjustment:
[0149]
[0150] Frame rate adjustment:
[0151]
[0152] The calculation process uses natural logarithm, and the compression bit rate calculated is About 1117.47kbps, which is higher than the basic bit rate to adapt to good network conditions and higher video complexity. It is 44.23fps, which is higher than the base frame rate, indicating that network conditions allow for the playback of higher frame rate videos to provide a smoother visual experience.
[0153] See also Figure 6 , the method to evaluate the stability of video stream is:
[0154] Based on the adjusted video streaming information, key performance indicators of the video stream are monitored in real time, and the key performance indicators include the number of buffering events, frame loss rate and delay time;
[0155] Based on key performance indicators, through the formula:
[0156]
[0157] Calculating the stability score of a video stream ,in, Indicates the stability score of the video stream. is the frequency of buffering events, is the frame loss rate, is the average delay time, , and is the adjustment factor, is the attenuation coefficient, is the base of natural logarithms;
[0158] Stability score based on video streaming ,according to The value is used to evaluate the stability of the video stream and obtain the video stability evaluation result.
[0159] formula:
[0160]
[0161] Parameter details and how to obtain them:
[0162] , and :The coefficient is quantitatively determined based on historical data and performance analysis, and is used to adjust the relative importance of video performance indicators in the stability score. It is adjusted according to different network environments and video content characteristics through experiments and optimization algorithms.
[0163] : The attenuation coefficient of the buffering event, which is used to adjust the impact of the buffering event frequency on video stability. This parameter is usually optimized through network performance testing and user experience feedback to reduce the negative impact of frequent buffering on user experience.
[0164] : Buffering event frequency, usually obtained directly from the video stream monitoring system, indicating the proportion of video playback interruptions due to buffering within a certain period of time.
[0165] : Frame loss rate, calculated by the ratio of the number of frames that cannot be displayed during video decoding to the total number of frames. This is obtained directly from the statistics interface of the video player or decoder.
[0166] : Average latency, obtained from network monitoring tools, reflects the transmission delay of video data from the source to the end user.
[0167] Calculation example:
[0168] Assume the parameters are set as follows:
[0169] , , , , (The probability of buffering event occurring is 2%). (frame loss rate is 1%), ms (average latency).
[0170] Calculate the buffer impact term:
[0171]
[0172] Calculate the frame loss impact:
[0173]
[0174] Calculate the delay impact term, using natural logarithm:
[0175]
[0176] calculate :
[0177]
[0178] What you get The value is about 2.28, which reflects the overall stability of the video stream. The values show that despite slight buffering and frame loss, the negative impact of average delay on video streaming stability has been effectively suppressed, and the video streaming is generally stable.
[0179] See also Figure 7 ,The steps to obtain real-time optimized streaming information are:
[0180] Stability score based on video streaming , set the stability threshold, through the formula:
[0181]
[0182] Calculate the optimized compression bitrate , and by the formula:
[0183]
[0184] Calculate optimized video frame rate ,in, is the current compression bit rate, is the current video frame rate, Rate the video stream stability, is the stability threshold, and is the adjustment factor, and is the power coefficient;
[0185] Based on the optimized compression bit rate and optimized video frame rate , adjust the compression bit rate and frame rate during video streaming to obtain real-time optimized streaming information.
[0186] formula:
[0187]
[0188] and
[0189]
[0190] Parameter details and how to obtain them:
[0191] and : The current compression bitrate and frame rate are directly obtained from the current settings of the video encoder or streaming server, representing the existing compression and frame rate levels of the live video stream.
[0192] : The stability score of the video stream, calculated by the previous step.
[0193] : Stability threshold, determined based on past video stream data analysis and user feedback, represents the lower limit of video stream performance and is used to define when the compression rate and frame rate need to be adjusted to maintain stream quality and continuity.
[0194] and : The adjustment factor is determined based on experiments and historical performance analysis. Its purpose is to adjust the sensitivity and adjustment range of the compression bitrate and frame rate when the stability score changes relative to the threshold.
[0195] and : The power coefficient is used to control the nonlinear adjustment ability of the formula to adapt to a wide range of stability changes. It is determined by the optimization algorithm to ensure that the compression bitrate and frame rate can change significantly when the stability score is far away from the threshold, and change more slowly when it is close to the threshold.
[0196] Calculation example:
[0197] Assume that the current parameter settings are as follows:
[0198] kbps, fps, , , , , , .
[0199] Compression bit rate calculation:
[0200]
[0201] Frame rate calculation:
[0202]
[0203] The calculation results show that the compression bitrate has increased slightly from 1500kbps to about 1507.04kbps, and the frame rate has also increased slightly from 30fps to about 30.045fps. The adjustment reflects the better stability of the video stream, allowing a slight increase in compression and frame rate to improve video quality while maintaining smoothness. This subtle adjustment helps optimize the performance of the video stream without sacrificing the user experience.
[0204] The method for adaptively processing a data video stream based on a stream processing technology comprises the following steps:
[0205] S1: Based on the video streaming environment, monitor and record the network status parameters during the streaming process, evaluate the streaming network quality, analyze and predict the network quality in the future period through the long short-term memory network and recurrent neural network, and obtain the predicted network quality data;
[0206] S2: Based on the video data to be streamed, the pixel difference between the frames is calculated, and the complexity of the video is evaluated by combining the static and dynamic changes of the picture to obtain the complexity information of the video stream;
[0207] S3: Based on the predicted network quality data and the video stream complexity information, combined with the screen size and pixel density of the video streaming terminal, the video compression bit rate and frame rate are adjusted to obtain adjusted video streaming information;
[0208] S4: Based on the adjusted video streaming information, monitor the video streaming process, analyze the video performance, evaluate the stability of the video stream, and obtain a video stability evaluation result;
[0209] S5: Based on the video stability evaluation result, the compression bit rate and frame rate of the video stream are optimized to obtain real-time optimized stream information.
[0210] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.
Claims
1. A data video stream adaptive processing system based on stream processing technology, characterized in that: The system comprises: The network quality prediction module is based on the video streaming environment, monitors the network status during the video streaming process, evaluates the streaming network quality, and analyzes the streaming network change trend based on the streaming network quality data over a period of time through the long short-term memory network combined with the recurrent neural network, predicts the streaming network quality in the future period, and obtains the predicted network quality data; The video complexity evaluation module evaluates the complexity of the video stream based on the video data to be streamed, by analyzing the number of differentiated pixels between video frames and combining the changes in the static and dynamic states of the video screen, and obtains the video stream complexity information; The compression bit rate adjustment module adjusts the compression bit rate and frame rate of the video based on the predicted network quality data and the video stream complexity information, combined with the screen size and screen pixel density of the video streaming terminal, to match the network and device conditions, and obtain the adjusted video streaming information; The real-time optimization and adjustment module monitors the video streaming process based on the adjusted video streaming information, evaluates the stability of the video stream by analyzing the video performance, and optimizes the video compression bit rate and frame rate according to the stability of the video stream to obtain real-time optimized streaming information; The method for evaluating the stability of the video stream is: Based on the adjusted video stream information, monitor key performance indicators of the video stream in real time, where the key performance indicators include the number of buffering events, frame loss rate, and delay time; Based on the key performance indicators, the formula: Calculating the stability score of a video stream ,in, Indicates the stability score of the video stream. is the frequency of buffering events, is the frame loss rate, is the average delay time, , and is the adjustment factor, is the attenuation coefficient, is the base of natural logarithms; Stability score based on the video stream ,according to The value is used to evaluate the stability of the video stream and obtain the video stability evaluation result; The steps for obtaining the real-time optimized streaming information are as follows: Stability score based on the video stream , set the stability threshold, through the formula: Calculate the optimized compression bitrate , and by the formula: Calculate optimized video frame rate ,in, is the current compression bit rate, is the current video frame rate, Rate the video stream stability, is the stability threshold, and is the adjustment factor, and is the power coefficient; Based on the optimized compression bit rate and optimized video frame rate , adjust the compression bit rate and frame rate during video streaming to obtain real-time optimized streaming information.
2. The data video stream adaptive processing system based on stream processing technology according to claim 1 is characterized in that: The method for evaluating the streaming network quality is: Based on the video streaming environment, monitor the network status during the video streaming process and collect network status data during the video streaming process. The network status data includes bandwidth, delay and packet loss rate. Based on the network status data, by the formula: Calculating network quality score ,in, is the network bandwidth, For network delay, is the packet loss rate, , and is the weight coefficient, Rate the network quality, is the base of natural logarithms; Based on the network quality score ,according to The value is used to evaluate the network quality and obtain the streaming network quality information.
3. The data video stream adaptive processing system based on stream processing technology according to claim 2 is characterized in that: The steps for obtaining the predicted network quality data are: Based on the streaming network quality information, collecting streaming network quality data within a period of time to obtain a network quality time data set; Based on the network quality time dataset, the data is processed by long short-term memory units to obtain the network quality data at each time point. The value is analyzed using the formula: Calculate network quality score for future time periods , get the predicted network quality data, where, Rate the current network quality. is the average of historical network quality scores. is the standard deviation of historical network quality scores, is the time difference factor, Score the network quality for future time periods, , and is the weight coefficient.
4. The data video stream adaptive processing system based on stream processing technology according to claim 1, characterized in that: The steps for obtaining the video stream complexity information are as follows: Based on the video data to be streamed, by comparing the video data frame by frame, identifying the change of pixel values between frames, calculating the number of differential pixels between video frames, and obtaining differential pixel number information; Based on the differentiated pixel quantity information, static and dynamic state changes in the video screen are analyzed, and the number of dynamic frames in the video is counted. The dynamic frame is judged by comparing the differentiated pixel quantity with a preset threshold to obtain dynamic frame quantity data; Based on the differentiated pixel quantity information and dynamic frame quantity data, the formula is: Calculating video stream complexity score , get the video stream complexity information, where, represents the number of differentiated pixels, is the number of dynamic frames, is the total number of frames in the video, , and is the weight coefficient, Score the video stream complexity.
5. The data video stream adaptive processing system based on stream processing technology according to claim 1 is characterized in that: The steps for obtaining the adjusted video stream information are as follows: Based on the predicted network quality data and video stream complexity information, the screen size and screen pixel density data of the video streaming terminal are obtained to obtain the stream association data; Based on the stream association data, by formula: Calculate the adjusted compression bitrate , and by the formula: Calculate the adjusted video frame rate , get the adjusted video stream information, where, Score the complexity of the video stream, Score the network quality for future time periods, , , and is the adjustment factor, Indicates the adjusted compression bit rate, is the basic compression bit rate, is the adjusted video frame rate, is the basic video frame rate, is the network bandwidth.
6. A data video stream adaptive processing method based on stream processing technology, characterized in that: The data video stream adaptive processing system based on the streaming processing technology according to any one of claims 1 to 5 comprises the following steps: Based on the video streaming environment, monitor and record the network status parameters during the streaming process, evaluate the streaming network quality, analyze and predict the network quality in the future period through the long short-term memory network and recurrent neural network, and obtain the predicted network quality data; Based on the video data to be streamed, the pixel difference between the frames is calculated, and the complexity of the video is evaluated by combining the static and dynamic changes of the picture to obtain the complexity information of the video stream; Based on the predicted network quality data and the video stream complexity information, combined with the screen size and pixel density of the video streaming terminal, the video compression bit rate and frame rate are adjusted to obtain adjusted video streaming information; Based on the adjusted video streaming information, monitor the video streaming process, analyze the video performance, evaluate the stability of the video stream, and obtain a video stability evaluation result; Based on the video stability evaluation result, the compression bit rate and frame rate of the video stream are optimized to obtain real-time optimized stream information.
