Video stream quality analysis and control system and method based on AI drive

Abstract

The invention relates to a real-time video stream quality analysis and adaptive control method based on AI driving, and belongs to the technical field of artificial intelligence and video stream transmission. The method comprises the following steps: receiving a real-time video stream from a server through client equipment, and the real-time video stream comprises a dynamic adaptive streaming media and a live broadcast stream; analyzing the video stream and network conditions according to the video stream in combination with a network monitoring tool to extract video features; inputting the extracted features into a machine model based on a neural network to generate quality evaluation including network conditions, video parameters and annotations; determining whether to switch to video representation forms with different bit rates or not by analyzing a quality evaluation result; and when the switching is determined, a request for updating the playing representation form is sent to the server, so that the artificial intelligence technology is realized to optimize the video quality, reduce buffering and improve the user experience, and the method is suitable for a real-time streaming media scene.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and video stream transmission technology, and specifically relates to a system and method for video stream quality analysis and control based on AI. Background Art

[0002] Video streaming has become the primary form of media consumption worldwide, and users have increasingly high expectations for high-quality, uninterrupted playback experience. Traditional adaptive streaming technologies, such as adaptive streaming, select appropriate video representations through heuristic rules based on network bandwidth and buffer status. However, these methods usually cannot fully consider the complexity of video content or the quality differences perceived by users. For example, fixed bitrate steps may cause compression artifacts in complex scenes or cause buffering problems when network conditions fluctuate. Artificial intelligence, especially machine learning technology, provides new possibilities for video stream quality analysis. By analyzing multiple characteristics of video streams and network conditions, artificial intelligence models can predict user-perceived quality or potential playback problems, thereby achieving more intelligent streaming parameter adjustment. The present invention uses these technologies to propose a method for real-time analysis of video stream quality and adaptive control on client devices to improve user experience. Summary of the Invention

[0003] To solve the above problems in the prior art, the present invention provides a system and method for video stream quality analysis and control based on AI. The purpose of the present invention can be achieved through the following technical solutions: S1: According to the video acquisition module and network quality monitoring module integrated in the client, basic transmission data of the video stream is collected in real time, and the basic transmission data is integrated into a video stream quality data set through data standardization mapping; S2: Based on the video stream quality dataset, an intelligent analysis model trained with a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism is called to predict the quality fluctuations of the video stream under the current network state and generate a video quality result; S3: Based on the quality prediction results and the current network bandwidth status, the encoding of the client video stream is optimized through artificial intelligence to generate an optimal configuration plan for the current video stream; S4: The feedback monitoring module analyzes the video playback effect of the adjusted optimal transmission video stream configuration scheme, and evaluates the video quality returned by the user end based on the evaluation indicators of the network layer and the video coding layer; if it is detected that the video quality does not meet the playback expectations under the current network status, the video stream configuration scheme is automatically iteratively adjusted.

[0004] Specifically, the video acquisition module includes a pre-encoding frame capture unit and a synchronization timestamp marking module. The pre-encoding frame capture unit stores the captured frames in a memory buffer according to a low-latency frame buffer mechanism before the encoder processes them, thereby capturing the original video frame stream; the synchronization timestamp marking module adds network synchronization timestamp information to each frame of video to ensure that the network transmission and the data timing of the receiving end are aligned.

[0005] Specifically, the standardized mapping maps the acquisition parameters to a standardized interval to remove platform differences to obtain the video stream quality dataset.

[0006] Specifically, the artificial intelligence optimized encoding control strategy includes: an adaptive bitrate optimization strategy based on network bandwidth and delay, and an adaptive resolution optimization strategy based on user experience. The adaptive bitrate optimization strategy based on network bandwidth and delay ensures that the video stream remains stable under changing network bandwidth by intelligently adjusting the bitrate; the adaptive bitrate optimization strategy based on network bandwidth and delay adjusts the resolution by analyzing the current network situation to reduce video buffering and loading time.

[0007] Specifically, the network quality monitoring module includes an embedded real-time bandwidth detection unit and a multi-threaded delay analyzer. The embedded real-time bandwidth detection unit collects the amount of video transmission data per unit time through the network traffic monitoring module to calculate the upstream and downstream bandwidth values ​​of the current network; the multi-threaded delay analyzer independently analyzes the sending delay, transmission delay, buffering delay and terminal rendering delay through a thread-level concurrent architecture to assist in determining the type of video playback abnormality.

[0008] Specifically, the feedback analysis of the video playback effect includes: end-to-end delay, instantaneous frame rate jitter, buffer ratio and quality switching frequency, with the changing trend within the playback time period as feedback, and the feedback monitoring module is used to detect whether the playback quality meets expectations under the current network status.

[0009] Specifically, the intelligent analysis model is a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism. It takes as input a sequence of video stream quality datasets constructed in chronological order. It obtains the contextual features of the quality status at each moment through bidirectional modeling and uses it to perform time series modeling on the key features of quality changes at each moment in the video quality dataset. The specific formula is: , , Among them, score (h t ) is the attention scoring function, α tis the attention weight used to evaluate the importance of the hidden state at each time step, v is the learnable weight vector, W is the learnable weight matrix, b is the learnable bias vector, tanh is the activation function, and c is the context vector.

[0010] Specifically, the attention mechanism simulates human visual focusing behavior to dynamically allocate weights in multi-dimensional input features, paying more attention to temporal segments that have a great impact on video quality fluctuations.

[0011] Specifically, the encoding control strategy includes dynamic resolution switching and frame rate adjustment; the encoding control strategy generates an encoding configuration scheme based on the current network status and user device performance through an artificial intelligence optimization algorithm.

[0012] Specifically, the automatic iterative adjustment of the video stream configuration scheme is implemented through an artificial intelligence-driven adjustment engine, which dynamically optimizes the encoding control strategy of the video stream based on feedback data from the network layer and video coding layer and the performance of the user device.

[0013] The beneficial effects of the present invention are: This invention provides an AI-driven real-time video stream quality analysis and adaptive control method that overcomes the problems of unstable video quality, lag in adjustment, low resource utilization, and inconsistent user experience in existing video transmission systems under complex and dynamic network environments. By introducing a deep learning model and reinforcement learning strategy, this method can comprehensively monitor and model the multidimensional quality parameters of video streams during transmission, predict the risk of quality fluctuations in real time, and thus achieve automatic optimization of encoding parameters and adaptive adjustment of transmission strategies. The intelligent analysis model of the present invention integrates a multi-layer bidirectional long-short-term memory neural network with an attention mechanism, focusing on the features and time nodes in historical data that are most sensitive to quality prediction, effectively improving the model's response speed and prediction accuracy to sudden changes in network status. At the same time, the AI ​​optimization engine based on reinforcement learning can dynamically adjust the encoding control strategy of the video stream, driven by changes in network bandwidth and user-end feedback, to achieve quality adaptation and resource allocation optimization under extreme network conditions such as low bandwidth and high fluctuations. In addition, the embedded bandwidth detection and multi-threaded delay analysis mechanism proposed in the present invention enables the system to monitor link quality with millisecond-level accuracy. Combined with the synchronized timestamp and feedback mechanism, it effectively improves the consistency and robustness of the overall video experience. In summary, the method of the present invention has technical advantages such as strong real-time performance, high prediction accuracy, strong dynamic optimization capability of control strategy, and excellent network adaptability. It is widely applicable to scenarios with high requirements on transmission quality, such as video conferencing, distance learning, security monitoring, and online entertainment, and has significant practical value and industrial promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0015] Figure 1 The figure is a flowchart of a system and method for AI-driven video stream quality analysis and control according to the present invention. DETAILED DESCRIPTION

[0016] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0017] See also Figure 1 , a system and method for AI-driven video stream quality analysis and control: S1: According to the video acquisition module and network quality monitoring module integrated in the client, basic transmission data of the video stream is collected in real time, and the basic transmission data is integrated into a video stream quality data set through data standardization mapping; S2: Based on the video stream quality dataset, an intelligent analysis model trained with a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism is called to predict the quality fluctuations of the video stream under the current network state and generate a video quality result; S3: Based on the quality prediction results and the current network bandwidth status, the encoding of the client video stream is optimized through artificial intelligence to generate an optimal configuration plan for the current video stream; S4: The feedback monitoring module analyzes the video playback effect of the adjusted optimal transmission video stream configuration scheme, and evaluates the video quality returned by the user end based on the evaluation indicators of the network layer and the video coding layer; if it is detected that the video quality does not meet the playback expectations under the current network status, the video stream configuration scheme is automatically iteratively adjusted.

[0018] In this embodiment, the method of the present invention is applied to a real-time video stream quality monitoring and optimization system deployed at an operator's edge node, suitable for industrial monitoring and distance education scenarios with high concurrency and low latency requirements. The system structure includes an edge acquisition module, a real-time analysis model, an adaptive controller, and a user feedback collector. The embedded video acquisition module deployed on the edge server receives video stream data from multiple cameras in real time, and at the same time enables a real-time bandwidth detection unit to obtain the effective throughput, jitter fluctuation, and delay distribution of the local link. The acquisition module constructs the video transmission status and basic parameters of each time period into a real-time video stream quality dataset, and the edge node simultaneously runs a multi-layer Bi-LSTM analysis model with an integrated attention mechanism. The model input is a constructed quality sequence, which can predict the potential quality fluctuation trend in the next few frame time windows within milliseconds, and output the quality risk level and recommended adjustment direction. For example, if network congestion is predicted to occur within the next 200ms, the model will recommend reducing the peak bitrate of the current video and increasing redundant cache frames. The system's built-in AI optimization engine receives the model output and adjusts the encoding control strategy in real time based on the device type, network bandwidth, and historical playback performance. For example, it dynamically adjusts the GOP length, reduces the quantization parameter QP, and uses an H.265 encoding template with a higher compression rate. The optimized video stream playback effect is fed back by the user client and input into the feedback monitoring module together with the network layer indicators recorded at the edge for multi-dimensional evaluation. If the optimization effect is not good, the control strategy will be automatically iterated, and the reinforcement learning mechanism will converge to better parameters in the current network scenario. This ensures high continuity, low latency, and good clarity of video services in edge network environments with unstable bandwidth, improving the user experience in remote scenarios.

[0019] Specifically, the video acquisition module includes a pre-encoding frame capture unit and a synchronization timestamp marking module. The pre-encoding frame capture unit stores the captured frames in a memory buffer according to a low-latency frame buffer mechanism before the encoder processes them, thereby capturing the original video frame stream; the synchronization timestamp marking module adds network synchronization timestamp information to each frame of video to ensure that the network transmission and the data timing of the receiving end are aligned.

[0020] Specifically, the standardized mapping maps the acquisition parameters to a standardized interval to remove platform differences to obtain the video stream quality dataset.

[0021] Specifically, the artificial intelligence optimized encoding control strategy includes: an adaptive bitrate optimization strategy based on network bandwidth and delay, and an adaptive resolution optimization strategy based on user experience. The adaptive bitrate optimization strategy based on network bandwidth and delay ensures that the video stream remains stable under changing network bandwidth by intelligently adjusting the bitrate; the adaptive bitrate optimization strategy based on network bandwidth and delay adjusts the resolution by analyzing the current network situation to reduce video buffering and loading time.

[0022] Specifically, the network quality monitoring module includes an embedded real-time bandwidth detection unit and a multi-threaded delay analyzer. The embedded real-time bandwidth detection unit collects the amount of video transmission data per unit time through the network traffic monitoring module to calculate the upstream and downstream bandwidth values ​​of the current network; the multi-threaded delay analyzer independently analyzes the sending delay, transmission delay, buffering delay and terminal rendering delay through a thread-level concurrent architecture to assist in determining the type of video playback abnormality.

[0023] In this embodiment, the video stream playback on the mobile terminal is taken as the object, and the terminal-side AI model is combined with the cloud parameter recommendation to achieve lightweight quality prediction and adaptive optimization control. Application scenarios include short video applications, mobile live broadcasts, and video conferencing. The client has a built-in lightweight video acquisition and monitoring module. The collected content includes the playback video clarity, buffering times, average bit rate, instantaneous frame loss, device CPU / GPU usage, network standard, current signal strength, etc. The collected information is constructed into a video stream quality data sequence after time alignment processing. A cropped and optimized two-layer Bi-LSTM attention model is deployed locally. The model is only 2.3MB in size and can run in real time in low-power mode on the mobile terminal. The model predicts playback risks within the next 30 seconds based on real-time sampling data, and generates control suggestions based on the user's past usage habits. The control strategy is adjusted by calling the player's core parameter interface through the system API, reducing the target resolution, dynamically switching the code control mode, starting the mobile adaptive buffering logic, and delaying the loading priority of high-bitrate video clips. In particular, the attention mechanism is used to enhance the model's ability to identify key fluctuation points. It can reduce the video resolution in advance before sudden signal weakening occurs, thereby avoiding playback interruptions. It is suitable for mobile video applications where users' network conditions are highly volatile, are sensitive to power consumption, and have high requirements for user experience.

[0024] Specifically, the feedback analysis of the video playback effect includes: end-to-end delay, instantaneous frame rate jitter, buffer ratio and quality switching frequency, with the changing trend within the playback time period as feedback, and the feedback monitoring module is used to detect whether the playback quality meets expectations under the current network status.

[0025] Specifically, the intelligent analysis model is a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism. It takes as input a sequence of video stream quality datasets constructed in chronological order. It obtains the contextual features of the quality status at each moment through bidirectional modeling and uses it to perform time series modeling on the key features of quality changes at each moment in the video quality dataset. The specific formula is: , , Among them, score (h t ) is the attention scoring function, α t is the attention weight used to evaluate the importance of the hidden state at each time step, v is the learnable weight vector, W is the learnable weight matrix, b is the learnable bias vector, tanh is the activation function, and c is the context vector.

[0026] Specifically, the attention mechanism simulates human visual focusing behavior to dynamically allocate weights in multi-dimensional input features, paying more attention to temporal segments that have a great impact on video quality fluctuations.

[0027] Specifically, the encoding control strategy includes dynamic resolution switching and frame rate adjustment; the encoding control strategy generates an encoding configuration scheme based on the current network status and user device performance through an artificial intelligence optimization algorithm.

[0028] In this embodiment, video service platforms with massive user bases, such as online video websites, live streaming platforms, and social media systems, are used as service points. The key goal is to use AI to uniformly schedule the streaming quality of different users to optimize the overall system load and maximize user experience. When each user accesses the platform, a virtual session control unit is activated, which includes a user bandwidth detector, a playback behavior monitor, and a video status recorder. The platform synchronously inputs data uploaded by multiple users into the cloud-based AI prediction module, a variant neural network model consisting of a multi-user time-series fusion encoder and an attention-weighted decoder. It can aggregate and model the playback quality trends of different users in different network environments, and group and model them based on metadata such as geographic location, network operator, and content type. The platform generates a hierarchical control plan based on the prediction results and issues different video encoding configurations to different user groups: a redundant frame enhancement algorithm is uniformly used for high-latency groups, a stronger compression rate and ABR code control strategy is applied to groups with large bandwidth fluctuations, and the platform is allowed to open higher bitrate streaming channels for VIP user groups. The system also introduces a user satisfaction prediction model, combining the user's historical like rate and viewing time with AI evaluation results as feedback signals to guide the real-time update of the video stream configuration strategy. This embodiment can significantly reduce the platform server bandwidth peak, improve the consistency of video playback experience, and achieve refined adaptive strategy scheduling on a large scale through parallel AI control.

[0029] Specifically, the automatic iterative adjustment of the video stream configuration scheme is implemented through an artificial intelligence-driven adjustment engine, which dynamically optimizes the encoding control strategy of the video stream based on feedback data from the network layer and video coding layer and the performance of the user device.

[0030] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A system and method for AI-driven video stream quality analysis and control, characterized in that: include: S1: According to the video acquisition module and network quality monitoring module integrated in the client, basic transmission data of the video stream is collected in real time, and the basic transmission data is integrated into a video stream quality data set through data standardization mapping; S2: Based on the video stream quality dataset, an intelligent analysis model trained with a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism is called to predict the quality fluctuations of the video stream under the current network state and generate a video quality result; S3: Based on the quality prediction results and the current network bandwidth status, the encoding of the client video stream is optimized through artificial intelligence to generate an optimal configuration plan for the current video stream; S4: The feedback monitoring module analyzes the video playback effect of the adjusted optimal transmission video stream configuration scheme, and evaluates the video quality returned by the user end based on the evaluation indicators of the network layer and the video coding layer; if it is detected that the video quality does not meet the playback expectations under the current network status, the video stream configuration scheme is automatically iteratively adjusted.

2. The system according to claim 1, wherein: The video acquisition module includes a pre-encoding frame capture unit and a synchronization timestamp marking module. The pre-encoding frame capture unit stores the captured frames in a memory buffer according to a low-latency frame buffer mechanism before the encoder processes them, thereby capturing the original video frame stream; the synchronization timestamp marking module ensures that the network transmission and the data timing of the receiving end are aligned by adding network synchronization timestamp information to each frame of video.

3. The system according to claim 1, wherein: The standardized mapping maps the acquisition parameters to the standardized interval to remove platform differences to obtain the video stream quality dataset.

4. The system according to claim 1, wherein: The artificial intelligence optimized encoding control strategy includes: an adaptive bitrate optimization strategy based on network bandwidth and delay, and an adaptive resolution optimization strategy based on user experience. The adaptive bitrate optimization strategy based on network bandwidth and delay ensures that the video stream remains stable under changing network bandwidth by intelligently adjusting the bitrate; the adaptive bitrate optimization strategy based on network bandwidth and delay adjusts the resolution by analyzing the current network situation, reducing video buffering and loading time.

5. The system according to claim 1, wherein: The network quality monitoring module includes an embedded real-time bandwidth detection unit and a multi-threaded delay analyzer. The embedded real-time bandwidth detection unit collects the amount of video transmission data per unit time through the network traffic monitoring module to calculate the upstream and downstream bandwidth values ​​of the current network; the multi-threaded delay analyzer independently analyzes the sending delay, transmission delay, buffering delay and terminal rendering delay through a thread-level concurrent architecture to assist in determining the type of video playback anomaly.

6. The system according to claim 3, wherein: The feedback analysis of the video playback effect includes: end-to-end delay, instantaneous frame rate jitter, buffer ratio and quality switching frequency. The changing trend within the playback time period is used as feedback, and the feedback monitoring module is used to detect whether the playback quality meets expectations under the current network status.

7. The method according to claim 4, characterized in that The intelligent analysis model is a multi-layer bidirectional long short-term memory neural network with an integrated attention mechanism. It takes as input a sequence of video stream quality datasets constructed in chronological order. Through bidirectional modeling, it obtains the contextual features of the quality status at each moment and uses them to perform time series modeling on the key features of quality changes at each moment in the video quality dataset.

8. The system according to claim 7, characterized in that The attention mechanism simulates human visual focusing behavior to dynamically allocate weights among multi-dimensional input features, paying more attention to temporal segments that have a significant impact on video quality fluctuations.

9. The system according to claim 1, wherein: The coding control strategy includes dynamic resolution switching and frame rate adjustment; the coding control strategy generates a coding configuration scheme based on the current network status and user equipment performance through an artificial intelligence optimization algorithm.

10. The system according to claim 4, wherein: The automatic iterative adjustment of the video stream configuration scheme is implemented through an artificial intelligence-driven adjustment engine, which dynamically optimizes the encoding control strategy of the video stream based on feedback data from the network layer and video coding layer and the performance of the user device.

Images