Operation management system based on data interaction

Abstract

The application provides an operation management system based on data interaction, relates to the technical field of data processing, and comprises an extraction module, a prediction module, a detection module, a strategy arrangement module and a processing execution module. The extraction module extracts a multi-dimensional feature vector of a video frame; the prediction module outputs a processing time consumption prediction distribution by using a generative adversarial network; the detection module identifies time consumption abnormal frames in real time; the strategy arrangement module dynamically selects a replacement processing strategy based on abnormal conditions and system resources and generates a task rewriting instruction; and the processing execution module realizes smooth switching of the strategy. The system improves the prediction accuracy, resource optimization and stability of the video processing task, effectively avoids processing delay fluctuation and visual quality decline, and significantly improves operation management efficiency and user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an operation management system based on data interaction. BACKGROUND

[0002] With the rapid development of cloud computing technology and video analysis applications, large-scale cloud video processing platforms have gradually become an important infrastructure in the fields of video analysis, video surveillance, media production, etc. These platforms often need to process a large amount of video data in real time or near real time, such as video enhancement, video recognition, content analysis, and automated editing, etc. In the operation management system based on data interaction, how to accurately manage and optimize video processing tasks in real time has become a key to improving the efficiency and quality of system operation.

[0003] However, in the actual operation management process, the processing time of different video frames may have large fluctuations due to factors such as content complexity, motion intensity, and background changes. Especially when some video frames contain complex scenes, intense motion, or local occlusions, traditional fixed processing strategies may cause a significant increase in processing time, thereby significantly reducing the efficiency and stability of the overall task batch processing, affecting the service quality and user experience of the operation management system based on data interaction. Existing operation management systems usually use static models or fixed processing procedures, which are difficult to dynamically adjust processing strategies according to the specific content of video frames and real-time processing time, resulting in unreasonable resource allocation and large processing delay fluctuations. In addition, the existing processing strategy switching method usually directly switches the new and old strategies instantly, lacks a transition and smooth processing mechanism, and is easy to cause obvious visual quality mutations and processing stability decline.

[0004] Therefore, how to realize the dynamic detection of video frame-level processing time exceptions in the operation management system based on data interaction, and provide adaptive and smooth alternative processing strategies, is one of the important technical problems to be solved in this field. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides an operation management system based on data interaction, comprising:

[0006] An extraction module is configured to obtain each video frame in a target video from a cloud video processing platform and extract a multi-dimensional feature vector corresponding to each video frame; the multi-dimensional feature vector includes spatial gradient entropy, motion vector, and texture complexity;

[0007] A prediction module is configured to generate a processing time prediction distribution of each video frame according to the multi-dimensional feature vector; the processing time prediction distribution includes an expected value and a standard deviation;

[0008] a detection module, configured to acquire an actual processing time consumption of each video frame in real time, and determine whether the each video frame is a processing time abnormal frame based on the actual processing time consumption and a processing time consumption prediction distribution;

[0009] a policy arrangement module, configured to determine a replacement processing policy corresponding to the processing time abnormal frame according to a preset replacement policy mapping table, and generate a task rewriting instruction in response to the operation of determining the processing time abnormal frame;

[0010] a processing execution module, configured to execute the corresponding replacement processing policy in the cloud video processing platform and generate an optimized video frame in response to the task rewriting instruction.

[0011] Optionally, the acquiring each video frame in a target video from the cloud video processing platform and extracting a multi-dimensional feature vector corresponding to the each video frame comprises:

[0012] decoding the target video to acquire a sequence of image frames arranged in time sequence in the target video, and sequentially extracting each image frame as the each video frame;

[0013] performing feature extraction processing on the each video frame to generate the corresponding multi-dimensional feature vector; wherein the feature extraction processing comprises:

[0014] calculating a spatial gradient entropy of each video frame to represent texture change intensity;

[0015] estimating a motion vector based on pixel displacement between adjacent frames;

[0016] performing gray scale distribution analysis on an image region to acquire a texture complexity index.

[0017] Optionally, the prediction module is configured to acquire an actual processing time consumption of each video frame in real time, and the prediction module comprises:

[0018] a generator, configured to receive the multi-dimensional feature vector and output an expected value and a standard deviation representing a processing time consumption prediction distribution of each video frame;

[0019] a discriminator, configured to perform authenticity determination on the expected value and the standard deviation based on the actual processing time consumption corresponding to the multi-dimensional feature vector in a model training stage, and use a determination result to update a model parameter of the generator;

[0020] wherein the generator and the discriminator constitute a generative adversarial network.

[0021] Optionally, the receiving the multi-dimensional feature vector and outputting the expected value and the standard deviation representing the processing time consumption prediction distribution of each video frame comprises:

[0022] Parallel sub-network processing is performed on the spatial gradient entropy, the motion vector and the texture complexity respectively to obtain a first hidden vector, a second hidden vector and a third hidden vector;

[0023] The first hidden vector, the second hidden vector and the third hidden vector are fused to obtain a fused feature vector;

[0024] The fused feature vector is input into a double-head output layer, a first output head generates the expected value, and a second output head generates the standard deviation.

[0025] Optionally, the fusion processing of the first hidden vector, the second hidden vector and the third hidden vector to obtain a fused feature vector comprises:

[0026] Based on the target video frame rate, a previous video frame motion vector adjacent to the current video frame and a next video frame motion vector adjacent to the current video frame are extracted;

[0027] According to a 0.5 frame time interval, the previous video frame motion vector and the next video frame motion vector are linearly interpolated to generate a half-frame interpolation motion vector;

[0028] The half-frame interpolation motion vector is used to replace the time sequence dimension representation in the second hidden vector to obtain a time sequence aligned second hidden vector;

[0029] The first hidden vector, the time sequence aligned second hidden vector and the third hidden vector are spliced in the channel dimension to obtain an initial fused vector;

[0030] Channel attention weighting processing is performed on the initial fused vector to obtain a weighted fused vector;

[0031] Linear transformation is performed on the weighted fused vector to obtain the fused feature vector.

[0032] Optionally, the channel attention weighting processing of the initial fused vector comprises:

[0033] A time-consuming abnormal frame history mark triggered by a motion vector feature corresponding to the current video frame is obtained, and an abnormal frame proportion of time-consuming abnormal frames triggered by the motion vector feature in all processing time-consuming abnormal frames within a set time window is counted;

[0034] Based on the abnormal frame proportion, a saliency suppression factor of a channel corresponding to the second hidden vector is calculated, which is used to dynamically reduce the initial weight of the channel in attention weighting;

[0035] The attention weight of each channel in the initial fused vector is adjusted by the saliency suppression factor to obtain a dynamically corrected channel weight;

[0036] perform weighted processing on the initial fusion vector based on the dynamically corrected channel weight to obtain the weighted fusion vector.

[0037] Optionally, the authenticity determination on the expected value and the standard deviation based on the actual processing time consumption corresponding to the multi-dimensional feature vector, and the use of the determination result to update the model parameter of the generator include:

[0038] Based on the difference between the expected value and the actual processing time consumption, the standard deviation is constructed to construct a negative log-likelihood function, and a prediction distribution authenticity error is calculated.

[0039] Statistically determine the proportion of samples marked as processing time consumption abnormal frames by the detection module in the current training batch of the discriminator, and generate an abnormal frame proportion weight coefficient based on a preset mapping function;

[0040] The prediction distribution authenticity error and the adversarial loss output by the discriminator are weighted and fused to generate a comprehensive adversarial loss;

[0041] The comprehensive adversarial loss is input into the saliency gate back propagation path of the generator, and the gradient of each hidden channel is distributed based on the channel attention weight to update the model parameter of the generator.

[0042] Optionally, the comprehensive adversarial loss generated by weighting and fusing the prediction distribution authenticity error and the adversarial loss output by the discriminator includes:

[0043] Based on the distribution of the offset amplitude of the motion vector in the multi-dimensional feature vector, the coefficient of variation of the motion vector feature in the training batch is extracted;

[0044] The fusion weight of the adversarial loss output by the discriminator is scaled and adjusted according to the coefficient of variation;

[0045] Based on the mean density index of the texture complexity in the multi-dimensional feature vector, an upper limit constraint factor of the fusion weight is generated;

[0046] The fusion weight of the authenticity error is limited according to the upper limit constraint factor;

[0047] Based on the scaled adversarial loss fusion weight and the limited authenticity error fusion weight, the authenticity error and the adversarial loss are weighted to generate a comprehensive adversarial loss.

[0048] Optionally, in response to determining the processing time consumption abnormal frame, according to a preset replacement strategy mapping table, a replacement processing strategy corresponding to the processing time consumption abnormal frame is determined, and a task rewriting instruction is generated.

[0049] extracting abnormal amplitude information, motion vector active channel identifier and current system resource state corresponding to the processing time-consuming abnormal frame;

[0050] retrieving a candidate alternative processing strategy set from a preset mapping table according to the abnormal amplitude information;

[0051] performing score calculation on each strategy in the candidate alternative processing strategy set based on the motion vector active channel identifier and the current system resource state, to obtain a strategy optimal score;

[0052] selecting a target alternative processing strategy with the highest strategy optimal score, and generating a task rewriting instruction according to a processing node definition of the target alternative processing strategy.

[0053] Optionally, the response to the task rewriting instruction includes:

[0054] after receiving the task rewriting instruction, starting a backup processing node and preloading network weights corresponding to the target alternative processing strategy in GPU memory;

[0055] during the weight preloading, adding the abnormal frame to be processed to a transition queue and continuing to generate cache frames using the original processing strategy;

[0056] after the weight preloading is completed, performing difference evaluation on the cache frames and comparison frames generated using the target alternative processing strategy, and switching to the target alternative processing strategy when the difference is lower than a preset threshold;

[0057] between a plurality of video frames before and after the switching, generating transition frames using a pixel-level weighted fusion method, and outputting a merged sequence of the transition frames and the target alternative processing strategy processing frames to generate optimized video frames.

[0058] Compared with the prior art, the application realizes accurate identification and rapid response of time-consuming abnormal frames by predicting the processing time-consuming distribution (expected value and standard deviation) of each video frame in real time in the video processing process, and combining real-time measurement of actual time-consuming for abnormal detection, effectively solving the technical problem that the overall throughput rate is significantly reduced due to the dramatic increase in processing time of individual frames in the traditional video batch processing process.

[0059] Secondly, the system realizes frame-level time-consuming prediction based on a clearly defined multi-dimensional feature (spatial gradient entropy, motion vector and texture complexity), improves the accuracy and robustness of the prediction, and effectively solves the problem that traditional single feature or simple combination of features cannot accurately express the characteristics of complex video scenes;

[0060] In addition, by introducing the policy arrangement module and the processing execution module, the automatic switching and dynamic rewriting mechanism of the policy for the time-consuming abnormal frame are realized, the adaptability and flexibility of the cloud video processing pipeline are significantly improved, the defects of unreasonable resource allocation and large task execution delay fluctuation under the traditional fixed processing policy are effectively avoided, and finally the overall performance and user experience of the system are improved. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 A schematic diagram of an operation management system based on data interaction is provided for the embodiments of the present application;

[0062] Figure 2 A flowchart of a method of performing channel attention weighting processing on the initial fusion vector is provided for the embodiments of the present application;

[0063] Figure 3 A flowchart of a method of using the determination result to update the model parameters of the generator is provided for the embodiments of the present application;

[0064] Figure 4 A flowchart of a method of generating a comprehensive adversarial loss is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0066] Referring to Figure 1 A schematic diagram of an operation management system based on data interaction is provided for the embodiments of the present application, the system includes an extraction module, a prediction module, a detection module, a policy arrangement module and a processing execution module, wherein:

[0067] The extraction module is configured to obtain each video frame in a target video from a cloud video processing platform, and extract a multi-dimensional feature vector corresponding to each video frame; the multi-dimensional feature vector includes a spatial gradient entropy, a motion vector and a texture complexity;

[0068] The prediction module is configured to generate a processing time consumption prediction distribution of each video frame according to the multi-dimensional feature vector; the processing time consumption prediction distribution includes an expected value and a standard deviation;

[0069] The detection module is configured to obtain an actual processing time consumption of each video frame in real time, and determine whether each video frame is a processing time consumption abnormal frame based on the actual processing time consumption and the processing time consumption prediction distribution;

[0070] The policy arrangement module is configured to determine a replacement processing policy corresponding to the processing time-consuming abnormal frame according to a preset replacement policy mapping table and generate a task rewriting instruction in response to the determination of the operation of processing the processing time-consuming abnormal frame.

[0071] The processing execution module is configured to execute the corresponding replacement processing policy in the cloud video processing platform and generate an optimized video frame in response to the task rewriting instruction.

[0072] The operation management system based on data interaction according to the embodiments can be applied to a video cloud processing scenario and is particularly suitable for performance management and scheduling optimization of real-time or quasi-real-time video analysis and processing tasks.

[0073] For example, the above modules can be deployed in a cloud containerized running environment, the containerized environment is uniformly managed by using a Kubernetes platform, and data interaction and synchronization between the modules are achieved through a Kafka data bus to ensure real-time performance and high throughput.

[0074] For the extraction module described above:

[0075] In a specific implementation, the cloud video processing platform is connected through a Kafka or HTTP interface to obtain compressed video stream data (such as H.264 or H.265 format) of a target video, and a real-time decoding operation is performed on the obtained video stream data based on an FFmpeg library to convert the compressed video stream into a sequence of original image frames, wherein the sequence of image frames is arranged in order according to a video original frame rate, for example, 30 frames per second or 60 frames per second, and the specific frame rate can be flexibly set according to an actual application scenario.

[0076] Subsequently, for each image frame, a feature extraction process is performed through a Python API interface of an OpenCV library, specifically including:

[0077] A Sobel operator convolution is performed based on a gray image of each image frame to obtain a spatial gradient field corresponding to the image frame, and then an entropy value is calculated according to a histogram of the amplitude of the spatial gradient field to form a spatial gradient entropy feature representing the intensity of texture change, and the entropy value usually takes a value in a range of 0 to 8 bits;

[0078] A PWC-Net model is used to perform an optical flow estimation operation on two consecutive adjacent video frames to obtain a motion vector feature of each video frame, and the motion vector feature is a two-dimensional vector (u, v) representing the displacement of a corresponding pixel in adjacent frames.

[0079] A local binary pattern (LBP) transformation is performed in a local window of each image frame to obtain a texture complexity feature, and specifically, a texture complexity index is generated by calculating an LBP feature histogram and combining a window gray distribution variance.

[0080] The spatial gradient entropy feature, the motion vector feature, and the texture complexity feature above jointly form a multi-dimensional feature vector corresponding to each video frame. The extraction module then sends the multi-dimensional feature vector to the data interface of the prediction module through the Kafka message queue system after necessary standardization or normalization processing, for subsequent processing time prediction operations.

[0081] For the prediction module above:

[0082] In a specific implementation, the multi-dimensional feature vector is subscribed and received through the Kafka bus and input into a neural network prediction model deployed on a GPU, wherein the prediction model is a feedforward neural network model including at least 4 fully connected layers, and the hidden layer activation function is ReLU;

[0083] Subsequently, a processing time prediction distribution of each video frame is calculated based on the forward propagation of the neural network model, and the prediction distribution is fitted as a Gaussian distribution, including an expected value μ and a standard deviation σ, wherein the unit of μ is millisecond (ms), and σ represents a confidence interval range.

[0084] The prediction result above is published to the data input interface of the detection module through Kafka.

[0085] For example, the prediction module is deployed on a graphics processing unit (GPU) computing environment (such as Nvidia Tesla T4), and the prediction module uses a feedforward neural network model for prediction, which specifically includes an input layer, 4 hidden layers, and an output layer, and the hidden layers respectively have 256, 256, 128, and 64 neurons, and each hidden layer uses a ReLU activation function. After the input multi-dimensional feature vector is forward calculated through the model, the prediction module outputs the expected value and the standard deviation of the processing time prediction distribution, wherein the unit of the expected value is millisecond (ms), and the standard deviation represents the uncertainty range of the prediction.

[0086] For the detection module above:

[0087] The detection module performs the following processing in real time:

[0088] Listen to the Kafka bus to obtain the task issuing time corresponding to the current video frame and the result returning time after execution, and calculate the actual processing time T_obs of each video frame according to the high-precision timestamps (at least millisecond level) of the start and end of task execution;

[0089] Subsequently, it is determined whether each video frame is a processing time abnormal frame based on the following abnormality detection rules:

[0090] When and only when the absolute difference between the actual processing time T_obs and the expected value μ in the processing time prediction distribution output by the prediction module is greater than 3 times the standard deviation σ (i.e., |T_obs - μ| > 3σ), the current frame is determined to be an abnormal processing time frame, and the flag information is generated and published to the policy orchestration module through Kafka.

[0091] For the above policy orchestration module:

[0092] After the policy orchestration module receives the abnormal frame flag information published by the Kafka bus, the following operations are performed:

[0093] First, according to the severity of the abnormal frame (i.e., the value of the abnormal amplitude |T_obs - μ|), access the preset replacement strategy mapping table stored in the Redis in-memory database, and the mapping table structure is in JSON format.

[0094] Subsequently, after retrieving the candidate set of replacement processing strategies, a suitable strategy (such as a resolution reduction strategy or a simplified model strategy) is selected, and corresponding task rewriting instructions are generated based on the selected replacement strategy, where the task rewriting instructions are encapsulated in JSON format. The task rewriting instructions are transmitted to the processing execution module through the Kafka bus.

[0095] For the above processing execution module:

[0096] First, subscribe to the task rewriting instructions from Kafka and parse the target replacement processing strategy and the corresponding node information;

[0097] Subsequently, adjust the processing DAG (Directed Acyclic Graph) structure of the current task according to the parsed replacement processing strategy, and hot update the task configuration of the corresponding node in the cloud video processing platform;

[0098] Finally, execute the processing task on the video frame under the new replacement strategy, output the optimized video frame to the downstream node of the video processing pipeline or store it in the object storage service (such as MinIO or Aliyun OSS), and realize dynamic optimization of abnormal processing time.

[0099] wherein the spatial gradient entropy refers to an image texture variation intensity indicator expressed in entropy with unit of bits. The motion vector refers to a two-dimensional vector of pixel displacement between adjacent video frames, including horizontal (u) and vertical (v) components. The texture complexity refers to an image texture feature calculated based on the joint of LBP local binary pattern and gray distribution variance, without unit. The processing time-consuming abnormal frame refers to a frame whose actual processing time significantly deviates from the expected value of the prediction distribution, satisfying the judgment criterion of |T_obs - μ| > 3σ. The task rewriting instruction refers to a JSON instruction containing job_id, target node, DAG configuration, and time to live (ttl), used to adjust the task running strategy.

[0100] For example, after receiving the task rewriting instruction, the processing execution module first preloads the model weights (such as MobileNet or EfficientNet-Lite model) required for the alternative processing strategy in the GPU memory of the target node, while caching the abnormal frames to continue processing using the original processing strategy until the preloading is completed. After the preloading is completed, the processing execution module performs image difference evaluation (such as PSNR or SSIM indicators) on the processing results of the new and old strategies, and when the difference meets the preset threshold requirement, seamless switching of the model strategy can be achieved. When switching, the processing execution module generates a series of smooth transition frames using pixel-level linear weighting of the video frames generated by the new and old strategies, ensuring visual stability and user experience during the switching process.

[0101] In this way, by predicting the processing time-consuming distribution (expected value and standard deviation) of each video frame in real time in the video processing flow, and combining the real-time measurement of the actual time-consuming for abnormal detection, the precise identification and rapid response of the time-consuming abnormal frame are achieved, effectively solving the technical problem of significant reduction of overall throughput rate caused by dramatic increase of processing time of individual frames in traditional video batch processing flow;

[0102] Secondly, the system realizes frame-level time-consuming prediction based on the clearly defined multi-dimensional features (spatial gradient entropy, motion vector, and texture complexity), improving the accuracy and robustness of the prediction, and effectively alleviating the problem that traditional single feature or simple combination of features cannot accurately express the characteristics of complex video scenes;

[0103] In addition, by introducing the strategy arrangement module and the processing execution module, the automatic switching and dynamic rewriting mechanism for time-consuming abnormal frames are realized, significantly improving the adaptability and flexibility of the cloud video processing pipeline, effectively avoiding the defects of unreasonable resource allocation and large task execution delay fluctuation under the traditional fixed processing strategy, and ultimately improving the overall performance and user experience of the system.

[0104] In actual video processing systems, although the prediction module can generate a time-consuming processing prediction distribution based on the feature vector, due to the complexity of the content of the video frame itself and the inherent uncertainty in the processing flow, the traditional prediction model usually has a large deviation between the prediction result and the actual time consumption, especially in complex scenarios, which is prone to large errors and difficult to ensure accurate identification of abnormal situations.

[0105] To solve the above problems, the prediction module of the embodiment adopts a generative adversarial network (GAN) structure to improve the accuracy and robustness of the processing time prediction. Specifically, the prediction module includes two sub-networks, a generator and a discriminator, which are responsible for prediction generation and prediction result authenticity determination, respectively.

[0106] The generator is used to receive the multi-dimensional feature vector provided by the extraction module, specifically including spatial gradient entropy, motion vector and texture complexity features, and generate a prediction distribution of the processing time of each video frame based on these feature vectors, outputting a prediction expectation value (μ) and a prediction standard deviation (σ) to represent the average level and fluctuation range of the processing time.

[0107] The discriminator is used in the model training stage, specifically by comparing the actual processing time data collected in real time with the prediction distribution output by the generator, and determining the authenticity of the expectation value and standard deviation given by the generator.

[0108] In specific implementation, the discriminator adopts a feedforward neural network structure, inputs the prediction expectation value and standard deviation output by the generator, and the actual processing time, calculates the prediction error through a loss function such as Negative Log-Likelihood, and outputs the authenticity determination result. The loss result output by the discriminator is used for back propagation to dynamically update the parameters of the generator model, gradually improving the accuracy of the prediction result of the generator.

[0109] In an exemplary embodiment, the generator in the generative adversarial network (GAN) specifically includes an input layer, multiple hidden layers and an output layer, each hidden layer adopts a ReLU activation function to mine effective information in the multi-dimensional feature vector. The discriminator also adopts a multi-layer feedforward neural network structure, with a similar number of hidden layers and neuron settings as the generator structure, to achieve effective training and determination. The generator and the discriminator are trained alternately through adversarial training, i.e. the generator tries to produce more accurate predictions, and the discriminator strives to more accurately identify the deviation of the predictions, both of which promote each other until a high prediction accuracy and balance state are reached.

[0110] For example, for the generator: five fully connected layers can be set in turn, the number of hidden units of the first four layers is 256, 256, 128, 64 respectively, and ReLU activation is used; the last layer is a linear output layer, which directly gives a two-dimensional output corresponding to the expected value μ and the standard deviation σ of the prediction distribution.

[0111] For the discriminator: four fully connected layers are set in turn, the number of hidden units of the first three layers is 256, 128, 64 respectively, and ReLU activation is used; the last layer is an output node with Sigmoid activation, used to output the authenticity judgment result.

[0112] Among them, the input of the generator is the multi-dimensional feature vector (including spatial gradient entropy, motion vector and texture complexity, etc.) output by the extraction module after standardization.

[0113] The input of the discriminator is the vector formed by splicing (μ, σ) output by the generator and the actual processing time of the same frame, which is used to judge the matching degree of the prediction distribution and the actual time consumption.

[0114] In addition, the generator comprehensive loss is composed of two parts: the negative log-likelihood item between the prediction distribution and the actual time consumption; and the adversarial loss item output by the discriminator, both of which are weighted and summed according to the preset coefficient.

[0115] The discriminator loss adopts Wasserstein distance and introduces gradient penalty during back propagation to improve the stability of training and alleviate the problem of gradient vanishing.

[0116] The optimizer uses Adam, and the learning rate of the generator can be set to 1×10⁻ 4 , and the learning rate of the discriminator can be slightly higher, for example, 4×10⁻ 4 ; the batch size can be set to 256; the training process adopts a 1:1 generator and discriminator alternating update strategy.

[0117] Through the implementation of the above-mentioned generative adversarial network (GAN), the embodiment effectively reduces the problem of insufficient prediction accuracy in traditional prediction methods, significantly improves the stability and robustness of processing time prediction, and thus more accurately detects and responds to abnormal time consumption in video processing tasks. In addition, this structure design and training mechanism also improves the adaptability and accuracy of the system in processing complex video scenes, effectively supporting the real-time optimization needs of cloud video processing tasks.

[0118] In the specific implementation process, in order to more effectively utilize the differentiated information among spatial gradient entropy, motion vector and texture complexity, and further improve the processing time prediction accuracy and robustness, the prediction module in the embodiment adopts a parallel feature processing and fusion structure, and the specific implementation scheme is as follows:

[0119] Firstly, after receiving the multi-dimensional feature vector sent by the extraction module, the prediction module inputs the spatial gradient entropy feature, the motion vector feature and the texture complexity feature contained in the feature vector into three independent sub-networks (i.e. parallel sub-networks) respectively, and each of them independently extracts and processes the features.

[0120] In a specific implementation, the spatial gradient entropy feature enters the first sub-network, the motion vector feature enters the second sub-network, and the texture complexity feature enters the third sub-network. Each sub-network can adopt an independent feedforward neural network structure, for example, an exemplary setting is 2 to 3 layers of fully connected hidden layers, the number of neurons in each hidden layer is 128 or 256, and the activation function adopts ReLU activation function. After processing by each sub-network, the first hidden vector, the second hidden vector and the third hidden vector are output respectively, which abstractly and effectively represent the original feature information.

[0121] Subsequently, the prediction module fuses the above three hidden vectors through a feature fusion layer, and in a specific implementation, a vector concatenation operation can be adopted to fully integrate the differentiated information of each feature channel. The fused feature vector is further processed by an additional fully connected fusion layer (for example, a hidden layer with 128 neurons) to form a fused feature vector, so as to provide more sufficient and effective feature expression and significantly improve the expression ability between the features and the target (processing time).

[0122] Finally, the prediction module inputs the fused feature vector into a network output layer with a double-head output structure.

[0123] Specifically, the output layer includes two parallel output heads (i.e. double-head structure), the first output head is a linear output for generating the expected value (μ) of the processing time prediction distribution; the second output head is also a linear output for generating the standard deviation (σ) of the processing time prediction distribution. The design of this double-head structure ensures that the prediction module can explicitly express the average level of the predicted time and the prediction uncertainty, providing more sufficient and clear information basis for subsequent anomaly detection and strategy decision.

[0124] In this way, the problem that a single feature or a simple combination of features is difficult to fully capture the time consumption fluctuation characteristics of video frames is alleviated, and the potential contribution of various features is fully tapped, further improving the prediction accuracy and stability. This network structure design is particularly beneficial for real-time or quasi-real-time video analysis tasks, ensuring that the system can accurately and quickly respond to various video content and uncertain factors in the processing flow, effectively guaranteeing the running efficiency and stability of the entire cloud video processing platform.

[0125] As an optional implementation, the fusing the first hidden vector, the second hidden vector and the third hidden vector to obtain a fused feature vector comprises:

[0126] extracting a previous video frame motion vector and a next video frame motion vector adjacent to the current video frame based on the target video frame rate;

[0127] linearly interpolating the previous video frame motion vector and the next video frame motion vector at a 0.5 frame time interval to generate a half-frame interpolation motion vector;

[0128] replacing a time sequence dimension in the second hidden vector with the half-frame interpolation motion vector to obtain a time sequence aligned second hidden vector;

[0129] concatenating the first hidden vector, the time sequence aligned second hidden vector and the third hidden vector in a channel dimension to obtain an initial fused vector;

[0130] performing channel attention weighting processing on the initial fused vector to obtain a weighted fused vector;

[0131] performing linear transformation on the weighted fused vector to obtain the fused feature vector.

[0132] In the pre-solution, although the first hidden vector, the second hidden vector and the third hidden vector correspond to different types of feature information respectively, due to the fact that different features exist in a certain misalignment in the time sequence dimension, for example, the motion vector feature exists between adjacent video frames, the feature expression after fusion may not be accurate enough, which affects the accuracy of the processing time prediction.

[0133] To solve the above problems, the embodiment adopts an accurate time sequence alignment and channel attention mechanism to improve the accuracy of feature fusion and prediction performance.

[0134] In a specific implementation, first, according to the frame rate information of the target video, the prediction module extracts the motion vector features of the two adjacent frames before and after the current video frame; then, by linear interpolation method, the half-frame interpolation motion vector at the intermediate time is calculated and generated between the two adjacent frames, to ensure that the motion vector is accurately matched in the time dimension of the current frame. Then, the original time sequence dimension data of the second hidden vector is replaced with the generated half-frame interpolation motion vector, to realize the time sequence accurate alignment of the motion vector feature.

[0135] For example, the prediction module first obtains the frame rate information of the current frame, for example, 30 frames / second or 60 frames / second, according to which the time interval between adjacent frames is determined. Then, the motion vector data at the intermediate position (0.5 frame position) is obtained by interpolation calculation, and the original time sequence dimension information of the second hidden vector is replaced to obtain the time sequence aligned second hidden vector.

[0136] Next, the prediction module directly concatenates the first hidden vector, the second hidden vector after time alignment, and the third hidden vector in the channel dimension to generate an initial fusion vector. To further enhance the expression ability of the fusion feature, the prediction module adopts a channel attention mechanism, such as a channel attention network (e.g., a Squeeze-and-Excitation structure), to calculate a corresponding attention weight for each channel of the initial fusion vector, and obtain a weighted fusion vector through attention weighting.

[0137] Finally, the prediction module performs linear transformation processing on the weighted fusion vector through a fully connected linear transformation layer to generate a fusion feature vector for predicting the expected value and standard deviation of the video frame processing time.

[0138] In this way, the misalignment of different features in the time sequence dimension is effectively solved, the accuracy and effectiveness of the fusion feature are improved, and the precision and stability of the video frame processing time prediction are further improved, thereby providing high-quality feature support for subsequent anomaly detection and response scheduling.

[0139] As an optional implementation, referring to Figure 2 A flowchart of a method for performing channel attention weighting processing on the initial fusion vector provided by the embodiment of the present application includes steps S101-S104, wherein:

[0140] S101: Obtain the time-consuming anomaly frame history label triggered by the motion vector feature corresponding to the current video frame, and count the abnormal frame proportion of the time-consuming anomaly frame triggered by the motion vector feature in all processing time-consuming anomaly frames within a set time window;

[0141] S102: Calculate the saliency suppression factor of the second hidden vector corresponding channel based on the abnormal frame proportion, which is used to dynamically reduce the initial weight of the channel in attention weighting;

[0142] S103: Perform saliency suppression factor adjustment on the attention weight of each channel in the initial fusion vector to obtain a dynamically corrected channel weight;

[0143] S104: Perform weighting processing on the initial fusion vector based on the dynamically corrected channel weight to obtain the weighted fusion vector.

[0144] Further, although the expression ability of the fused features is enhanced by the initial fusion vector and the channel attention weighting processing, there are still problems in actual applications: the features of certain channels (such as motion vector features) frequently trigger time-consuming abnormal frames in certain cases, making the influence of this feature channel on the prediction result too large, thereby possibly masking the information of other important feature channels, resulting in an unsatisfactory feature fusion effect.

[0145] To solve the above problems, the embodiment introduces a dynamic channel attention weight adjustment strategy, which dynamically adjusts the channel weight through the statistical information of historical abnormal frames, effectively balances the contribution degree of each channel in the fused features, avoids the over-dominance of a single channel, and improves the effectiveness and prediction stability of the fused features.

[0146] In specific implementation, when performing the channel attention weighting processing, the prediction module not only depends on the initial attention weight generated by the traditional channel attention network, but also additionally considers the abnormal labels related to the motion vector features of the current video frame in the historical frame data. Specifically, the prediction module first counts the proportion of time-consuming abnormal frames in the historical frames similar to the motion vector features of the current video frame within a set time window, and calculates a saliency suppression factor for the motion vector feature channel according to the abnormal proportion. The saliency suppression factor is used to dynamically adjust the attention weight of the channel, effectively reducing the initial influence of the channel in feature fusion, and preventing the over-dominance of the channel that triggers abnormal frames too frequently.

[0147] For example, first, the prediction module obtains frames with similar motion vector features from the historical data as the current video frame, and identifies the number of frames labeled as time-consuming abnormal frames in these historical frames; next, within a given time window, the proportion of the above abnormal frames in the total number of historical frames is calculated, i.e. the proportion of time-consuming abnormal frames is obtained; based on the proportion of time-consuming abnormal frames, the prediction module calculates a saliency suppression factor, for example, the higher the abnormal proportion, the larger the calculated saliency suppression factor, and the more the initial value of the channel weight is reduced;

[0148] Then, the prediction module applies the calculated saliency suppression factor to the initial attention weight of the corresponding motion vector feature channel in the initial fusion vector to dynamically adjust the weight of the channel.

[0149] The prediction module performs channel attention weighting processing on the initial fusion vector based on the dynamically corrected channel weight to generate the final weighted fusion vector.

[0150] For example, the time window of the statistics is set to the past 60 seconds, the prediction module analyzes the history frames similar to the motion vector features corresponding to the current video frame, finds a total of 120 history frames, of which 24 frames are marked as time-consuming abnormal frames, and the proportion of abnormal frames is 20%. According to the preset linear or nonlinear function relationship (for example, linear scaling or logarithmic function), the corresponding significance suppression factor is calculated. Subsequently, the significance suppression factor is applied to the motion vector channel weight generated by the original attention network, so as to dynamically adjust the initial weight of the channel, and avoid excessive influence on the fusion feature vector.

[0151] The dynamic channel attention weight adjustment strategy adopted in the embodiment can effectively reduce the excessive influence of the feature channel frequently triggering time-consuming abnormalities on the fusion feature, improve the balance and accuracy of the fusion feature, and thus significantly improve the stability and overall performance of the video frame processing time-consuming prediction.

[0152] As an optional implementation, refer to Figure 3 A flowchart of a method for updating the model parameters of the generator using the determination result is provided in the embodiment of the present application, including steps S201-S204, wherein:

[0153] S201: Based on the difference between the expected value and the actual processing time-consuming, a negative log-likelihood function is constructed in combination with the standard deviation, and a prediction distribution real error is calculated;

[0154] S202: The proportion of samples marked as processing time-consuming abnormal frames by the detection module in the current training batch of the discriminator is counted, and an abnormal frame proportion weight coefficient is generated based on a preset mapping function;

[0155] S203: The prediction distribution real error and the adversarial loss output by the discriminator are weighted and fused to generate a comprehensive adversarial loss;

[0156] S204: The comprehensive adversarial loss is input into the significance gating backpropagation path of the generator, and the gradient of each hidden channel is distributed based on the channel attention weight, so as to update the model parameters of the generator.

[0157] Further, although the prediction module improves the accuracy of processing time prediction by using a generative adversarial network (GAN) structure, in the actual training process, the traditional GAN training method only focuses on the overall error in evaluating the authenticity of the prediction result, and lacks differentiated processing of abnormal situations and the importance of different feature channels. This homogeneous loss feedback may cause the model to be unable to effectively cope with frequent changes in abnormal processing time, thereby reducing the robustness of the prediction result under abnormal conditions. The embodiment is based on an abnormal frame proportion weight adjustment and channel attention weighting loss feedback mechanism, which differentiates abnormal conditions from normal conditions and combines a channel attention significance gating mechanism to achieve differentiated and refined model parameter updates.

[0158] First, the discriminator receives the expected value (μ) and standard deviation (σ) predicted by the generator, combines the actual measured processing time, calculates the prediction error, and constructs the negative log-likelihood function to quantify the authenticity error of the prediction distribution.

[0159] Then, the prediction module counts the proportion of samples of the processing time abnormal frames explicitly marked by the detection module in the current training batch of the discriminator, and obtains an abnormal frame proportion weight coefficient according to a pre-defined nonlinear or linear mapping function. The weight coefficient is used to dynamically adjust the weight between the authenticity error and the adversarial loss output by the discriminator to highlight the contribution of abnormal frames in the loss function.

[0160] Subsequently, the prediction module weights and fuses the prediction distribution authenticity error and the adversarial loss generated by the discriminator to calculate a comprehensive adversarial loss.

[0161] Next, the prediction module feeds the comprehensive adversarial loss into the significance gating backpropagation path of the generator model training. Specifically, the significance gating mechanism differentiates the gradient information and allocates it to different hidden channels of the generator network according to the distribution of the channel attention weight, and adjusts the parameters of each hidden channel to optimize the channel weight and overall prediction accuracy.

[0162] wherein the discriminator inputs the expected value (μ) and standard deviation (σ) predicted by the generator in each training batch, combines the actual measured processing time, and calculates the corresponding negative log-likelihood function value, i.e. the prediction distribution authenticity error. In the same training batch, the prediction module counts the proportion of abnormal frames marked by the detection module in the data received by the discriminator, for example, if there are 256 samples in a batch, and 64 of them are marked as abnormal frames, the abnormal frame proportion is 25%.

[0163] The prediction module inputs the above abnormal frame proportion into a pre-defined mapping function (such as a linear or exponential function) to calculate an abnormal frame proportion weight coefficient, which is used to dynamically adjust the weight between the prediction distribution authenticity error and the adversarial loss.

[0164] The prediction distribution authenticity error and the discriminator output adversarial loss are fused by weighting to form a comprehensive adversarial loss.

[0165] The prediction module inputs the comprehensive adversarial loss as a loss signal for back propagation to the generator network, and the significance gating mechanism dynamically allocates the gradient of the comprehensive adversarial loss to different hidden channels of the generator according to the channel importance weight calculated by the channel attention in the prediction module, so as to realize fine parameter update.

[0166] For example, the proportion of abnormal frames in a certain training batch is 20%, and the abnormal frame proportion weight coefficient is obtained as 0.8 through a preset linear mapping function. The prediction authenticity error calculated by the discriminator is 0.4, and the adversarial loss is 0.3, so the comprehensive adversarial loss is (0.8 x 0.4) + (0.2 x 0.3) = 0.38. The loss is then transmitted into the significance gating back propagation path of the generator model, and the channel attention mechanism differentially allocates the gradient of each hidden channel, so that the more critical channel obtains a higher gradient adjustment amplitude, thereby realizing accurate optimization of the model parameters.

[0167] In this way, the shortcomings of the traditional homogeneous loss feedback mechanism in processing abnormal data are effectively solved, the accuracy of the generator model in processing video frame processing time prediction and the robustness to abnormal changes are improved, and the adaptability of the model in complex video scenes is significantly improved.

[0168] As an optional implementation, refer to Figure 4 A flowchart of a method for generating a comprehensive adversarial loss provided by the embodiment of the application includes steps S301-S305.

[0169] S301: Based on the distribution of the displacement amplitude of the motion vector in the multi-dimensional feature vector, the coefficient of variation of the motion vector feature in the training batch is extracted;

[0170] S302: The fusion weight of the adversarial loss output by the discriminator is scaled and adjusted according to the coefficient of variation;

[0171] S303: Based on the mean density index of the texture complexity in the multi-dimensional feature vector, an upper limit constraint factor of the fusion weight is generated;

[0172] S304: The fusion weight of the authenticity error is limited according to the upper limit constraint factor;

[0173] S305: Based on the scaled adversarial loss fusion weight and the limited authenticity error fusion weight, the authenticity error and the adversarial loss are weighted to generate a comprehensive adversarial loss.

[0174] Further, although the comprehensive adversarial loss combines the prediction distribution authenticity error and the adversarial loss of the discriminator output, this fusion method does not fully consider the dynamic change characteristics of various features (such as motion vector and texture complexity) in actual scenes, which may lead to excessive simplicity in the model parameter updating process and difficulty in effectively processing conditions with obvious feature distribution changes or frequent fluctuations, thereby limiting the adaptability of the model under complex video content. Based on the dynamic weight adjustment strategy of the motion vector feature variation coefficient and the texture complexity average density index, the accuracy and relevance of the loss function fusion are improved through more precise dynamic weight regulation, and the processing capability of the model for complex video scenes is enhanced.

[0175] First, the prediction module extracts the offset amplitude distribution of the motion vector feature based on the multi-dimensional feature vector of each training batch, and calculates the variation coefficient of the feature distribution to quantify the fluctuation degree of the motion vector feature within the batch.

[0176] Then, the prediction module scales and adjusts the adversarial loss fusion weight output by the discriminator according to the calculated motion vector feature variation coefficient, that is, the larger the variation coefficient, the more obvious the feature fluctuation, and the weight of the discriminator adversarial loss is correspondingly increased to more effectively respond to the dramatic changes in the feature.

[0177] At the same time, the prediction module calculates the average density index within the batch for the texture complexity feature, which reflects the concentration degree of the texture complexity feature distribution. Based on this average density index, an upper limit constraint factor is generated to limit the fusion weight of the authenticity error, ensuring that the authenticity error maintains a reasonable proportion in the fused loss.

[0178] Finally, the prediction module combines the adversarial loss fusion weight after scaling adjustment and the authenticity error fusion weight after upper limit constraint to fuse the prediction distribution authenticity error and the adversarial loss output by the discriminator in a weighted manner, obtaining the comprehensive adversarial loss.

[0179] In specific implementation, in each training batch, the prediction module calculates the offset amplitude distribution of the motion vector feature in the batch, for example, calculates the mean and standard deviation of the motion vector amplitude in the batch, and further calculates the variation coefficient (the ratio of the standard deviation to the mean) to represent the fluctuation degree of the motion vector feature within the batch. According to the variation coefficient of the motion vector feature (such as specific values of 0.2 or 0.5), a linear or nonlinear mapping function is used to scale and adjust the adversarial loss fusion weight output by the discriminator, for example, when the variation coefficient is high, the adversarial loss weight of the discriminator is correspondingly increased to 1.2 times or 1.5 times.

[0180] In the same batch, the prediction module calculates the average density index of the texture complexity feature to measure the concentration degree of the texture feature distribution, for example, calculates the mean and variance of the texture complexity within the batch.

[0181] The upper limit constraint factor of the fusion weight is generated based on the texture complexity average density index. For example, when the texture complexity concentration is high, the upper limit factor is set to a lower value (such as 0.7) to limit the maximum value of the authenticity error fusion weight, so as to avoid that the error term excessively dominates the loss fusion. Finally, the two loss terms are combined in a weighted manner according to the adjusted adversarial loss fusion weight and the limited authenticity error fusion weight to form a comprehensive adversarial loss.

[0182] In this way, the application can dynamically and effectively adjust the weight of the fused loss, and more accurately respond to the dynamic changes of different features, thereby significantly improving the adaptability of the prediction model to complex and changing scenes, and improving the accuracy and stability of the video frame processing time prediction.

[0183] As an optional implementation, the operation of determining the alternative processing strategy corresponding to the processing time abnormal frame and generating the task rewriting instruction according to the preset alternative strategy mapping table in response to the determination of the processing time abnormal frame comprises:

[0184] extracting abnormal amplitude information, motion vector active channel identification and current system resource state corresponding to the processing time abnormal frame;

[0185] retrieving a candidate alternative processing strategy set in a preset mapping table according to the abnormal amplitude information;

[0186] performing score calculation on each strategy in the candidate alternative processing strategy set based on the motion vector active channel identification and the current system resource state to obtain a strategy optimization score;

[0187] selecting a target alternative processing strategy with the highest strategy optimization score, and generating a task rewriting instruction according to the processing node definition of the target alternative processing strategy.

[0188] Further, the existing strategy selection is usually based on static or simple preset rules, and different abnormal situations and dynamic system resource states are not fully considered, which may easily lead to that the selected alternative strategy does not have optimality or insufficient adaptability when actually executed, so as to fail to effectively improve the overall efficiency of the video processing task. The dynamic strategy optimization method based on abnormal amplitude information, motion vector active channel identification and current system resource state is used in the embodiment to realize fine and dynamic alternative processing strategy selection, and the adaptability of the alternative strategy execution and the system performance are significantly improved.

[0189] Wherein, the policy arrangement module first extracts the following three types of information after receiving the time-consuming abnormal frame marking information sent by the detection module: abnormal amplitude information (the difference between the actual processing time and the predicted expected value), motion vector activation channel identifier (indicating the motion vector feature channel triggering the exception), and current system resource state (including CPU, GPU resource utilization and memory remaining situation, etc.).

[0190] Subsequently, the policy arrangement module accesses the pre-stored alternative strategy mapping table based on the abnormal amplitude information to retrieve a candidate alternative strategy set, which may include but is not limited to reducing video frame processing resolution, selecting a lightweight model, adjusting processing priority, reducing frame rate, etc.

[0191] Next, the policy arrangement module performs scoring calculation on each strategy in the candidate alternative strategy set according to the motion vector activation channel identifier and the current system resource state. The scoring process is as follows: first, preset a basic score for each alternative strategy; then, adjust the weight according to the motion vector activation channel identifier for strategy applicability, for example, when the motion vector channel exception frequency is high, increase the strategy score of reducing model complexity or frame rate; further adjust the strategy execution cost and resource demand based on the current system resource state, for example, when the GPU load is high, appropriately increase the score of the strategy of reducing resolution or selecting a lightweight model.

[0192] In specific implementation, the arrangement module receives abnormal frame marking information from the detection module in real time, and extracts abnormal amplitude information and motion vector activation channel identifier from the marking information.

[0193] The system resource monitoring component provides current system resource state data (such as CPU utilization, GPU memory occupancy rate, and memory occupancy) to the policy arrangement module in real time.

[0194] The policy arrangement module accesses the alternative strategy mapping table pre-stored in Redis or other high-efficiency memory database, and retrieves a candidate strategy set suitable for the current abnormal situation according to the abnormal amplitude.

[0195] The policy arrangement module scores each strategy in the candidate strategy set. The scoring algorithm specifically includes:

[0196] Initially, each strategy is assigned a basic score;

[0197] According to the historical abnormal situation of the motion vector activation channel, dynamically adjust the score weight of a specific strategy;

[0198] Based on the real-time system resource state, further dynamically adjust the score of the strategy, for example, when the GPU load is too high, reduce the score of the strategy of using a heavy model;

[0199] The preferred score of each strategy is calculated according to the above factors.

[0200] According to the scoring results, the strategy arrangement module selects the alternative processing strategy with the highest preferred score, and encapsulates the detailed processing node definition, processing mode, target execution node and other information of the strategy as a task rewriting instruction.

[0201] For example, if the abnormal amplitude of the current abnormal frame is large (such as the actual processing time exceeds the expected value by 50%), the motion vector activation channel identifier shows that the channel has a high frequency of historical abnormalities, and the real-time system resource monitoring shows that the GPU load reaches 90%. At this time, after the strategy arrangement module retrieves the candidate strategy set, the score of the lightweight model strategy is obviously increased, and the score of the heavy model strategy is reduced, and finally the preferred score of the "lightweight model + reduced resolution" strategy is the highest. The strategy arrangement module then generates a corresponding task rewriting instruction, encapsulates the specific task node definition, resource allocation and execution priority in JSON format, and then passes it to the processing execution module through the Kafka bus.

[0202] In this way, the response capability of the system to different video processing abnormal situations is effectively improved, the overall efficiency and stability of the video processing task are improved, and the resource allocation and performance of the cloud video processing platform are optimized.

[0203] As an optional implementation, the executing the corresponding alternative processing strategy on the cloud video processing platform and generating the optimized video frame in response to the task rewriting instruction comprises:

[0204] After receiving the task rewriting instruction, starting the backup processing node and preloading the network weight corresponding to the target alternative processing strategy in the GPU memory;

[0205] During the weight preloading, the abnormal frame to be processed is added to the transition queue, and the original processing strategy is continued to be used to generate the cache frame;

[0206] After the weight preloading is completed, the difference degree evaluation is performed on the cache frame and the comparison frame generated by using the target alternative processing strategy, and when the difference degree is lower than a preset threshold, the target alternative processing strategy is switched to;

[0207] Between a plurality of video frames before and after the switching, a transition frame is generated by using a pixel-level weighted fusion method, and a merged sequence of the transition frame and the target alternative processing strategy processing frame is output to generate the optimized video frame.

[0208] Further, when the strategy switching is performed to respond to the processing time abnormal frame, the original processing strategy is often directly stopped and the new strategy is immediately executed. This way may cause obvious visual jump or delay in video processing, affecting user experience and smooth running of the system.

[0209] To solve the above problems, the embodiment realizes the switching of the replacement strategy in a smooth transition manner, ensures the continuous stability of the video processing process, and effectively avoids the decline in visual quality.

[0210] Firstly, after receiving the task rewriting instruction sent by the strategy scheduling module, the processing execution module immediately starts the backup processing node and preloads the network model weight file required by the target replacement processing strategy in the GPU display memory, to ensure the rapidity and real-time performance of the subsequent policy switching operation.

[0211] Secondly, during the model weight preloading, the processing execution module does not immediately stop the current original processing strategy, but temporarily puts the time-consuming abnormal video frames to be processed subsequently into a specially set transition queue, and continues to use the original processing strategy to process these video frames to generate cache frames, so as to ensure the continuity of the system processing pipeline.

[0212] When the model weight preloading of the target replacement strategy is completed, the processing execution module uses the target replacement processing strategy to execute processing on the same batch of cache frames, obtains the corresponding comparison frames, and further evaluates the difference degree between the cache frames and the comparison frames. The difference degree evaluation can use image quality evaluation indexes such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), or mean square error (MSE) to determine the visual difference between the video frames generated by the new and old strategies.

[0213] The processing execution module determines whether the difference degree value obtained by the above evaluation is lower than the preset switching threshold, and only when the difference degree meets the predetermined condition, the formal switching to the target replacement processing strategy is started, to ensure that the visual quality of the video processing will not be significantly reduced during the policy switching process.

[0214] When performing the policy switching formally, the processing execution module generates a series of smooth transition frames between the continuous video frames before and after the switching through pixel-level linear weighting fusion. For example, for a plurality of continuous video frames, the weight of the frames generated by the old strategy is gradually reduced, and the weight of the frames generated by the new strategy is gradually increased, until the processing result of the target replacement processing strategy is completely used, to realize the smooth transition of the video processing strategy.

[0215] In specific implementation, after receiving the task rewriting instruction, the processing execution module uses Docker or Kubernetes to quickly start the backup processing node and immediately loads the model weight file required by the new strategy to the GPU display memory.

[0216] A transition queue is set during the model preloading stage, the video frames are continuously processed using the original strategy, and the generated cache frames are stored in the memory cache area.

[0217] After the model is loaded, the backup processing node uses the target replacement strategy to process the cache frame again and generate a comparison frame, and then calculates the difference between the cache frame and the comparison frame to determine whether the difference is less than a set threshold (for example, PSNR is greater than 35 dB or SSIM is greater than 0.95).

[0218] If the difference threshold condition is met, the formal policy switching is started, the fusion weight of the generated frame is gradually changed (for example, the new policy weight is increased by 0.2 steps), and a series of transition frames are generated to ensure the continuity and stability of user perception.

[0219] The final output of the optimized video frame sequence includes a series of smooth transition frames and a stable frame sequence processed by the target replacement processing strategy, ensuring visual quality and smooth operation.

[0220] In this way, the visual smooth transition of policy switching is realized, the video quality mutation problem caused by sudden policy switching in the traditional scheme is effectively avoided, the stability and user experience of video processing are significantly improved, and the overall performance of the cloud video processing platform is further optimized.

[0221] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

Claims

1. An operation management system based on data interaction, characterized in that: include: An extraction module is used to obtain each video frame in the target video from the cloud video processing platform and extract the multidimensional feature vector corresponding to each video frame; The multi-dimensional feature vector includes: spatial gradient entropy, motion vector and texture complexity; A prediction module, configured to generate a predicted distribution of processing time for each video frame based on the multidimensional feature vector; the predicted distribution of processing time includes an expected value and a standard deviation; a detection module, configured to obtain in real time the actual processing time of each video frame, and determine whether each video frame is a frame with abnormal processing time based on the actual processing time and the predicted distribution of the processing time; a strategy arrangement module, configured to, in response to determining the operation of processing the time-consuming abnormal frame, determine an alternative processing strategy corresponding to the time-consuming abnormal frame according to a preset alternative strategy mapping table, and generate a task rewriting instruction; A processing execution module is used to execute a corresponding alternative processing strategy on the cloud video processing platform in response to the task rewriting instruction and generate an optimized video frame.

2. The data interaction-based operation management system according to claim 1, characterized in that: The step of obtaining each video frame in the target video from the cloud video processing platform and extracting a multi-dimensional feature vector corresponding to each video frame includes: Decoding the target video, obtaining a sequence of image frames in the target video arranged in time sequence, and sequentially extracting each image frame as each video frame; Performing feature extraction processing on each video frame to generate a corresponding multi-dimensional feature vector; wherein the feature extraction processing includes: Calculate the spatial gradient entropy of each video frame to characterize the intensity of texture change; Estimating motion vectors based on pixel displacements between adjacent frames; Grayscale distribution analysis is performed on the image region to obtain texture complexity indicators.

3. The data interaction-based operation management system according to claim 1 or 2, characterized in that: The prediction module includes: A generator, configured to receive the multidimensional feature vector and output an expected value and a standard deviation representing a predicted distribution of processing time for each video frame; A discriminator is used to determine the authenticity of the expected value and the standard deviation based on the actual processing time corresponding to the multidimensional feature vector during the model training phase, and use the determination result to update the model parameters of the generator; The generator and the discriminator form a generative adversarial network.

4. The data interaction-based operation management system according to claim 3, characterized in that: The receiving the multidimensional feature vector and outputting the expected value and standard deviation representing the predicted distribution of the processing time of each video frame includes: Performing parallel sub-network processing on the spatial gradient entropy, the motion vector, and the texture complexity respectively to obtain a first hidden vector, a second hidden vector, and a third hidden vector; fusing the first latent vector, the second latent vector, and the third latent vector to obtain a fused feature vector; The fused feature vector is input into a two-head output layer, the first output head generates the expected value, and the second output head generates the standard deviation.

5. The data interaction-based operation management system according to claim 4, characterized in that: The fusing the first hidden vector, the second hidden vector, and the third hidden vector to obtain a fused feature vector includes: Extracting a previous video frame motion vector and a next video frame motion vector adjacent to the current video frame based on the target video frame rate; Performing linear interpolation on the previous video frame motion vector and the next video frame motion vector at a 0.5 frame time interval to generate a half-frame interpolation motion vector; Replacing the temporal dimension representation in the second hidden vector with the field interpolation motion vector to obtain a temporally aligned second hidden vector; splicing the first hidden vector, the time-aligned second hidden vector, and the third hidden vector in the channel dimension to obtain an initial fusion vector; Performing channel attention weighted processing on the initial fusion vector to obtain a weighted fusion vector; Performing a linear transformation on the weighted fusion vector to obtain the fusion feature vector.

6. The data interaction-based operation management system according to claim 5, characterized in that: The performing channel attention weighted processing on the initial fusion vector includes: Obtain a historical marker of a time-consuming abnormal frame triggered by a motion vector feature corresponding to the current video frame, and calculate the abnormal frame ratio of the time-consuming abnormal frame triggered by the motion vector feature in all processed time-consuming abnormal frames within a set time window; Calculating a significance suppression factor of a channel corresponding to the second hidden vector based on the abnormal frame ratio, so as to dynamically reduce an initial weight of the channel in attention weighting; Performing a saliency suppression factor adjustment on the attention weight of each channel in the initial fusion vector to obtain a dynamically corrected channel weight; The initial fusion vector is weighted based on the dynamically corrected channel weights to obtain the weighted fusion vector.

7. The data interaction-based operation management system according to claim 6, characterized in that: The performing authenticity determination on the expected value and the standard deviation based on the actual processing time corresponding to the multi-dimensional feature vector, and using the determination result to update the model parameters of the generator includes: Based on the difference between the expected value and the actual processing time, a negative log-likelihood function is constructed in combination with the standard deviation, and a prediction distribution authenticity error is calculated; The statistical discriminator calculates the proportion of samples in the current training batch that are marked by the detection module as time-consuming abnormal frames, and generates a weight coefficient for the abnormal frame proportion based on a preset mapping function; The predicted distribution authenticity error is weightedly fused with the adversarial loss output by the discriminator to generate a comprehensive adversarial loss; The comprehensive adversarial loss is input into the saliency gated back-propagation path of the generator, and the gradients of each hidden channel are distributed based on the channel attention weights to update the model parameters of the generator.

8. The data interaction-based operation management system according to claim 7, characterized in that: The weighted fusion of the predicted distribution authenticity error and the adversarial loss output by the discriminator to generate a comprehensive adversarial loss includes: Extracting the coefficient of variation of the motion vector features in the training batch based on the offset amplitude distribution of the motion vector in the multidimensional feature vector; Scaling and adjusting the fusion weight of the adversarial loss output by the discriminator according to the coefficient of variation; generating an upper limit constraint factor of the fusion weight based on a mean density index of texture complexity in the multidimensional feature vector; Limiting the fusion weight of the authenticity error according to the upper limit constraint factor; Based on the scaled adversarial loss fusion weight and the constrained authenticity error fusion weight, weighted processing is performed on the authenticity error and the adversarial loss to generate a comprehensive adversarial loss.

9. The operation management system based on data interaction according to claim 6, characterized in that: In response to determining the operation of processing the time-consuming abnormal frame, determining an alternative processing strategy corresponding to the time-consuming abnormal frame according to a preset alternative strategy mapping table, and generating a task rewriting instruction includes: Extracting abnormal amplitude information, motion vector activation channel identifier, and current system resource status corresponding to the abnormal processing time-consuming frame; Retrieving a set of candidate alternative processing strategies in a preset mapping table according to the abnormal amplitude information; Performing a scoring calculation on each strategy in the candidate alternative processing strategy set based on the motion vector activation channel identifier and the current system resource state to obtain a strategy optimization score; The target replacement processing strategy with the highest strategy preference score is selected, and a task rewriting instruction is generated according to the processing node definition of the target replacement processing strategy.

10. The operation management system based on data interaction according to claim 1, characterized in that: In response to the task rewriting instruction, executing a corresponding alternative processing strategy on the cloud video processing platform and generating an optimized video frame includes: After receiving the task rewrite instruction, starting the standby processing node and preloading the network weights corresponding to the target alternative processing strategy into the GPU memory; During the weight preloading period, the abnormal frames to be processed are added to the transition queue, and the original processing strategy is continued to be used to generate cache frames; After the weight preloading is completed, performing a difference evaluation on the cached frame and the comparison frame generated by the target replacement processing strategy, and when the difference is lower than a preset threshold, switching to the target replacement processing strategy; Between a number of consecutive video frames before and after the switching, a pixel-level weighted fusion method is used to generate a transition frame, and a merged sequence of the transition frame and the target alternative processing strategy processed frame is output to generate an optimized video frame.

Images