A video recognition method and system based on artificial intelligence

Abstract

The present invention proposes a video recognition method and system based on artificial intelligence, the method includes: constructing an adaptive framework based on video content analysis, dynamically adjusting the model structure and parameters according to the complexity of real-time video frames; applying efficient image preprocessing technology and deep learning algorithms to extract features from video frames according to model parameters; using generative adversarial networks to process the features extracted from video frames, stylizing key frames to obtain stylized video frames, and strengthening visual behavioral features; designing hybrid neural networks to perform behavior recognition and behavior prediction based on historical data according to stylized video frames. The present invention not only significantly improves the accuracy and efficiency of video recognition, but also enhances the real-time processing capability of the system and the foresight of future behavior prediction, effectively solving the core problems existing in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and in particular relates to a video recognition method and system based on artificial intelligence. Background Art

[0002] In the current technical environment, video recognition systems have been widely used in many fields such as security monitoring, content recommendation, and traffic management. Their core function is to automatically parse and identify video content through computer vision technology and deep learning algorithms. Traditional video recognition systems usually rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to process video frames and identify objects, actions, or events in them. These systems can achieve high accuracy in standardized environments, but they still have many shortcomings when facing complex and changing practical application scenarios. For example, the recognition accuracy of traditional models will drop significantly when dealing with behaviors in very complex or dynamically changing backgrounds. In addition, most existing systems need to rely on a large amount of labeled data for training, which is not only costly, but also further affects the generalization ability and practicality of the model when the quality of data annotation is inconsistent. In addition, existing video recognition technology still faces technical challenges in real-time performance processing, adaptive adjustment of models, and prediction of future behaviors.

[0003] Therefore, the adaptability of existing technologies to complex scenarios, processing efficiency, and learning ability of unlabeled data need to be improved. Summary of the invention

[0004] The purpose of the present invention is to design a video recognition method and system based on artificial intelligence, which integrates multiple new technologies for intelligent video recognition and prediction to solve the above problems.

[0005] In order to achieve the above object, a first aspect of the present invention provides a video recognition method based on artificial intelligence, the method comprising the following steps:

[0006] S1. Build an adaptive framework based on video content analysis to dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

[0007] S2. According to the model parameters, efficient image preprocessing technology and deep learning algorithm are applied to extract features from the video frame; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to use the improved Sobel operator to extract the edge information of the video frame, and then use the local binary pattern operator to analyze the image texture features, specifically:

[0008]

[0009]

[0010] Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

[0011]

[0012] Among them, g c represents the center pixel value, g p represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y) is the local binary pattern value calculated at the image coordinate (x,y);

[0013] S3. Use the generative adversarial network to process the features extracted from the video frames, stylize the key frames to obtain stylized video frames, and strengthen the visual behavioral features, including:

[0014] S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

[0015]

[0016] Among them, F k represents the feature extraction function for the key feature k, K represents the predefined associated feature set, λ k Represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss. I represents the original frame. G(z) represents the output of the generator G when the input is the hidden variable z.

[0017] The total loss function is expressed as follows:

[0018]

[0019] Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

[0020] S303, training the generative adversarial network using a progressive training method;

[0021] S4. Designing a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

[0022] The method further comprises at least the following steps:

[0023] A. Integrate unsupervised learning algorithms to dynamically adjust the models and parameters of S1-S4 based on real-time feedback;

[0024] B. Establish a feedback mechanism to dynamically adjust strategies based on user and performance feedback and continuously optimize strategy execution.

[0025] Furthermore, the S1 specifically includes:

[0026] S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network;

[0027] S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including:

[0028] D t =||V t -V t-1 ||2

[0029] Among them, D t Indicates the change in the content of the frame between time t and t-1, V t Represents the feature vector of each frame of video at time t, V t-1 Represents the feature vector of each frame of video at time t-1;

[0030] The average of these differences is calculated through a sliding window of fixed size W to obtain the scene complexity C t :

[0031]

[0032] S103, according to C t The comparison result with the preset threshold θ automatically adjusts the configuration of the network;

[0033] S104. Combined with real-time performance feedback, the back propagation algorithm is used to adjust network weights and structural parameters.

[0034] Furthermore, the illumination correction is to apply automatic white balance and exposure compensation to the input video frame, which is expressed as follows:

[0035]

[0036] Among them, I(x,y) is the original pixel value, I corr (x, y) is the corrected pixel value, I min and I max are the minimum and maximum pixel values ​​in the frame, respectively;

[0037] The noise removal is to apply a bilateral filter to remove potential image noise and maintain edge clarity, which is expressed as follows:

[0038]

[0039] Among them, W p is the normalization coefficient, f r and f s are Gaussian functions based on intensity and spatial approximation, respectively, to ensure that only adjacent pixels with similar intensities affect the current pixel value. (x, y) represents the coordinates of the pixel, (x', y') represents the coordinates of the adjacent pixels involved in the calculation when applying the filter, and I filtered (x,y) represents the pixel value of the image at coordinate (x,y) after bilateral filtering.

[0040] Furthermore, the input of the generative network is the feature vector of each frame of video at time t, and then multiple convolutional layers are used, each layer is followed by batch normalization and ReLU activation function, and the last layer uses tanh activation function to output a stylized image, which is expressed as follows:

[0041] I styled =tanh(Conv(BN(ReLU(Conv(...V t …)))))

[0042] The input of the discriminant network includes I styled And the original frame I, use multiple convolution layers for feature extraction, each layer of convolution is connected to the normalization and LeakyReLU activation function, and the last layer uses the sigmoid function to output the probability, which is expressed as follows:

[0043] P real =σ(Conv(LReLU(BN(Conv(…I styled ,I…)))))

[0044] Among them, σ represents the sigmoid activation function.

[0045] Furthermore, in S303, the progressive training method means first training the network at a low resolution and gradually transitioning to a high resolution.

[0046] Furthermore, the hybrid neural network includes a feature fusion layer, a temporal fusion layer, and a behavior recognition and prediction output layer; the feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of the image, which is expressed as follows:

[0047] F spatial =CNN(I styled )

[0048] Among them, F spatial represents the spatial feature vector extracted from the stylized image;

[0049] The temporal fusion layer transforms the feature vectors F of consecutive frames.spatial As input, the temporal dependencies and dynamic changes are processed through the recurrent neural network module, which is expressed as follows:

[0050] S t =RNN(F spatial ,S t-1 )

[0051] Among them, S t represents the hidden state at time t, carrying the accumulated information of past video frames; S t-1 represents the hidden state at time t-1;

[0052] The behavior recognition and prediction output layer is combined with F spatial and S t Generate the final behavior recognition and behavior prediction output, as follows:

[0053] Y acti on=Softmax(Dense(S t ))

[0054] Y predict =Softmax(Dense(S t ,F spatial ))

[0055] Among them, Y action is the behavior recognition result of the current frame, Y predict It is the result of predicting future behavior based on current and past information.

[0056] Furthermore, the cross entropy loss function is used to simultaneously optimize the accuracy of behavior recognition and prediction, which is expressed as follows:

[0057]

[0058] Among them, y c is the one-hot encoding of the true behavior label, is the probability distribution predicted by the model, and C is the number of behavior categories.

[0059] Furthermore, the step A specifically comprises the following steps:

[0060] A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames;

[0061] Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect abnormal behaviors or salient events in video frames;

[0062] Define the total loss function Reconstruction error and behavior recognition error The weighted sum of is calculated by introducing additional regularization terms to enhance the generalization and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights;

[0063] Integrate dynamic elements to adapt to changing video content and optimize feature transformation networks;

[0064] Introduce a model adaptive adjustment strategy to automatically adjust the network architecture or parameters based on the currently detected behavior patterns and historical data.

[0065] Furthermore, the step B specifically comprises the following steps:

[0066] Regularly collect feedback data and process the collected data using statistical analysis and machine learning methods;

[0067] Based on feedback data, define parameter adjustment strategies and dynamically adjust learning rates according to model performance changes;

[0068] Perform comprehensive evaluations periodically and update the model's operating parameters based on the evaluation results, which are then fed back to the operating parameter adjustment function and the learning rate update strategy.

[0069] In a second aspect of the present invention, a video recognition system based on artificial intelligence is provided, the system comprising:

[0070] An adaptive framework building module is used to build an adaptive framework based on video content analysis and dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

[0071] The algorithm design module is used to extract features from video frames by applying efficient image preprocessing technology and deep learning algorithm according to model parameters; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to extract edge information of the video frame using an improved Sobel operator, and then analyze image texture features using a local binary pattern operator, specifically:

[0072]

[0073]

[0074] Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

[0075]

[0076] Among them, g c represents the center pixel value, g prepresents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y0 is the local binary pattern value calculated at the image coordinate (x,y);

[0077] The enhanced feature module is used to process the features extracted from the video frames using a generative adversarial network, stylize the key frames to obtain stylized video frames, and enhance the visual behavioral features, including:

[0078] S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

[0079] S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

[0080]

[0081] Among them, F k represents the feature extraction function for the key feature k, K represents the predefined associated feature set, λ k Represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss. I represents the original frame. G(z) represents the output of the generator G when the input is the hidden variable z.

[0082] The total loss function is expressed as follows:

[0083]

[0084] Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

[0085] S303, training the generative adversarial network using a progressive training method;

[0086] A behavior prediction module, used to design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

[0087] The method further comprises at least 0 of the following modules:

[0088] Unsupervised learning algorithm module, used to integrate unsupervised learning algorithms and dynamically adjust the models and parameters of S1-S4 based on real-time feedback;

[0089] The feedback mechanism optimization module is used to establish a feedback mechanism, dynamically adjust strategies based on user and performance feedback, and continuously optimize strategy execution.

[0090] The beneficial technical effects of the present invention are at least as follows:

[0091] The present invention proposes an intelligent video recognition and prediction system integrating multiple new technologies in a video recognition method and system based on artificial intelligence. First, an adaptive learning model dynamic adjustment technology is introduced, which can automatically adjust the structure and parameters of the deep learning model according to the complexity of the input video, which is crucial to improving the adaptability and efficiency of the system in a changing environment. Secondly, through the stylized processing of video frames based on generative adversarial networks (GANs), the system can improve the accuracy and robustness of action recognition through visual information without adding additional sensors, especially in visually complex or low-quality video data. In addition, combined with time series analysis and video-to-video translation technology, the system can not only identify the current behavior, but also predict the next possible behavior sequence, which is particularly valuable in security monitoring and emergency response systems. Finally, the use of unsupervised learning technology enables the system to continuously learn and optimize the model without labeled data, solving the problem of relying on a large amount of labeled data in the prior art. The combination of these technologies not only significantly improves the accuracy and efficiency of video recognition, but also enhances the real-time processing capability of the system and the foresight of future behavior prediction, effectively solving the core problems existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0093] Figure 1 The present invention is a flowchart of a video recognition method based on artificial intelligence.

[0094] Figure 2 This is a framework diagram of a video recognition system based on artificial intelligence according to an embodiment of the present invention. DETAILED DESCRIPTION

[0095] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0096] In one or more embodiments, Figure 1As shown, the present invention discloses a video recognition method based on artificial intelligence, the method includes steps 1 to 6, including:

[0097] S1. Build an adaptive framework based on video content analysis to dynamically adjust the model structure and parameters according to the complexity of real-time video frames, including:

[0098] S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network;

[0099] Specifically, a sequence of video frames is input, and each frame is first processed by a specific feature extraction subnetwork. A simplified ResNet model is used, which is lightweight and designed for real-time data streams. Through this network, each frame of video is converted into a feature vector V t , where V t It is a vector of fixed dimension, containing key visual information of the frame such as edges, textures, color distribution, etc.

[0100] S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including:

[0101] D t =||V t -V t-1 ||2

[0102] Among them, D t Indicates the change in the content of the frame between time t and t-1, V t Represents the feature vector of each frame of video at time t, V t-1 Represents the feature vector of each frame of video at time t-1;

[0103] The average of these differences is calculated through a sliding window of fixed size W to obtain the scene complexity C t :

[0104]

[0105] Among them, i represents the i-th moment, D i Indicates the change of frame content at the i-th time point;

[0106] S103, according to C t The comparison result with the preset threshold θ automatically adjusts the configuration of the network;

[0107] Specifically, according to C t The system automatically adjusts the network configuration based on the comparison result with the preset threshold θ. tIf the value exceeds the threshold θ, it indicates that the scene changes greatly and a more complex model is needed to capture the details. In this case, the system may increase the depth of the convolution layer or the size of the convolution kernel. On the contrary, if C t If it is lower than θ, the network structure is simplified, such as reducing the number of convolutional layers or using smaller convolutional kernels, to increase processing speed and reduce computing resource consumption.

[0108] S104, combining real-time performance feedback, using back propagation algorithm to adjust network weights and structural parameters;

[0109] In the embodiment of the present invention, combined with real-time performance feedback, the system uses the back propagation algorithm to adjust the network weights and structural parameters to meet the needs of the current video content. This includes adjusting the learning rate, optimizing the gradient descent strategy, etc., ensuring that each adjustment is based on the latest performance evaluation data, thereby minimizing errors and optimizing overall performance.

[0110] S2. According to the model parameters, applying efficient image preprocessing technology and deep learning algorithm to extract features from the video frame; wherein the image preprocessing technology includes illumination correction and noise removal;

[0111] Here, lighting correction means applying automatic white balance and exposure compensation to the input video frames to standardize the lighting conditions. Use the formula:

[0112]

[0113] Among them, I(x,y) is the original pixel value, I corr (x, y) is the corrected pixel value, I min and I max are the minimum and maximum pixel values ​​in the frame, respectively.

[0114] Noise removal is to apply a bilateral filter to remove potential image noise, maintain edge clarity, and calculate the new pixel value using the following formula:

[0115]

[0116] Among them, W p is the normalization coefficient, f r and f s are Gaussian functions based on intensity and spatial approximation, respectively, to ensure that only adjacent pixels with similar intensities affect the current pixel value. (x, y) represents the coordinates of the pixel, (x', y') represents the coordinates of the adjacent pixels involved in the calculation when applying the filter, and I filtered (x,y0 represents the pixel value of the image at coordinate (x,y) after bilateral filtering.

[0117] The feature extraction from the video frame is specifically to use the improved Sobel operator to extract the edge information of the video frame, and then use the local binary pattern operator to analyze the image texture features, specifically:

[0118]

[0119] Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

[0120]

[0121] Among them, g c represents the center pixel value, g p represents the neighborhood pixel value, P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, P represents the total number of pixels in the neighborhood; LBP(x,y0) is the local binary pattern value calculated at the image coordinate (x,y).

[0122] As an embodiment of the present invention, through the above steps, video frame preprocessing will standardize the input data and reduce the interference of environmental variables, while feature extraction focuses on obtaining visual information that is critical to subsequent behavior analysis. In addition, the physical basis methods of illumination correction and noise filtering ensure the universal applicability and efficiency of the processing process, while edge detection and texture analysis provide the necessary detailed visual description for video content analysis, laying a solid foundation for subsequent in-depth analysis. Such a processing flow ensures that the video surveillance system can maintain efficient and accurate operation under various environmental conditions and meet the needs of real-time monitoring.

[0123] S3. Use the generative adversarial network to process the features extracted from the video frames, stylize the key frames to obtain stylized video frames, and strengthen the visual behavioral features, including:

[0124] S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

[0125] Specifically, the generator network (G) is designed to input V t is the feature vector of each frame of video obtained from step 2. The network structure uses multiple convolutional layers, each layer is followed by batch normalization and ReLU activation function, and the last layer uses tanh activation function to output stylized images. The goal of the generative network is to generate visually striking images based on the input feature vector, highlighting the key content, as shown below:

[0126] I styled=tanh(Conv(BN(ReLU(Conv(...V t …))))) (5)

[0127] The discriminant network (D) is designed with input I styled And the original frame I, use multiple layers of convolutional layers for feature extraction, each layer of convolution is connected to the normalization and LeakyReLU activation function, and the last layer uses the sigmoid function to output the probability of whether the image is real or generated by the generative network, which is expressed as follows:

[0128] P real =σ(Conv(LReLU(BN(Conv(…I styled ,I…))))) (6)

[0129] Among them, σ represents the sigmoid activation function, which is used to convert the output into a probability value.

[0130] S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

[0131]

[0132] Among them, F k represents the feature extraction function for key feature k, K represents the predefined key feature set, λ k Represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss. I represents the original frame. G(z) represents the output of the generator G when the input is the hidden variable z.

[0133] The total loss function is expressed as follows:

[0134]

[0135] Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

[0136] S303, training the generative adversarial network using a progressive training method;

[0137] Specifically, a progressive training method is adopted to first train the network at a low resolution and gradually transition to a high resolution. This method helps the network capture large-scale behavior patterns at an early stage and then gradually refine to important details at a small scale, thereby more effectively processing complex or blurred video scenes.

[0138] S4. According to the stylized video frames, a hybrid neural network is designed to perform behavior recognition and behavior prediction based on historical data; wherein the hybrid neural network includes a CNN network and an RNN network, which are specifically expressed as follows:

[0139] The hybrid neural network includes a feature fusion layer, a temporal fusion layer and a behavior recognition and prediction output layer;

[0140] The feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of image, which is expressed as follows:

[0141] F spatial =CNN(I styled ) (9)

[0142] Among them, F spatial represents the spatial feature vector extracted from the stylized image;

[0143] The temporal fusion layer transforms the feature vectors F of consecutive frames. spatial As input, the temporal dependencies and dynamic changes are processed through the recurrent neural network module, which is expressed as follows:

[0144] S t =RNN(F spatial ,S t-1 ) (10)

[0145] Among them, S t represents the hidden state at time t, carrying the accumulated information of past video frames; S t-1 represents the hidden state at time t-1;

[0146] The behavior recognition and prediction output layer is combined with F spatial and S t Generate the final behavior recognition and behavior prediction output, as follows:

[0147]

[0148] Among them, Y action is the behavior recognition result of the current frame, Y predict It is the result of predicting future behavior based on current and past information.

[0149] The cross entropy loss function is used to simultaneously optimize the accuracy of behavior recognition and prediction, which is expressed as follows:

[0150]

[0151] Among them, y c is the one-hot encoding of the true behavior label, is the probability distribution predicted by the model, and C is the number of behavior categories.

[0152] S5, integrates unsupervised learning algorithms to dynamically adjust the models and parameters of S1-S4 based on real-time feedback, including:

[0153] A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames.

[0154] In the embodiment of the present invention, a small convolutional neural network is constructed, whose goal is to extract and transform features from the original video frames to better meet the model recognition requirements. The network uses reversible feature transformation technology to not only extract features, but also enhance adaptability to key variables such as illumination changes and motion simulation, as shown below:

[0155] F′=FTN(I frame ) (13)

[0156] Among them, F' is the transformed eigenvector, I frame is the input video frame.

[0157] Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect unusual behaviors or salient events in video frames.

[0158] Specifically, abnormal behaviors or prominent events in video frames are detected using an anomaly detection technique based on an autoencoder, which attempts to reconstruct input features, and the reconstruction error is used to judge anomalies. The reconstruction and anomaly scores are expressed as follows:

[0159]

[0160] Among them, a high anomaly score indicates that the current frame may contain abnormal or unlearned behavior features.

[0161] Define the total loss function Reconstruction error and behavior recognition error At the same time, an additional regularization term is introduced to enhance the generalization ability and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights.

[0162] Specifically, define the total loss function Reconstruction error and behavior recognition error The weighted sum of , but an additional regularization term is introduced to enhance the generalization ability and adaptability of the model, which is expressed as follows:

[0163]

[0164] in, Measures the accuracy of reconstruction after feature transformation; is the regularization term of the model, such as L2 regularization, which is used to avoid overfitting; α, β, γ are weight parameters, which are adjusted according to the importance of different tasks.

[0165] Then, a dynamic weight adjustment mechanism based on feedback is introduced. The values ​​of α, β, and γ are automatically adjusted according to the detection frequency of abnormal behaviors in real-time monitoring data to optimize the performance of the model in a changing environment, as shown below:

[0166]

[0167] Where f, g, h are functions that adjust weights based on historical performance feedback.

[0168] Integrate dynamic elements to optimize feature transformation network to adapt to changing video content.

[0169] In the embodiment of the present invention, the feature transformation network not only performs feature encoding and decoding, but also integrates dynamic elements to adapt to the ever-changing video content. For example, time-dependent components such as small RNNs or attention mechanisms are introduced to enhance the understanding of time series and feature extraction capabilities:

[0170] F″=Attention(F,context) (17)

[0171] Among them, context is the contextual information extracted from the video sequence, which enhances the feature vector's ability to describe the current scene.

[0172] In the embodiment of the present invention, a model adaptive adjustment strategy is introduced to automatically adjust the network architecture or parameters based on the currently detected behavior pattern and historical data. For example, the model pruning technology or neural network search (NAS) is used to automatically adjust the complexity and structure of the network to adapt to different behavior recognition requirements.

[0173] This step not only improves the accuracy of behavior recognition, but also enhances the system's ability to adapt to new environments and unknown behavior patterns by comprehensively considering the reconstruction error, behavior recognition accuracy and dynamic optimization of model complexity. It is suitable for complex and dynamically changing monitoring scenarios.

[0174] S6. Establish a feedback mechanism to dynamically adjust strategies based on user and performance feedback and continuously optimize strategy execution, including:

[0175] Feedback data is collected regularly and processed using statistical analysis and machine learning methods.

[0176] Specifically, the system regularly collects data including user feedback (error reports, annotation corrections), performance indicators automatically generated by the system (recognition accuracy, processing delay), etc. Statistical analysis and machine learning methods are used to process the collected data, identify common problems and performance bottlenecks, and output performance improvement reports and adjustment suggestions.

[0177] Based on feedback data, define parameter adjustment strategies and dynamically adjust learning rates according to model performance changes, including:

[0178] Automatically adjust the learning rate to increase or decrease the learning rate based on the performance evaluation results;

[0179] When new behavior patterns are detected to occur frequently, it may be necessary to adjust the network structure (such as increasing the number of layers, adjusting the filter size) to better learn these features;

[0180] Adjust the weights of each item in the loss function according to the needs of the specific task, and make the following adjustments:

[0181]

[0182] Among them, θ represents the model parameters, η t is the dynamically adjusted learning rate, is a composite loss function that takes feedback into account, R t It is the adjustment suggestions from feedback. The data includes performance data and user feedback.

[0183] Then dynamically adjust the learning rate according to the performance changes of the model. When the model performance improves, gradually reduce the learning rate to refine the learning; when the model performance decreases, increase the learning rate to quickly adapt to new situations. The update strategy formula is:

[0184] η t+1 =η t ·(1+δ·sign(ΔPerf t )) (19)

[0185] Where δ is the adjustment factor, ΔPerf t is the rate of change of performance, and the sign function gives the corresponding sign according to whether the performance is improving or deteriorating.

[0186] Perform comprehensive evaluations periodically and update the model's operating parameters based on the evaluation results, which are then fed back to the operating parameter adjustment function and the learning rate update strategy.

[0187] In the embodiment of the present invention, the system is periodically comprehensively evaluated, including precision, recall rate and F1 score, and compared with previous performance data. The operating parameters of the system are updated according to the evaluation results and fed back to the parameter adjustment function and the learning rate update strategy.

[0188] In one or more embodiments, Figure 2 As shown, the present invention discloses a video recognition system based on artificial intelligence, comprising:

[0189] An adaptive framework building module 101 is used to build an adaptive framework based on video content analysis, and dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

[0190] The algorithm design module 102 is used to extract features from the video frame by applying efficient image preprocessing technology and deep learning algorithm according to the model parameters; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to extract the edge information of the video frame by using the improved Sobel operator, and then analyze the image texture features by using the local binary pattern operator, specifically:

[0191]

[0192]

[0193] Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

[0194]

[0195] Among them, g c represents the center pixel value, g p represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y) is the local binary pattern value calculated at the image coordinate (x,y);

[0196] The feature enhancement module 103 is used to process the features extracted from the video frames using a generative adversarial network, stylize the key frames to obtain stylized video frames, and enhance the visual behavioral features, including:

[0197] S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

[0198] S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

[0199]

[0200] Among them, Fk represents the feature extraction function for key feature k, K represents the predefined key feature set, λ k represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, and I represents the original frame;

[0201] The total loss function is expressed as follows:

[0202]

[0203] Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term;

[0204] S303, training the generative adversarial network using a progressive training method;

[0205] The behavior prediction module 104 is used to design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

[0206] The method further comprises at least 0 of the following modules:

[0207] An unsupervised learning algorithm module 105, for integrating an unsupervised learning algorithm and dynamically adjusting the models and parameters of S1-S4 based on real-time feedback;

[0208] The feedback mechanism optimization module 106 is used to establish a feedback mechanism, dynamically adjust the strategy according to user and performance feedback, and continuously optimize the strategy execution.

[0209] In summary, the present invention initializes an adaptive learning model framework to set the foundation of the system, which can automatically adjust its network structure and parameters according to the complexity of the real-time video content. This adaptive ability is the key to solving the application of video surveillance in a changing environment. It ensures that no matter how complex the environment is, the system can adjust itself to achieve the best processing performance and recognition accuracy. Based on this framework, real-time video frames are preprocessed and feature extracted to ensure that the most critical information is obtained from each frame of video. This step is efficiently executed because the adaptive framework has provided it with model parameters and structures that are most suitable for the current video content, thereby optimizing the feature extraction process and improving the speed and accuracy of data processing. The extracted features are stylized using generative adversarial networks (GANs), which enhances the visual expression of key behaviors in the video frames and makes subsequent behavior recognition more accurate. Stylization is performed based on optimized features, so it can more effectively highlight behavioral features, especially in scenes with complex or low-quality visual information. A hybrid neural network is used to perform behavior recognition and future behavior prediction on the stylized video frames. This step is effective because the stylized video frames have strengthened the necessary behavioral features, allowing the neural network to learn and predict more accurately. Unsupervised online learning mechanisms are integrated into the system, allowing the model to self-optimize in actual operation, and continuously adjust parameters and strategies through real-time feedback. The introduction of this mechanism enables the system to adapt to long-term changes in the operating environment, reduces dependence on manual intervention, and improves the autonomy and reliability of the system. By establishing a system feedback mechanism, user and performance data are continuously collected to dynamically optimize the entire system. This closed-loop feedback mechanism ensures that the system can continue to improve and adjust its strategies in real time to meet new challenges and needs.

[0210] The present invention solves the problems of insufficient recognition accuracy, poor adaptability to complex environments, low data processing efficiency, and reliance on a large amount of labeled data in the background technology. Through the introduction of adaptive learning models, the system can automatically optimize processing strategies for different video contents; through the application of stylized processing and hybrid neural networks, the accuracy and predictive ability of behavior recognition are greatly improved; and the combination of unsupervised learning and feedback mechanisms ensures the long-term effectiveness and autonomous optimization capabilities of the system. This design is not only technologically innovative, but also provides significant performance improvements in practical applications, which fully meets the needs of modern intelligent video surveillance systems.

[0211] These are just some preferred embodiments of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A video recognition method based on artificial intelligence, characterized in that: The method comprises the following steps: S1. Build an adaptive framework based on video content analysis to dynamically adjust the model structure and parameters according to the complexity of real-time video frames, including: S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network; S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including: D t =||V t -V t-1 ||2 Among them, D t Indicates the change in the content of the frame between time t and t-1, V t Represents the feature vector of each frame of video at time t, V t-1 Represents the feature vector of each frame of video at time t-1; The average of these differences is calculated through a sliding window of fixed size W to obtain the scene complexity C t : S103, according to C t The comparison result with the preset threshold θ automatically adjusts the configuration of the network; S104, combining real-time performance feedback, using back propagation algorithm to adjust network weights and structural parameters; S2. According to the model parameters, efficient image preprocessing technology and deep learning algorithm are applied to extract features from the video frame; wherein the image preprocessing technology includes illumination correction and noise filtering; the feature extraction from the video frame is specifically to use the improved Sobel operator to extract the edge information of the video frame, and then use the local binary pattern operator to analyze the image texture features, specifically: Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength; Among them, g c represents the center pixel value, g p represents the neighborhood pixel value, P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel; S3. Use the generative adversarial network to process the features extracted from the video frames, stylize the key frames to obtain stylized video frames, and strengthen the visual behavioral features, including: S301, respectively designing a generative adversarial network, including a generative network and a discriminative network; S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows: Among them, F k represents the feature extraction function for the key feature k, K represents the predefined associated feature set, λ k represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, I represents the original frame, and G(z) represents the output of the generator G when the input is the hidden variable z; The total loss function is expressed as follows: Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network; S303, training the generative adversarial network using a progressive training method; S4. Design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network, specifically including: The hybrid neural network includes a feature fusion layer, a temporal fusion layer and a behavior recognition and prediction output layer; The feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of image, which is expressed as follows: F spatial =CNN(I styled ) Among them, F spatial represents the spatial feature vector extracted from the stylized image; The temporal fusion layer transforms the feature vectors F of consecutive frames. spatial As input, the temporal dependencies and dynamic changes are processed through the recurrent neural network module, which is expressed as follows: S t =RNN(F spatial ,S t-1 ) Among them, S t represents the hidden state at time t, carrying the accumulated information of past video frames; S t-1 represents the hidden state at time t-1; The behavior recognition and prediction output layer is combined with F spatial and S t Generate the final behavior recognition and behavior prediction output, expressed as follows: AND action =Softmax(Dense(S t )) AND predict =Softmax(Dense(S t ,F spatial )) Among them, Y action is the behavior recognition result of the current frame, Y predict It is the result of predicting future behavior based on current and past information; The method further comprises at least one of the following steps: A. Integrate unsupervised learning algorithms to dynamically adjust the models and parameters of S1-S4 based on real-time feedback, including the following steps: A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames; Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect abnormal behaviors or salient events in video frames; Define the total loss function Reconstruction error and behavior recognition error The weighted sum of is calculated by introducing additional regularization terms to enhance the generalization and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights; Integrate dynamic elements to adapt to changing video content and optimize feature transformation networks; Introduce model adaptive adjustment strategies to automatically adjust network architecture or parameters based on currently detected behavior patterns and historical data; B. Establish a feedback mechanism to dynamically adjust strategies based on user and performance feedback and continuously optimize strategy execution.

2. The video recognition method based on artificial intelligence according to claim 1, characterized in that: The illumination correction is to apply automatic white balance and exposure compensation to the input video frame, which is expressed as follows: Among them, I(x,y) is the original pixel value, I corr (x, y) is the corrected pixel value, I min and I max are the minimum and maximum pixel values ​​in the frame, respectively; The noise filtering is to apply a bilateral filter to remove potential image noise and maintain edge clarity, which is expressed as follows: Among them, W p is the normalization coefficient, f r and f s are Gaussian functions based on intensity and spatial approximation, respectively, to ensure that only adjacent pixels with similar intensities affect the current pixel value. (x, y) represents the coordinates of the pixel, (x', y') represents the coordinates of the adjacent pixels involved in the calculation when applying the filter, and I filtered (x,y) represents the pixel value of the image at coordinate (x,y) after bilateral filtering.

3. The video recognition method based on artificial intelligence according to claim 1, characterized in that: The input of the generative network is the feature vector of each frame of video at time t, and then multiple convolutional layers are used, each layer is followed by batch normalization and ReLU activation function, and the last layer uses tanh activation function to output the stylized image, which is expressed as follows: I styled =tanh(Conv(BN(ReLU(Conv(…V t …))))) The input of the discriminant network includes I styled And the original frame I, use multiple convolution layers for feature extraction, each layer of convolution is connected to the normalization and LeakyReLU activation function, and the last layer uses the sigmoid function to output the probability, which is expressed as follows: P real =σ(Conv(LReLU(BN(Conv(…I styled ,I…))))) Among them, σ represents the sigmoid activation function.

4. The video recognition method based on artificial intelligence according to claim 3 is characterized in that: In S303, the progressive training method means first training the network at a low resolution and gradually transitioning to a high resolution.

5. The video recognition method based on artificial intelligence according to claim 1, characterized in that: The cross entropy loss function is used to simultaneously optimize the accuracy of behavior recognition and prediction, which is expressed as follows: Among them, y c is the one-hot encoding of the true behavior label, is the probability distribution predicted by the model, and C is the number of behavior categories.

6. The video recognition method based on artificial intelligence according to claim 1, characterized in that: The step B specifically comprises the following steps: Regularly collect feedback data and process the collected data using statistical analysis and machine learning methods; Based on feedback data, define parameter adjustment strategies and dynamically adjust learning rates according to model performance changes; Perform comprehensive evaluations periodically and update the model's operating parameters based on the evaluation results, which are then fed back to the operating parameter adjustment function and the learning rate update strategy.

7. A video recognition system based on artificial intelligence, characterized in that: The system comprises: The adaptive framework building module is used to build an adaptive framework based on video content analysis, dynamically adjusting the model structure and parameters according to the complexity of real-time video frames, including: S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network; S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including: D t =||V t -V t-1 ||2 Among them, D t Indicates the change in the content of the frame between time t and t-1, V t Represents the feature vector of each frame of video at time t, V t-1 Represents the feature vector of each frame of video at time t-1; The average of these differences is calculated through a sliding window of fixed size W to obtain the scene complexity C t : S103, according to C t The comparison result with the preset threshold θ automatically adjusts the configuration of the network; S104, combining real-time performance feedback, using back propagation algorithm to adjust network weights and structural parameters; The algorithm design module is used to extract features from video frames by applying efficient image preprocessing technology and deep learning algorithm according to model parameters; wherein the image preprocessing technology includes illumination correction and noise filtering; the feature extraction from the video frame is specifically to extract edge information of the video frame using an improved Sobel operator, and then analyze image texture features using a local binary pattern operator, specifically: Among them, G x and G y Represent the horizontal and vertical gradients respectively, and G represents the final edge strength; Among them, g c represents the center pixel value, g p represents the neighborhood pixel value, P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel; The enhanced feature module is used to process the features extracted from the video frames using a generative adversarial network, stylize the key frames to obtain stylized video frames, and enhance the visual behavioral features, including: S301, respectively designing a generative adversarial network, including a generative network and a discriminative network; S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows: Among them, F k represents the feature extraction function for the key feature k, K represents the predefined associated feature set, λ k represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, I represents the original frame, and G(z) represents the output of the generator G when the input is the hidden variable z; The total loss function is expressed as follows: Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network; S303, training the generative adversarial network using a progressive training method; The behavior prediction module is used to design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network, specifically including: The hybrid neural network includes a feature fusion layer, a temporal fusion layer and a behavior recognition and prediction output layer; The feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of image, which is expressed as follows: F spatial =CNN(I styled ) Among them, F spatial represents the spatial feature vector extracted from the stylized image; The temporal fusion layer transforms the feature vectors F of consecutive frames. spatial As input, the temporal dependencies and dynamic changes are processed through the recurrent neural network module, which is expressed as follows: S t =RNN(F spatial ,S t-1 ) Among them, S t represents the hidden state at time t, carrying the accumulated information of past video frames; S t-1 represents the hidden state at time t-1; The behavior recognition and prediction output layer is combined with F spatial and S t Generate the final behavior recognition and behavior prediction output, expressed as follows: AND action =Softmax(Dense(S t )) AND predict =Softmax(Dense(S t ,F spatial )) Among them, Y action is the behavior recognition result of the current frame, Y predict It is the result of predicting future behavior based on current and past information; The method further comprises at least one of the following modules: The unsupervised learning algorithm module is used to integrate the unsupervised learning algorithm and dynamically adjust the models and parameters of the adaptive framework building module, algorithm design module, enhanced feature module and behavior prediction module based on real-time feedback. Specifically, it includes the following steps: A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames; Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect abnormal behaviors or salient events in video frames; Define the total loss function Reconstruction error and behavior recognition error The weighted sum of is calculated by introducing additional regularization terms to enhance the generalization and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights; Integrate dynamic elements to adapt to changing video content and optimize feature transformation networks; Introduce model adaptive adjustment strategies to automatically adjust network architecture or parameters based on currently detected behavior patterns and historical data; The feedback mechanism optimization module is used to establish a feedback mechanism, dynamically adjust strategies based on user and performance feedback, and continuously optimize strategy execution.

Images