Video data behavior detection method and system and electronic equipment

Abstract

The invention provides a video data behavior detection method and system and electronic equipment, and relates to the field of video data processing, and the method can adjust an iterative calculation process by using a learning rate parameter, obtains feature data contained in video data through a self-attention mechanism, and further improves the recognition precision of behavior detection. In addition, in the process of calculating the behavior occurrence probability, behavior probability data of multiple levels are considered, the robustness and accuracy of behavior detection are improved, and the accuracy of behavior detection can be ensured even in a complex service scene.

Description

Technical Field

[0001] The present invention relates to the field of video data processing, and in particular to a video data behavior detection method, system and electronic equipment. Background Art

[0002] Conducting behavioral detection on people or objects in video data, discovering abnormal behaviors of the subject, and handling abnormal behaviors through timely early warning processing is crucial to reducing the risk of the subject and improving the security of related systems. In actual scenarios, behavioral detection of video data mainly relies on preset static rules or thresholds to detect abnormal behaviors, but existing behavioral detection methods are incapable of coping with complex business scenarios, and there are still problems such as low accuracy of behavioral detection, high false alarm rate, and low processing efficiency. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a video data behavior detection method, system and electronic device, which utilizes a learning rate parameter to adjust the iterative calculation process, and obtains feature data contained in the video data through a self-attention mechanism, thereby further improving the recognition accuracy of behavior detection; in addition, in the process of calculating the probability of occurrence of a behavior, behavior probability data from multiple levels are considered, thereby improving the robustness and accuracy of behavior detection, and the accuracy of behavior detection can be guaranteed even in complex business scenarios.

[0004] In a first aspect, the present invention provides a method for detecting video data behavior, the method comprising: Acquire the video data to be detected, and preprocess the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data; After performing feature extraction processing on the preprocessed data according to the spatial feature parameters and the temporal feature parameters corresponding to the preprocessed data, first feature data corresponding to the preprocessed data is obtained; After performing self-attention calculation on the first feature data, second feature data corresponding to the preprocessed data is obtained, and after performing iterative calculation on the second feature data based on a preset learning rate parameter, a plurality of behavior probability data corresponding to the preprocessed data is output; The probability average of the behavior probability data is calculated, and the probability average is used to determine the occurrence probability of the behavior in the video data.

[0005] Optionally, obtaining the video data to be detected, and preprocessing the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data, including: Determine a cleaning processing strategy and an enhancement processing strategy corresponding to the video data according to the preprocessing strategy; Determine a frame extraction strategy and a normalization strategy for the video data based on the cleaning processing strategy, and obtain cleaned data corresponding to the video data after processing the video data using the frame extraction strategy and the normalization strategy; Determine a denoising strategy and an expansion strategy for the video data based on the enhancement processing strategy, and obtain enhanced data corresponding to the video data after processing the video data using the denoising strategy and the expansion strategy; Determine the preprocessed data using cleaned data and augmented data.

[0006] Optionally, after performing feature extraction processing on the preprocessed data according to the spatial feature parameters and the temporal feature parameters corresponding to the preprocessed data, obtaining first feature data corresponding to the preprocessed data includes: Obtain the feature processing strategy corresponding to the preprocessed data; Determine spatial feature parameters and temporal feature parameters using feature processing strategies; Perform feature extraction on the preprocessed data based on the spatial feature parameters and the temporal feature parameters to obtain the spatial features and temporal features corresponding to the video data; The first feature data is determined according to the spatial feature and the temporal feature.

[0007] Optionally, obtain the feature processing strategy corresponding to the preprocessed data, including: Determine the convolutional neural network corresponding to the video data, and obtain the convolutional layer contained in the convolutional neural network; Obtain a feature extraction layer included in the convolution layer, and determine a feature extraction strategy based on the feature extraction layer; Obtain a feature fusion layer contained in the convolution layer, and determine a feature fusion strategy based on the feature fusion layer; A feature mining layer included in the convolution layer is obtained, and a feature mining strategy is determined based on the feature mining layer.

[0008] Optionally, performing self-attention calculation on the first feature data to obtain second feature data corresponding to the preprocessed data includes: Obtaining a spatial characteristic parameter and a temporal characteristic parameter corresponding to the first characteristic data; Performing self-attention calculation on the first feature data using the spatial feature parameter and the temporal feature parameter to obtain context feature data corresponding to the preprocessed data; The characteristic value corresponding to the preprocessed data is calculated according to the context characteristic data, and the context characteristic data is updated and calculated using the characteristic value to obtain the second characteristic data.

[0009] Optionally, after iteratively calculating the second feature data based on a preset learning rate parameter, a plurality of behavior probability data corresponding to the preprocessed data is output, including: Acquire a verification set and a training set corresponding to the video data based on the second feature data; The prediction value and loss value of the second feature data are calculated using the validation set and the training set, and the loss value of the second feature data is iteratively calculated using the learning rate parameter; When the loss value meets the preset threshold relationship, the iterative calculation process is stopped, and the behavior probability data is determined using the current prediction value.

[0010] Optionally, the loss value of the second feature data is iteratively calculated using a learning rate parameter, including: Based on the loss value, obtain a first loss value corresponding to the validation set and a second loss value corresponding to the training set; Determine an adjustment strategy for a learning rate parameter according to a numerical relationship between the first loss value and the second loss value, and update the learning rate parameter using the adjustment strategy; The loss value is iteratively calculated using the updated learning rate parameter.

[0011] Optionally, calculating a probability average of the behavior probability data, and using the probability average to determine the occurrence probability of the behavior in the video data, includes: Obtaining type parameters corresponding to the behavior probability data, and determining weight values ​​corresponding to the behavior probability data based on the type parameters; calculating a weighted average of the behavior probability data using the weight values, and determining the probability average as the probability average; The probability of behavior occurrence in the video data is determined using the probability average.

[0012] In a second aspect, the present invention provides a video data behavior detection system, the system comprising: The preprocessing data acquisition module is used to acquire the video data to be detected, and obtain the preprocessing data corresponding to the video data after preprocessing the video data according to the preset preprocessing strategy; A feature data extraction module, used to obtain first feature data corresponding to the preprocessed data after performing feature extraction processing on the preprocessed data according to the spatial feature parameters and temporal feature parameters corresponding to the preprocessed data; A behavior probability data calculation module, used to obtain second feature data corresponding to the preprocessed data after performing self-attention calculation on the first feature data, and output multiple behavior probability data corresponding to the preprocessed data after performing iterative calculation on the second feature data based on a preset learning rate parameter; The behavior occurrence probability determination module is used to calculate the probability average of the behavior probability data and use the probability average to determine the behavior occurrence probability in the video data.

[0013] In a third aspect, the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the video data behavior detection method provided in the first aspect.

[0014] In a fourth aspect, the present invention further provides a storage medium storing computer executable instructions. When the computer executable instructions are called and executed by a processor, the computer executable instructions prompt the processor to implement the steps of the video data behavior detection method provided in the first aspect.

[0015] The present invention provides a method, system and electronic device for detecting behavior of video data. In the process of detecting behavior of video data, the method first obtains the video data to be detected, preprocesses the video data according to a preset preprocessing strategy, and obtains the preprocessed data corresponding to the video data; then extracts the preprocessed data according to the spatial feature parameters and time feature parameters corresponding to the preprocessed data, and obtains the first feature data corresponding to the preprocessed data; then performs self-attention calculation on the first feature data to obtain the second feature data corresponding to the preprocessed data, and iteratively calculates the second feature data based on the preset learning rate parameter to output multiple behavior probability data corresponding to the preprocessed data; finally, the probability average of the behavior probability data is calculated, and the probability average is used to determine the probability of occurrence of the behavior in the video data. The method can adjust the iterative calculation process by using the learning rate parameter, and obtains the feature data contained in the video data by the self-attention mechanism, so as to further improve the recognition accuracy of the behavior detection; in addition, in the calculation process of calculating the probability of occurrence of the behavior, the behavior probability data from multiple levels are considered, so as to improve the robustness and accuracy of the behavior detection, and the accuracy of the behavior detection can be guaranteed even in complex business scenarios.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a video data behavior detection method provided by an embodiment of the present invention; Figure 2 A flowchart of step S101 in a video data behavior detection method provided by an embodiment of the present invention; Figure 3 A flowchart of step S102 in a video data behavior detection method provided by an embodiment of the present invention; Figure 4 A flowchart of step S301 in a video data behavior detection method provided by an embodiment of the present invention; Figure 5 A flowchart of obtaining second feature data corresponding to preprocessed data after performing self-attention calculation on first feature data in a video data behavior detection method provided by an embodiment of the present invention; Figure 6 A flowchart of a method for detecting behavior of video data provided by an embodiment of the present invention, in which a second feature data is iteratively calculated based on a preset learning rate parameter and then a plurality of behavior probability data corresponding to the preprocessed data is output; Figure 7 A flowchart of iteratively calculating the loss value of second feature data using a learning rate parameter in a video data behavior detection method provided by an embodiment of the present invention; Figure 8 A flowchart of step S104 in a video data behavior detection method provided by an embodiment of the present invention; Fig. 9 A flowchart of a second video data behavior detection method provided by an embodiment of the present invention; Fig.10 A flowchart of a third video data behavior detection method provided by an embodiment of the present invention; Fig.11 A flowchart of using a learning rate to adjust a model in a video data behavior detection method provided by an embodiment of the present invention; Fig.12 A flowchart for predicting feature extraction results in a video data behavior detection method provided by an embodiment of the present invention; Fig.13 A flowchart of a fourth video data behavior detection method provided by an embodiment of the present invention; Fig.14 A flowchart of a fifth video data behavior detection method provided by an embodiment of the present invention; Fig.15 A schematic diagram of the structure of a video data behavior detection system provided by an embodiment of the present invention; Fig.16 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0020] icon: 1510-preprocessing data acquisition module; 1520-feature data extraction module; 1530-behavior probability data calculation module; 1540-behavior occurrence probability determination module; 101 - processor; 102 - memory; 103 - bus; 104 - communication interface. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Behavior detection of people or objects in video data, and finding abnormal behaviors of the subject, and subsequent processing of abnormal behaviors through timely early warning processing are crucial to reducing the risk of the subject and improving the security of related systems. In actual scenarios, behavior detection of video data mainly relies on preset static rules or thresholds to detect abnormal behaviors. This method has the following limitations: Single rules and high false positive rate. Since the rules are fixed, false positives are prone to occur when faced with complex and diverse behavior patterns.

[0023] Poor scalability, difficulty in processing large amounts of data, and inability to adapt to the growth of large-scale video data, resulting in decreased system performance.

[0024] Lack of self-learning ability, low prediction accuracy, and inability to self-adjust and optimize based on historical data.

[0025] It can be seen that the existing behavior detection methods are powerless in the face of complex business scenarios, and there are still problems such as low behavior detection accuracy, high false alarm rate, and low processing efficiency. Based on this, the present invention implements a video data behavior detection method, system, and electronic device. The method uses a learning rate parameter to adjust the iterative calculation process, and obtains feature data contained in the video data through a self-attention mechanism to further improve the recognition accuracy of behavior detection; in addition, in the process of calculating the probability of behavior occurrence, behavior probability data from multiple levels are considered, which improves the robustness and accuracy of behavior detection, and can ensure the accuracy of behavior detection even in complex business scenarios.

[0026] To facilitate understanding of this embodiment, a video data behavior detection method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: Step S101, obtaining video data to be detected, and preprocessing the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data; Step S102, performing feature extraction processing on the preprocessed data according to the spatial feature parameters and temporal feature parameters corresponding to the preprocessed data, and obtaining first feature data corresponding to the preprocessed data; Step S103, performing self-attention calculation on the first feature data to obtain second feature data corresponding to the preprocessed data, and performing iterative calculation on the second feature data based on a preset learning rate parameter to output multiple behavior probability data corresponding to the preprocessed data; Step S104, calculating the probability average of the behavior probability data, and using the probability average to determine the occurrence probability of the behavior in the video data.

[0027] In the process of video data behavior detection, the video data to be detected is first preprocessed to ensure the quality of the data to be processed; then the preprocessed data is feature extracted, and the surface features and depth features of the video data are captured using spatial feature parameters and temporal feature parameters, thereby determining the first feature data corresponding to the preprocessed data. After the first feature data is obtained, it is calculated using the self-attention mechanism, and the feature context relationship in the preprocessed data is obtained based on the calculation result to obtain the second feature data.

[0028] After the second feature data is obtained, it is iteratively calculated using the learning rate parameter. The updated second feature data is obtained by adjusting the learning rate parameter, and the probability of occurrence of the behavior of the relevant subject in the video data is calculated based on the second feature data. A plurality of behavior probability data are obtained from the perspective of different behavior types, and then the probability average of these behavior probability data is calculated to finally determine the probability of occurrence of the behavior.

[0029] Optionally, the step S101 of obtaining the video data to be detected and preprocessing the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data is as follows: Figure 2 As shown, including: Step S201, determining a cleaning processing strategy and an enhancement processing strategy corresponding to the video data according to a preprocessing strategy; Step S202, determining a frame extraction strategy and a normalization strategy for the video data based on the cleaning processing strategy, and processing the video data using the frame extraction strategy and the normalization strategy to obtain cleaned data corresponding to the video data; Step S203, determining a denoising strategy and an expansion strategy for the video data based on the enhancement processing strategy, and processing the video data using the denoising strategy and the expansion strategy to obtain enhanced data corresponding to the video data; Step S204: determine pre-processed data using the cleaned data and enhanced data.

[0030] In the process of preprocessing the video data, the cleaning processing strategy and the enhancement processing strategy are first obtained according to the preset preprocessing strategy, and the video data is processed from two levels. The cleaning processing strategy is mainly to simplify the video data, and the frame extraction strategy and the normalization strategy of the video data are obtained through the cleaning processing strategy. After the video data is simplified by using the obtained frame extraction strategy and normalization strategy, the cleaning data corresponding to the video data is obtained.

[0031] Similarly, the enhancement processing strategy is mainly to improve the video data. The denoising strategy and expansion strategy of the video data are obtained through the enhancement processing strategy, and the video data is enhanced by using the obtained denoising strategy and expansion strategy to obtain the enhanced data corresponding to the video data.

[0032] After the cleaned data and enhanced data are obtained, they are determined as preprocessed data corresponding to the video data. On this basis, after feature extraction processing is performed on the preprocessed data according to the spatial feature parameters and temporal feature parameters corresponding to the preprocessed data, step S102 of obtaining first feature data corresponding to the preprocessed data is performed. Figure 3 As shown, including: Step S301, obtaining a feature processing strategy corresponding to the preprocessed data; Step S302, determining spatial feature parameters and temporal feature parameters using a feature processing strategy; Step S303, extracting features from the preprocessed data based on the spatial feature parameters and the temporal feature parameters to obtain spatial features and temporal features corresponding to the video data; Step S304: determining first feature data according to the spatial feature and the temporal feature.

[0033] In the specific implementation process, firstly, the feature processing strategy corresponding to the preprocessed data is obtained, and feature extraction, feature fusion, feature mining, etc. are performed on the preprocessed data through the feature processing strategy, so as to determine the spatial feature parameters and time feature parameters corresponding to the preprocessed data, and then the spatial features and time features corresponding to the video data are obtained after feature extraction of the preprocessed data using the above parameters, and the spatial features and time features corresponding to the video data are determined as the first feature data. Optionally, step S301 of obtaining the feature processing strategy corresponding to the preprocessed data, such as Figure 4 As shown, including: Step S401, determining a convolutional neural network corresponding to the video data, and obtaining a convolutional layer included in the convolutional neural network; Step S402, obtaining a feature extraction layer included in the convolution layer, and determining a feature extraction strategy based on the feature extraction layer; Step S403, obtaining a feature fusion layer included in the convolution layer, and determining a feature fusion strategy based on the feature fusion layer; Step S404, obtaining a feature mining layer included in the convolution layer, and determining a feature mining strategy based on the feature mining layer.

[0034] In actual scenarios, relevant convolutional neural networks can be used to determine the corresponding feature processing strategies, and the relevant convolutional layers in the convolutional neural network can be used to realize multi-level feature extraction of video data. Specifically, the feature extraction strategy can be determined by obtaining the feature extraction layer contained in the convolutional layer, the feature fusion strategy can be determined by obtaining the feature fusion layer contained in the convolutional layer, and the feature mining strategy can be determined by obtaining the feature mining layer contained in the convolutional layer. Using multi-layer convolutional neural networks to realize feature extraction can identify complex and hidden behaviors contained in video data.

[0035] Optionally, the first feature data is subjected to self-attention calculation to obtain second feature data corresponding to the preprocessed data, such as Figure 5 As shown, including: Step S501, obtaining a spatial characteristic parameter and a temporal characteristic parameter corresponding to the first characteristic data; Step S502, performing self-attention calculation on the first feature data using the spatial feature parameter and the temporal feature parameter to obtain context feature data corresponding to the preprocessed data; Step S503, calculating a characteristic value corresponding to the preprocessed data according to the context characteristic data, and using the characteristic value to update the context characteristic data to obtain second characteristic data.

[0036] The acquisition process of the second feature data is realized through the self-attention mechanism, which is used to capture important information in the video data. Specifically, the spatial feature parameters and temporal feature parameters corresponding to the first feature data are first obtained, and then the first feature data is self-attentionally calculated using the above parameters to obtain the context feature data between the relevant features in the preprocessed data, thereby capturing the long-distance dependency in the video data and improving the recognition accuracy of abnormal behavior.

[0037] After the context feature data is acquired, the data is used to calculate the eigenvalues ​​corresponding to the preprocessed data, and the obtained eigenvalues ​​are used to update the context feature data, thereby using the self-attention mechanism to enhance the understanding of the relationship in the context feature data, and finally obtain the updated second feature data.

[0038] Optionally, after iterative calculation of the second feature data based on a preset learning rate parameter, multiple behavior probability data corresponding to the preprocessed data are output, such as Figure 6 As shown, including: Step S601, obtaining a verification set and a training set corresponding to the video data based on the second feature data; Step S602, using the validation set and the training set to calculate the predicted value and loss value of the second feature data, and using the learning rate parameter to iteratively calculate the loss value of the second feature data; Step S603, when the loss value satisfies the preset threshold relationship, the iterative calculation process is stopped, and the behavior probability data is determined using the current prediction value.

[0039] Taking the convolutional neural network model as an example, during the iteration process, the validation set and the training set are determined based on the second feature data, and the prediction value and the loss value of the second feature data are calculated using the validation set and the training set. Specifically, the prediction value of the second feature data can be calculated using the training set, and then the loss value of the second feature data can be calculated, and then the loss value can be evaluated and calculated using the validation set, and then the learning rate parameter can be used for iteration until the loss value meets the preset threshold relationship, and the iterative calculation process is stopped, and the final behavior probability data is determined using the current prediction value.

[0040] Optionally, the loss value of the second feature data is iteratively calculated using the learning rate parameter, such as Figure 7 As shown, including: Step S701, obtaining a first loss value corresponding to the validation set and a second loss value corresponding to the training set based on the loss value; Step S702, determining an adjustment strategy for a learning rate parameter according to a numerical relationship between the first loss value and the second loss value, and updating the learning rate parameter using the adjustment strategy; Step S703, iteratively calculate the loss value using the updated learning rate parameter.

[0041] During the iterative calculation of the loss value of the second feature data using the learning rate parameter, the value of the learning rate can be dynamically adjusted. The dynamic learning rate adjustment is based on the loss changes of the training set and the validation set. By comparing the loss values ​​of the current and previous rounds, the value of the learning rate is adjusted to avoid overfitting or underfitting. Specifically, based on the loss value, the first loss value corresponding to the validation set and the second loss value corresponding to the training set are obtained, and then the size of the two is determined; if the first loss value is greater than the second loss value, it indicates that the loss of the validation set has increased, indicating that the iterative process is overfitting and the learning rate needs to be reduced; if the first loss value is less than the second loss value, it indicates that the loss of the validation set has decreased, indicating that the iterative process is underfitting and the learning rate needs to be increased.

[0042] Optionally, the step S104 of calculating the probability average of the behavior probability data and using the probability average to determine the occurrence probability of the behavior in the video data is as follows: Figure 8 As shown, including: Step S801, obtaining type parameters corresponding to the behavior probability data, and determining weight values ​​corresponding to the behavior probability data based on the type parameters; Step S802, calculating a weighted average of the behavior probability data using the weight value, and determining the probability average as the probability average; Step S803: determine the occurrence probability of the behavior in the video data using the probability average value.

[0043] The calculation process of the probability of occurrence of a behavior can be implemented by calculating the average value or by weighted average. Specifically, the corresponding weight value needs to be considered in the calculation process of the weighted average. The weight value is determined by the type parameter corresponding to the behavior probability data, and then the weight value is used to calculate the weighted average value of the behavior probability data to determine the final probability average value.

[0044] With respect to the video data behavior detection method mentioned in the above embodiment, the process of abnormal behavior detection in behavior video data is introduced below. Fig. 9 The second video data behavior detection method flow chart shown in the figure generally includes the following steps: Data preprocessing: preprocess the input video data, including frame extraction, normalization, denoising and other operations; Feature extraction: Use convolutional neural networks to extract features from video data to obtain spatial and temporal features; Feature optimization: Optimize the extracted features through the self-attention mechanism to improve the expressiveness of key features; Model training: Based on feature optimization, the model is trained using a dynamic learning rate adjustment strategy to ensure rapid convergence and efficient learning of the model; Anomaly prediction: Use the trained model to predict abnormal behavior of video data; Result integration: Integrate the prediction results of multiple independently trained models to generate the final abnormal behavior prediction results.

[0045] Fig. 9 The convolutional layer (CNN) in is used for feature extraction, the self-attention mechanism is used to optimize feature representation, and the dynamic learning rate adjustment is used to optimize model training. Finally, the prediction results are output through multi-model integration. Fig.10 The flowchart of the third video data behavior detection method is shown in Figure 1, which shows the processing flow from input video data to the final abnormal behavior prediction result. The process includes data preprocessing, feature extraction, feature optimization, model training and prediction, and result integration.

[0046] During model training and optimization, it is necessary to adjust the learning rate. Fig.11 The flowchart shown is a method for detecting behavior in video data using learning rate to adjust the model; the learning rate is dynamically adjusted according to the change of the loss function to ensure that the model converges stably and efficiently during the training process.

[0047] like Fig.12 The flowchart of predicting feature extraction results in a method for detecting behavior in video data is shown in the figure; the figure shows the architecture of implementing abnormal behavior prediction by integrating multiple neural network models. The integration layer generates a comprehensive prediction result through weighted average or voting mechanism based on the prediction results of different models.

[0048] like Fig.13 The flowchart of the fourth method for detecting behavior in video data is shown in Figure 1. The input data in this process first undergoes complex data preprocessing, including data cleaning, normalization, and expansion, to ensure the quality of the data received by the model. Then, the data is subjected to deep feature extraction through a multi-layer convolutional neural network (CNN), which not only captures surface features but also captures deep patterns. Next, the self-attention mechanism further enhances the model's understanding of contextual relationships, ensuring more accurate detection of abnormal behaviors. Finally, the predicted output is given in the form of a probability distribution, and combined with a dynamic adjustment and feedback mechanism, the model is automatically optimized and updated to make it highly adaptable and robust.

[0049] like Fig.14The flowchart of the fifth video data behavior detection method shown in the figure is different from the above in that this process is more detailed and complex, showing a multi-level data processing process, and reflecting the application and improvement of advanced technologies through the interaction of multiple connection lines and subsystems. This structure can more accurately and efficiently identify abnormal behaviors, significantly improving the adaptability and robustness of the video data behavior detection process.

[0050] The pseudo code of the above video data behavior detection method is as follows. In the initialization and training phase of the model, a neural network model is first defined and the initial learning rate is set. During the training process, the model calculates the predicted value through forward propagation, and performs back propagation according to the result of the loss function to update the weight. After each round of training, the learning rate is dynamically adjusted to adapt to the progress of training.

[0051] def initialize_model(): model = NeuralNetwork()# Initialize the neural network model learning_rate = initial_learning_rate # Set the initial learning rate return model, learning_rate def train_model(model, training_data, training_labels, validation_data, validation_labels, learning_rate): for epoch in range(num_epochs): # Forward propagation: calculate predicted values predictions = model.forward(training_data) # Calculate loss training_loss = compute_loss(predictions, training_labels) # Backpropagation: Calculate gradients and update weights gradients = model.backward(predictions, training_labels) model.update_weights(gradients, learning_rate) # Evaluate on the validation set validation_predictions = model.forward(validation_data) validation_loss = compute_loss(validation_predictions, validation_labels) # Dynamically adjust the learning rate learning_rate = adjust_learning_rate(training_loss, validation_loss,learning_rate) # Output training progress print(f"Epoch {epoch + 1} / {num_epochs} - Training Loss: {training_loss:.4f} - Validation Loss: {validation_loss:.4f}") return model The dynamic learning rate is adjusted based on the loss changes of the training set and the validation set. By comparing the losses of the current and previous rounds, the learning rate can be appropriately increased or decreased to avoid overfitting or underfitting.

[0052] def adjust_learning_rate(training_loss, validation_loss, learning_rate): if validation_loss>training_loss: # If the validation set loss increases, it means that the model may be overfitting, reduce the learning rate learning_rate = decrease_learning_rate(learning_rate) elif validation_loss <training_loss: # If the validation set loss decreases, it means the model may be underfitting, increase the learning rate learning_rate = increase_learning_rate(learning_rate) return learning_rate Data preprocessing and enhancement are crucial during training, especially when facing multi-task learning. The following pseudo code shows how to process the dataset to enhance the generalization ability of the model.

[0053] def preprocess_data(raw_data): # Data cleaning and standardization cleaned_data = clean_data(raw_data) normalized_data = normalize(cleaned_data) # Data enhancement: such as random cropping, rotation, flipping, etc. augmented_data = data_augmentation(normalized_data) return augmented_data In order to verify the generalization ability of the model, an unseen data set (test set) is used for evaluation. At the same time, the generalization ability of the model is further improved through methods such as cross-validation.

[0054] def evaluate_model(model, test_data, test_labels): # Evaluate the generalization ability of the model through the test set test_predictions = model.forward(test_data) test_loss = compute_loss(test_predictions, test_labels) test_accuracy = compute_accuracy(test_predictions, test_labels) print(f"Test Loss: {test_loss:.4f} - Test Accuracy: {test_accuracy:.2f}%") return test_loss, test_accuracy def k_fold_cross_validation(model, data, labels, k=5): # k-fold cross validation: used to evaluate the generalization performance of the model total_accuracy = 0 total_loss = 0 for fold in range(k): # Divide the data into k subsets, select one of them as the validation set, and the rest as the training set training_data, validation_data = split_data(data, fold, k) training_labels, validation_labels = split_data(labels, fold, k) # Train the model model = train_model(model, training_data, training_labels,validation_data, validation_labels, initial_learning_rate) # Validate model val_loss, val_accuracy = evaluate_model(model, validation_data,validation_labels) total_loss += val_loss total_accuracy += val_accuracy # Calculate average loss and accuracy average_loss = total_loss / k average_accuracy = total_accuracy / k print(f"Average Validation Loss: {average_loss:.4f} - AverageValidation Accuracy: {average_accuracy:.2f}%") return average_loss, average_accuracy Multi-task learning enables multiple models to share the learning process, thereby improving overall performance. In the process of model integration, the prediction results of each model are weighted averaged to improve the accuracy and robustness of the final prediction.

[0055] def multi_task_learning(models, data, labels): predictions = [] for model in models: # Make predictions for each model pred = model.predict(data) predictions.append(pred) # Integrate the prediction results of multiple models final_prediction = integrate_predictions(predictions) return final_prediction def integrate_predictions(predictions): # Integrate the prediction results of multiple models by weighted average weights = [0.2, 0.3, 0.5] # Example weights weighted_sum = sum(weight * pred for weight, pred in zip(weights,predictions)) final_prediction = weighted_sum / sum(weights) return final_prediction The introduction of convolutional neural networks and multi-model integration can significantly improve the effect of anomaly detection. First, adaptive learning is achieved. This method can make adaptive adjustments according to data changes. It has strong adaptability and can handle different types of data and scenarios. Secondly, multi-model integration is achieved. By integrating the prediction results of multiple models, the robustness and reliability of anomaly detection are improved, and the false alarm rate is reduced.

[0056] Through the verification of multiple actual cases, the present invention is superior to the traditional technology in the following aspects: Accuracy: The accuracy of anomaly detection has been improved by 30%, effectively reducing missed reports and false positives.

[0057] Processing efficiency: Data processing efficiency has been increased by 50%, enabling it to handle larger-scale real-time data streams.

[0058] False alarm rate: The false alarm rate was reduced by 40%, which greatly reduced the workload of managers and improved the practicality and reliability of the overall process.

[0059] Through dynamic learning rate adjustment, self-attention mechanism and multi-model integration, the accuracy and processing efficiency of abnormal behavior prediction are significantly improved. This method is suitable for anomaly detection tasks in various fields, such as intelligent monitoring systems, financial data security, etc.

[0060] In summary, this method uses the learning rate parameter to adjust the iterative calculation process, and obtains the feature data contained in the video data through the self-attention mechanism, further improving the recognition accuracy of behavior detection; in addition, in the process of calculating the probability of behavior occurrence, the behavior probability data at multiple levels are considered, which improves the robustness and accuracy of behavior detection, and the accuracy of behavior detection can be guaranteed even in complex business scenarios.

[0061] Corresponding to the above-mentioned video data behavior detection method embodiment, the embodiment of the present invention also provides a video data behavior detection system, such as Fig.15 As shown, the system includes: The preprocessing data acquisition module 1510 is used to acquire the video data to be detected, and obtain the preprocessing data corresponding to the video data after preprocessing the video data according to a preset preprocessing strategy; The feature data extraction module 1520 is used to obtain first feature data corresponding to the preprocessed data after performing feature extraction processing on the preprocessed data according to the spatial feature parameters and temporal feature parameters corresponding to the preprocessed data; The behavior probability data calculation module 1530 is used to obtain second feature data corresponding to the preprocessed data after performing self-attention calculation on the first feature data, and output multiple behavior probability data corresponding to the preprocessed data after performing iterative calculation on the second feature data based on a preset learning rate parameter; The behavior occurrence probability determination module 1540 is used to calculate the probability average of the behavior probability data and determine the behavior occurrence probability in the video data using the probability average.

[0062] From the video data behavior detection system mentioned in the above embodiment, it can be seen that the system can adjust the iterative calculation process using the learning rate parameter, and obtain the feature data contained in the video data through the self-attention mechanism, so as to further improve the recognition accuracy of behavior detection; in addition, in the process of calculating the probability of occurrence of behavior, the behavior probability data from multiple levels are considered, which improves the robustness and accuracy of behavior detection, and can ensure the accuracy of behavior detection even in complex business scenarios.

[0063] The video data behavior detection system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned video data behavior detection method embodiment. For the sake of brief description, for parts not mentioned in the method embodiment, reference can be made to the corresponding contents in the aforementioned video data behavior detection method embodiment.

[0064] This embodiment also provides an electronic device. The structural diagram of the electronic device is as follows: Fig.16 As shown, the device includes a processor 101 and a memory 102; wherein the memory 102 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the steps of the above-mentioned video data behavior detection method.

[0065] Fig.16 The electronic device shown further includes a bus 103 and a communication interface 104 , and the processor 101 , the communication interface 104 and the memory 102 are connected via the bus 103 .

[0066] The memory 102 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The bus 103 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.16 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0067] The communication interface 104 is used to connect to at least one user terminal and other network units through a network interface, and send the encapsulated IPv4 message or IPv4 message to the user terminal through the network interface.

[0068] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and completes the steps of the method of the above embodiment in combination with its hardware.

[0069] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the video data behavior detection method in the above embodiment are executed.

[0070] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0071] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0073] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0074] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A video data behavior detection method, characterized in that: The method comprises: Acquire the video data to be detected, and preprocess the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data; After performing feature extraction processing on the preprocessed data according to the spatial feature parameters and the temporal feature parameters corresponding to the preprocessed data, first feature data corresponding to the preprocessed data is obtained; Performing self-attention calculation on the first feature data to obtain second feature data corresponding to the preprocessed data, and performing iterative calculation on the second feature data based on a preset learning rate parameter to output a plurality of behavior probability data corresponding to the preprocessed data; The probability average of the behavior probability data is calculated, and the occurrence probability of the behavior in the video data is determined using the probability average.

2. The video data behavior detection method according to claim 1, characterized in that: Acquiring video data to be detected, and preprocessing the video data according to a preset preprocessing strategy to obtain preprocessed data corresponding to the video data, including: Determine a cleaning processing strategy and an enhancement processing strategy corresponding to the video data according to the preprocessing strategy; Determining a frame extraction strategy and a normalization strategy for the video data based on the cleaning processing strategy, and processing the video data using the frame extraction strategy and the normalization strategy to obtain cleaning data corresponding to the video data; Determining a denoising strategy and an expansion strategy for the video data based on the enhancement processing strategy, and obtaining enhanced data corresponding to the video data after processing the video data using the denoising strategy and the expansion strategy; The pre-processed data is determined using the cleaned data and the enhanced data.

3. The video data behavior detection method according to claim 1, characterized in that: After performing feature extraction processing on the preprocessed data according to the spatial feature parameters and the temporal feature parameters corresponding to the preprocessed data, obtaining first feature data corresponding to the preprocessed data includes: Obtaining a feature processing strategy corresponding to the preprocessed data; Determining the spatial feature parameter and the temporal feature parameter using the feature processing strategy; Performing feature extraction on the preprocessed data based on the spatial feature parameters and the temporal feature parameters to obtain spatial features and temporal features corresponding to the video data; The first feature data is determined according to the spatial feature and the temporal feature.

4. The video data behavior detection method according to claim 3, characterized in that: Obtaining a feature processing strategy corresponding to the preprocessed data includes: Determine a convolutional neural network corresponding to the video data, and obtain a convolutional layer included in the convolutional neural network; Obtaining a feature extraction layer included in the convolutional layer, and determining a feature extraction strategy based on the feature extraction layer; Obtaining a feature fusion layer included in the convolution layer, and determining a feature fusion strategy based on the feature fusion layer; A feature mining layer included in the convolution layer is obtained, and a feature mining strategy is determined based on the feature mining layer.

5. The video data behavior detection method according to claim 1, characterized in that: The first feature data is subjected to self-attention calculation to obtain second feature data corresponding to the preprocessed data, including: Acquire the spatial characteristic parameter and the temporal characteristic parameter corresponding to the first characteristic data; Performing self-attention calculation on the first feature data using the spatial feature parameter and the temporal feature parameter to obtain context feature data corresponding to the preprocessed data; The characteristic value corresponding to the preprocessed data is calculated according to the context characteristic data, and the context characteristic data is updated and calculated using the characteristic value to obtain the second characteristic data.

6. The video data behavior detection method according to claim 1, characterized in that: After iteratively calculating the second feature data based on a preset learning rate parameter, a plurality of behavior probability data corresponding to the preprocessed data is output, including: Acquire a verification set and a training set corresponding to the video data based on the second feature data; Calculate the predicted value and loss value of the second feature data using the validation set and the training set, and iteratively calculate the loss value of the second feature data using the learning rate parameter; When the loss value satisfies a preset threshold value, the iterative calculation process is stopped, and the behavior probability data is determined using the current prediction value.

7. The video data behavior detection method according to claim 6, characterized in that: Iteratively calculating the loss value of the second feature data using the learning rate parameter includes: Based on the loss value, obtain a first loss value corresponding to the validation set and a second loss value corresponding to the training set; Determining an adjustment strategy for the learning rate parameter according to a numerical relationship between the first loss value and the second loss value, and updating the learning rate parameter using the adjustment strategy; The loss value is iteratively calculated using the updated learning rate parameter.

8. The video data behavior detection method according to claim 1, characterized in that: Calculating the probability average of the behavior probability data, and determining the occurrence probability of the behavior in the video data using the probability average, including: Acquire a type parameter corresponding to the behavior probability data, and determine a weight value corresponding to the behavior probability data based on the type parameter; calculating a weighted average of the behavior probability data using the weight values, and determining the probability average as the probability average; The probability of occurrence of the behavior in the video data is determined using the probability average.

9. A video data behavior detection system, characterized in that: The system comprises: A preprocessing data acquisition module is used to acquire the video data to be detected, and obtain the preprocessing data corresponding to the video data after preprocessing the video data according to a preset preprocessing strategy; A feature data extraction module, configured to perform feature extraction processing on the preprocessed data according to the spatial feature parameters and the temporal feature parameters corresponding to the preprocessed data, and obtain first feature data corresponding to the preprocessed data; A behavior probability data calculation module, configured to obtain second feature data corresponding to the preprocessed data after performing self-attention calculation on the first feature data, and output a plurality of behavior probability data corresponding to the preprocessed data after performing iterative calculation on the second feature data based on a preset learning rate parameter; The behavior occurrence probability determination module is used to calculate the probability average of the behavior probability data and determine the behavior occurrence probability in the video data using the probability average.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the video data behavior detection method according to any one of claims 1 to 8.

Images