Behavior detection model-based behavior recognition method, device and equipment suitable for fuzzy tag

Abstract

The invention relates to a behavior recognition method, device and equipment suitable for fuzzy labels and based on a behavior detection model, and relates to the technical field of behavior recognition, and the method comprises the steps: obtaining a confusion data sample, and training an auxiliary branch in a constructed initial behavior detection model according to the confusion data sample, and performing feature extraction and feature fusion training on the main path model based on the confusion data sample to obtain a main path fusion feature, performing multi-dimensional loss constraint on the main path model to obtain a plurality of loss functions, and completing collaborative distillation training of the auxiliary branch and the main path in the initial behavior detection model through the plurality of loss functions to obtain a main path fusion feature. And the target line detection model is obtained, feature recognition and behavior detection processing are carried out on the video stream through the target behavior detection model, a behavior recognition result is obtained, the influence of an original fuzzy label on the model performance is reduced through collaborative distillation learning, and the detection precision of the behavior detection model on confusion behaviors and the landing use effect are improved.

Description

Technical Field

[0001] The present application relates to the field of behavior recognition technology, and in particular to a behavior recognition method, device and equipment based on a behavior detection model suitable for fuzzy labels. Background Art

[0002] At present, as a key research direction in the field of computer vision, behavioral target detection has made significant progress in recent years driven by deep learning technology. Researchers have provided a basis for model training and evaluation by constructing and annotating a large number of behavioral datasets. In terms of model architecture, innovations continue to emerge, including the integration of convolutional neural networks, recurrent neural networks, and attention mechanisms to improve the accuracy and robustness of behavior recognition.

[0003] In practical application scenarios, such as behavior recognition in examination rooms, the system is usually required to be able to monitor the behavior of examinees in real time and quickly identify violations, such as cheating or whispering, to ensure the fairness of the examination. This requires the behavior detection model to be able to process the video stream in a very short time and accurately identify the relevant behavior. The detection accuracy of the behavior detection model in the existing scheme usually depends on the annotation quality of the data set (i.e., sample data). The model training is carried out using a high-quality data set so that the trained model can recognize various behaviors. However, the atomic action set of human behavior in the sample data usually has behaviors that are not obvious for a certain feature presented in the image due to the lack of information about the previous and next frame sequences. In addition, different annotators have different annotation standards, and the same sample data often gives different labels, and there are fuzzy labels (uncertain labels). As a result, no matter how strong the detection model is, it is impossible to train a high-precision model. Due to insufficient model accuracy, video streams with fuzzy behaviors cannot be detected with high precision. It can be seen that the existing behavior recognition method cannot accurately detect the input data with high precision, and the detection efficiency is low. Summary of the invention

[0004] The present application provides a behavior recognition method, device and equipment based on a behavior detection model suitable for fuzzy labels. In the model training stage, auxiliary branch training and main path collaborative distillation training are completed by controlling the weight of the loss function, thereby reducing the impact of the original fuzzy labels on the model performance. A high-precision behavior detection model can be trained for annotated data sets with poor quality, meeting actual use needs, effectively improving the detection accuracy and on-site use effect of the behavior detection model for confused label data sets, thereby improving the accuracy of the behavior detection algorithm and the behavior detection model, and solving the problem of insufficient behavior recognition detection accuracy caused by the inability to train a high-precision recognition model in existing behavior recognition methods.

[0005] In a first aspect, the present application provides a behavior recognition method based on a behavior detection model applicable to fuzzy labels, comprising:

[0006] Constructing an initial behavior detection model and obtaining an obfuscated data sample, wherein the initial behavior detection model comprises a main path model and at least one auxiliary branch, wherein the auxiliary branch corresponds one-to-one to a behavior category in the obfuscated data sample;

[0007] The auxiliary branch is trained according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category;

[0008] Based on the obfuscated data sample, feature extraction and feature fusion training are performed on the main path model to obtain main path fusion features;

[0009] According to the branch features, the main path fusion features and the potential distribution soft labels, multi-dimensional loss constraints are performed on the main path model to obtain a divergence loss function and a distillation loss function;

[0010] Based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained by using preset weight information to obtain a target behavior detection model, and the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model;

[0011] The target behavior detection model is used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior to obtain a behavior recognition result.

[0012] Optionally, the training of the auxiliary branch according to the obfuscated data sample to obtain the auxiliary branch loss function, branch features, and potential distribution soft labels corresponding to the behavior categories includes:

[0013] Extracting a sample image and a sample label corresponding to the sample image from the obfuscated data sample, and inputting the sample image and the sample label into a corresponding auxiliary branch;

[0014] The auxiliary branch performs classification prediction and feature extraction on the sample image and the sample label to obtain a predicted anchor frame, a predicted category corresponding to the predicted anchor frame, a potential distribution soft label of the obfuscated sample data on the behavior category, and branch features output by each auxiliary branch;

[0015] For the predicted anchor box and the predicted category, the sample image and the sample label are subjected to classification loss constraints by the auxiliary branch to obtain an auxiliary branch category loss function and an auxiliary branch anchor box regression loss function;

[0016] An auxiliary branch loss function is determined based on the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function.

[0017] Optionally, performing classification loss constraints on the sample image and the sample label through the auxiliary branch to obtain an auxiliary branch category loss function and an auxiliary branch anchor frame regression loss function includes:

[0018] According to the formula Determine the auxiliary branch category loss function;

[0019] According to the formula Determine the auxiliary branch anchor frame regression loss function;

[0020] in, is the auxiliary branch category loss function, p t is the confidence of the predicted box category, α is a parameter used to control the balance of positive and negative samples, and γ is a modulation factor parameter used to reduce the proportion of easy-to-separate samples in model training. is the auxiliary branch anchor box regression loss function, Pred bbox is the predicted anchor box, GT box To label the anchor box, CIoU loss is used to minimize the deviation between the predicted anchor box and the labeled anchor box.

[0021] Optionally, performing multi-dimensional loss constraints on the main path model according to the branch features, the main path fusion features, and the potential distribution soft labels to obtain a divergence loss function and a distillation loss function includes:

[0022] According to the potential distribution soft label, the main path model is subjected to spatial classification loss constraints by a preset main path detection algorithm to obtain a divergence loss function, where the divergence loss function is used to minimize the deviation between the confidence of the predicted target box of the main path model and the potential distribution soft label;

[0023] Based on the branch features and the main path fusion features, the main path model is subjected to loss constraints in the feature space dimension to obtain a distillation loss function.

[0024] Optionally, the step of performing spatial classification loss constraints on the main path model through a preset main path detection algorithm based on the potential distribution soft label to obtain a divergence loss function includes:

[0025] Obtaining a branch prediction target frame predicted by the auxiliary branch for the confused sample data, and obtaining a main path prediction target frame predicted by the main path model for the potential distribution soft label and a category confidence corresponding to the main path prediction target frame;

[0026] An intersection-over-union (IoU) matching is performed based on the first target box and the second target box, and a divergence loss function is calculated according to the category confidence.

[0027] Optionally, performing intersection-over-union matching based on the first target box and the second target box, and calculating a divergence loss function according to the category confidence includes:

[0028] According to the formula Calculate the divergence loss function, and according to Perform intersection and union matching;

[0029] Among them, D KL is the KL loss function, p(x) is the potential distribution soft label, and q(x) is the confidence distribution of the target box category predicted by the main path.

[0030] Optionally, performing loss constraints on the main path model in a feature space dimension based on the branch features and the main path fusion features to obtain a distillation loss function includes:

[0031] According to the formula Perform loss constraints on the feature space dimension to obtain the distillation loss function;

[0032] Among them, L SP is the distillation loss, G main is the main path fusion feature, G bi For each branch feature, the distillation loss function is used to make the target branch's ability to extract features closer to multiple auxiliary branches.

[0033] Optionally, the weight information includes auxiliary branch training weights and main path training weights, and the initial behavior detection model is trained based on the auxiliary branch loss function, the divergence loss function, and the distillation loss function through preset weight information, including:

[0034] according to Determine the auxiliary branch loss function;

[0035] according to Determine the main branch loss function;

[0036] according to Determine the calculation loss function and perform model training on the initial behavior detection model;

[0037] in, is the main branch category classification loss, ε is the auxiliary branch training weight, (1-ε) is the main path training weight, the auxiliary branch training weight is inversely proportional to the main path training weight, and is adjusted based on the training cycle of the initial behavior detection model.

[0038] In a second aspect, the present application provides a behavior recognition device based on a behavior detection model applicable to fuzzy labels, comprising:

[0039] A construction module, used to construct an initial behavior detection model and obtain an obfuscated data sample, wherein the initial behavior detection model includes a main path model and at least one auxiliary branch, and the auxiliary branch corresponds one-to-one to the behavior category in the obfuscated data sample;

[0040] A first training module is used to train the auxiliary branch according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category;

[0041] An extraction module, used for performing feature extraction and feature fusion training on the main path model based on the obfuscated data sample to obtain a main path fusion feature;

[0042] A loss constraint module, used to perform multi-dimensional loss constraints on the main path model according to the branch features, the main path fusion features and the potential distribution soft labels, to obtain a divergence loss function and a distillation loss function;

[0043] A second training module is used to perform model training on the initial behavior detection model based on the auxiliary branch loss function, the divergence loss function and the distillation loss function through preset weight information to obtain a target behavior detection model, wherein the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model;

[0044] The behavior recognition module is used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

[0045] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0046] Memory, used to store computer programs;

[0047] The processor is used to implement the steps of the behavior recognition method based on the behavior detection model applicable to fuzzy tags as described in any embodiment of the first aspect when executing the program stored in the memory.

[0048] In summary, in the model training stage of the behavior detection model, the embodiment of the present application obtains confused data samples and constructs an initial behavior detection model including a main path model and an auxiliary branch, and makes the auxiliary branch correspond to the behavior category in the confused sample data one by one, and then trains the auxiliary branch in the initial behavior detection model according to the confused data samples to obtain the auxiliary branch loss function, branch features, and potential distribution soft labels corresponding to the behavior category, so as to perform feature extraction and feature fusion training on the main path model based on the confused data samples to obtain the main path fusion features, and then use the branch features, the main path fusion features, and the potential distribution soft labels to perform multi-dimensional loss constraints on the main path model to obtain the divergence loss function and the distillation loss function, and finally, based on the auxiliary branch loss function, the divergence loss function, and the distillation loss function, the initial behavior detection model is model trained by preset weight information to obtain the target behavior detection model, thereby realizing the collaborative distillation training of the auxiliary branch and the main path in the initial behavior detection model by controlling the weights of multiple loss functions, so that the trained target behavior detection model can be applied to perform feature recognition and high-precision behavior detection on the experimental video stream containing fuzzy behavior to obtain the behavior recognition result. This application reduces the impact of original fuzzy labels on model performance through collaborative distillation learning, effectively improving the detection accuracy and practical use effect of the behavior detection model for confused label data sets, so that the behavior detection model can be used to perform feature recognition and behavior detection on the input video stream with high precision, thereby improving the accuracy and efficiency of behavior detection, and solving the problem of insufficient behavior recognition detection accuracy caused by the inability to train a high-precision recognition model in existing behavior recognition methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 A flow chart of a behavior recognition method based on a behavior detection model applicable to fuzzy labels provided in an embodiment of the present application;

[0052] Figure 2 It is a schematic diagram of the steps of a behavior recognition method based on a behavior detection model applicable to fuzzy labels provided by an optional embodiment of the present application;

[0053] Figure 3It is a model structure diagram of a behavior detection model provided by an optional example of the present application;

[0054] Figure 4 A structural block diagram of a behavior recognition device based on a behavior detection model applicable to fuzzy tags provided in an embodiment of the present application;

[0055] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0057] In related technologies, in addition to being used in scenarios such as examination room behavior recognition, behavioral target detection is also widely used in the field of security detection, which requires behavioral detection models to quickly respond to potential security threats, such as illegal intrusions or abnormal activities, so that timely action can be taken. These may involve lightweight network design, the use of more efficient convolution operations, the introduction of attention mechanisms to focus on key areas, and the use of multi-scale processing techniques. Through these methods, the model can achieve fast processing while maintaining high accuracy to meet the needs of real-time monitoring.

[0058] In order to achieve high detection accuracy, the existing behavior detection model mainly relies on the annotation quality of the data set during the training stage. However, the data set used for behavior detection model training has many defects, including but not limited to: ① For the atomic action set of human behavior, due to the lack of information about the previous and next frame sequences, different annotators often give different labels for a behavior whose features are not obvious in the image, and there are cases of fuzzy labels (uncertain labels); ② Each annotator has different standards for judging behavior, such as "left or right head tilt", and the standards (head tilt amplitude) for annotators to judge whether the action is "left or right head tilt" are also different; ③ When there are too many targets in the same image, missing labels and mislabeling will inevitably occur; ④ Long-tail data problem. The above factors are the main reasons for the poor quality of the annotated data set, which also leads to the inability to train a high-precision model no matter how strong the detection model is. The existing solution can only repeatedly check the data annotation to reduce fuzzy labels in order to improve the quality of the data set, which wastes human resources and fails to meet actual needs.

[0059] In order to solve the problem of insufficient behavior recognition accuracy of the behavior detection model, the present application proposes a behavior recognition method based on a behavior detection model suitable for fuzzy labels. In the model training stage, by innovatively improving the framework of the behavior detection model, based on the existing target detection and anomaly detection algorithms, N auxiliary branches (assuming the number of categories is N) are constructed to extract the potential feature distribution and attributes (labels) of the confused data samples, and the auxiliary branch training and the main path collaborative distillation training are completed by controlling the loss function weights, thereby effectively improving the detection accuracy and implementation effect of the basic algorithm for the confused label data set, and obtaining a target behavior detection model that can recognize fuzzy behaviors with high precision. The target behavior detection model is used to perform high-precision behavior detection and recognition on the input video stream, which solves the problem of insufficient behavior recognition detection accuracy caused by the inability to train a high-precision recognition model in the existing behavior recognition method.

[0060] To facilitate the understanding of the embodiments of the present application, further explanation will be given below in conjunction with the drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present application.

[0061] Figure 1 A flow chart of a method provided in an embodiment of the present application. Figure 1 As shown, the method provided in the embodiment of the present application may specifically include the following steps:

[0062] Step 110: construct an initial behavior detection model and obtain obfuscated data samples.

[0063] Wherein, the initial behavior detection model comprises a main path model and at least one auxiliary branch, and the auxiliary branch corresponds one-to-one to the behavior category in the obfuscated data sample;

[0064] Specifically, the obfuscated data samples may be composed of sample images / videos and sample annotations corresponding to multiple behavior categories. To improve the detection accuracy of the behavior detection model, the sample labels in the obfuscated data samples may include a portion of fuzzy labels.

[0065] When constructing an initial behavior detection model, the embodiment of the present application can simultaneously construct a main path model (or main branch) and multiple auxiliary branches in the initial behavior detection model. The auxiliary branches can correspond one-to-one to the behavior categories. For example, when the confused data sample has N behavior categories, N auxiliary branches can be constructed in the initial behavior model, and the N auxiliary branches correspond one-to-one to the N behavior categories.

[0066] Step 120, training the auxiliary branch according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and potential distribution soft labels corresponding to the behavior categories.

[0067] Specifically, the auxiliary branch loss function may be composed of a category loss and an anchor frame regression loss. Since the output of the auxiliary branch is the anchor frame coordinates and the predicted category, this embodiment defines the loss function of the auxiliary branch as a classification loss and an anchor frame coordinate regression loss. When the confused data sample is classified and predicted by the auxiliary branch, the output predicted category and anchor frame coordinates can be obtained. The classification loss may correspond to the predicted category, and the anchor frame coordinate regression loss may correspond to the anchor frame coordinates. The classification loss is determined by the predicted category, and the anchor frame coordinate regression loss is determined by the anchor frame coordinates, and then the auxiliary branch loss function is determined by combining the category loss and the anchor frame regression loss.

[0068] In a specific implementation, the auxiliary branch can be pre-trained for feature extraction using data samples so that the auxiliary branch can have feature extraction capabilities. Each auxiliary branch can then perform feature recognition and feature extraction based on the input data samples of the corresponding behavior category to obtain branch features.

[0069] In actual processing, the sample annotation corresponding to the sample image can be understood as the target box category label. In this embodiment, the auxiliary branch identifies the potential distribution soft label of the corresponding behavior category behavior based on the input data sample. For samples with the target box category label of category i, it is input into N auxiliary branches to obtain the potential distribution of the corresponding target box in each behavior category, that is, the potential distribution soft label. Therefore, the embodiment of the present application realizes the classification loss constraint from the sample label space, and uses N single-category detectors to predict the potential distribution of samples in each category, where the samples labeled with category i use N-1 single-category detectors other than detector i to predict the potential distribution.

[0070] Step 130 , performing feature extraction and feature fusion training on the main path model based on the obfuscated data sample to obtain the main path fusion feature.

[0071] In a specific implementation, in addition to pre-training the auxiliary branch for feature extraction, this embodiment can also pre-train the main path model for feature extraction and feature fusion using data samples, so that the main path model can have both feature extraction and feature fusion capabilities. Therefore, when performing collaborative distillation training of the main path model and the auxiliary branch, the confused data samples can be simultaneously input into the main path model and the auxiliary branch, and through feature recognition and extraction, the main path fusion features output by the main path model and the branch features output by the auxiliary branch are obtained, and the features output by the two different paths are used for subsequent collaborative distillation training.

[0072] Step 140, based on the branch features, the main path fusion features and the potential distribution soft labels, multi-dimensional loss constraints are performed on the main path model to obtain a divergence loss function and a distillation loss function.

[0073] In this embodiment, the divergence loss function is also called KL (Kullback-LeiblerDivergence, referred to as KL divergence) loss function. In this embodiment, the latent distribution soft label is used as the main path detection algorithm to coordinate distillation learning and training soft labels, and the KL loss function is added to minimize the deviation between the confidence of the main path prediction target box and the latent distribution soft label. The latent distribution soft label is used to constrain the classification loss from the sample label space to construct the KL loss function.

[0074] In actual implementation, for the branch features output by the auxiliary branch and the main path fusion features output by the main path model, this embodiment uses the distillation loss function to constrain these two features at the same time, and by using the distillation loss function, the ability of the target branch to extract features is brought closer to the N auxiliary branches.

[0075] Step 150, based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained through preset weight information to obtain a target behavior detection model.

[0076] The weight information is adjusted accordingly based on the training cycle of the initial behavior detection model.

[0077] Specifically, the weight information may include weights corresponding to the auxiliary branches and weights corresponding to the main path model, and this embodiment does not impose any restrictions on this. During the training phase of the model, this embodiment makes corresponding adjustments to the above two weights according to the training cycle of the initial behavior detection model. Preferably, the two weights may be inversely proportional, so that during model training, as the training cycle changes, the training focus may gradually shift from the auxiliary branches to the main path model, or from the main path model to the auxiliary branches, and this embodiment does not impose any restrictions on this.

[0078] In a specific implementation, after constructing the auxiliary branch loss function, the divergence loss function, and the distillation loss function, this embodiment can combine these loss functions together to construct a new constraint function for model training. As can be seen from the foregoing, different loss functions have different constraint meanings. This embodiment combines multiple loss functions together to perform collaborative distillation learning on the auxiliary branch and main path models, and uses weights to control the training center of gravity of the auxiliary branch and main path models, thereby reducing the impact of the original fuzzy labels on the model performance and effectively improving the accuracy of the behavior detection algorithm.

[0079] Step 160: Perform feature recognition and behavior detection processing on the experimental video stream containing the fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

[0080] In this embodiment, after completing the model training of the behavior detection model, the target behavior detection model with the highest recognition accuracy can be selected. The experimental video stream containing fuzzy behavior or not containing model behavior is input into the target behavior detection model, and the input experimental video stream is processed by the target behavior detection model for feature recognition extraction and behavior detection to obtain the behavior recognition result.

[0081] It can be seen that the embodiment of the present application aims to improve the behavior recognition accuracy of the behavior detection model for the input video stream, by obtaining obfuscated data samples and constructing an initial behavior detection model including a main path model and an auxiliary branch, and making the auxiliary branch correspond one-to-one to the behavior category in the obfuscated sample data, and then training the auxiliary branch in the initial behavior detection model according to the obfuscated data samples to obtain the auxiliary branch loss function, branch features and potential distribution soft labels corresponding to the behavior category, so as to perform feature extraction and feature fusion training on the main path model based on the obfuscated data samples to obtain the main path fusion features, and then use the branch features, the main path fusion features and the potential distribution soft labels to perform multi-dimensional loss constraints on the main path model to obtain the divergence loss function and the distillation loss function, and finally, based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained by preset weight information to obtain the target behavior detection model, thereby realizing the collaborative distillation training of the auxiliary branch and the main path in the initial behavior detection model by controlling the weights of multiple loss functions. After the present embodiment completes the model training to obtain the target behavior detection model, the target behavior detection model can be used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior to obtain the behavior recognition result. Therefore, the present embodiment reduces the influence of the original fuzzy label on the model performance through collaborative distillation learning, effectively improves the detection accuracy and practical use effect of the behavior detection model for the confused label data set, and can use the model to perform high-precision detection of fuzzy behavior in the input video stream in the application stage of the model, realize rapid processing, meet the needs of real-time monitoring, and solve the problem that the existing behavior detection model training method is limited by the quality of the labeled data set and cannot train a high-precision detection model.

[0082] Reference Figure 2 , shows a schematic flow chart of a behavior recognition method based on a behavior detection model applicable to fuzzy labels provided by an optional embodiment of the present application. The behavior recognition method based on a behavior detection model applicable to fuzzy labels may specifically include the following steps:

[0083] Step 210: construct an initial behavior detection model and obtain obfuscated data samples.

[0084] The initial behavior detection model includes a main path model and at least one auxiliary branch, and the auxiliary branch corresponds one-to-one to the behavior category in the obfuscated data sample.

[0085] In actual implementation, refer to Figure 3 As shown, the behavior detection model in this embodiment is mainly composed of the main path model and the auxiliary branch. The RetinaNet algorithm is used as the main path for target detection. The auxiliary branch does not participate in the calculation during actual deployment. The basic detection algorithms used in this embodiment include but are not limited to common models such as RetinaNet, YOLO series, SSD, and RCNN series.

[0086] In this embodiment, the main path model can be composed of Backbone (feature extraction network), FPN (feature pyramid network) and Head (regression, classification head). The function is to input the image to the main path model to obtain the behavior target prediction box and category prediction score. In addition, combined with Figure 3 ,The auxiliary branch can also include Backbone and FPN for feature extraction and feature fusion.

[0087] In the behavior detection task, various behaviors are not mutually exclusive, and the detection algorithm is improved to multi-category detection. Therefore, assuming that the behavior categories are divided into N categories, there is a label 1 containing: [behavior 1, behavior 2, behavior 3...behavior n], the corresponding position of the existing behavior is set to 1, and the position without behavior is set to 0. For example, the multi-hot encoding corresponding to label 1 can be [1,0,1,...,0].

[0088] For auxiliary branches. Assuming that the behaviors are divided into N categories, this embodiment can use N single-category detectors (i.e., N auxiliary branches) to predict the potential distribution of samples in each category. Preferably, samples labeled with category i use N-1 single-category detectors other than detector i to predict the potential distribution. Taking into account computational efficiency, sharing low-level features (such as backbone and FPN in this embodiment), all branches have the same structure, and the Head module adopts the SSH (Single Stage Headless) model structure.

[0089] In this embodiment, N behavior categories can be understood as N types of detection tasks. In this embodiment, when training the model of the auxiliary branch, each auxiliary branch can be trained using a data sample of a separate category, that is, for the obfuscated data samples of the same behavior category, the corresponding auxiliary branch is input for training. Exemplarily, for the i-th behavior category, the obfuscated data samples of the i-th behavior category are input into the corresponding i-th auxiliary branch. At this time, the i-th auxiliary branch only trains the obfuscated data samples of the i-th behavior category, thereby effectively reducing the bias effect of long-tail data and avoiding the problem of insufficient model generalization ability caused by category imbalance.

[0090] Step 220: extracting a sample image and a sample label corresponding to the sample image from the obfuscated data sample, and inputting the sample image and the sample label into a corresponding auxiliary branch.

[0091] Step 230, classify and predict the sample image and the sample label and extract features through the auxiliary branch to obtain a predicted anchor box, a predicted category corresponding to the predicted anchor box, a potential distribution soft label of the confused sample data on the behavior category, and branch features output by each auxiliary branch.

[0092] Step 240: For the predicted anchor box and the predicted category, the auxiliary branch is used to perform classification loss constraints on the sample image and the sample label to obtain an auxiliary branch category loss function and an auxiliary branch anchor box regression loss function.

[0093] A unified description of steps 220 to 240 is given as follows:

[0094] In actual implementation, refer to Figure 3 , sample images and sample labels can be input into the auxiliary branch and the main path model at the same time. The sample image can first pass through the auxiliary branch for feature extraction (exemplarily, ResNet50 can be used for feature extraction in the experiment), and then the extracted features are input into the FPN pyramid for feature fusion. The obtained feature featuremap_i is used as the branch feature and input into N auxiliary branches. The auxiliary branches can share data with the main path model to construct the loss function.

[0095] In a specific implementation, the auxiliary branch performs classification prediction on the input data, thereby obtaining multiple data, including but not limited to: the predicted anchor box, the predicted category corresponding to the predicted anchor box, the potential distribution soft label, the confidence corresponding to the predicted category, and the branch features used for the subsequent definition of the loss function. Specifically, in this embodiment, the data sample is input into the auxiliary branch, and each of the N auxiliary branches is trained to detect a category. For the jth auxiliary branch, only the jth image and the jth label are used for training during the training calculation. The loss of the auxiliary branch uses FocalLoss as the classification loss and CIoUloss as the anchor box loss. The auxiliary branch is used to classify and predict the input data to obtain the predicted anchor box, predicted category, confidence, etc., and the auxiliary branch category loss function and the auxiliary branch anchor box regression loss function are defined.

[0096] This embodiment inputs the confused data samples into the corresponding auxiliary branch according to the behavior category, and the auxiliary branch predicts and trains the sample image and sample label according to the input data sample. Since the output of the auxiliary branch is the anchor frame coordinates and the predicted category, in order to ensure that the auxiliary branch can accurately perform feature recognition and extraction on the input data, this embodiment defines a loss function of the auxiliary branch, and further defines it as classification loss and anchor frame coordinate regression loss. Among them, the classification loss can be determined using the auxiliary branch category loss function, corresponding to the output prediction category; the anchor frame coordinate regression loss can use the auxiliary branch anchor frame regression loss function, corresponding to the output anchor frame coordinates.

[0097] Based on the above analysis, at this time, the samples with target box category label i are input into N branches to obtain the potential distribution of the corresponding target box in each behavior category.

[0098] In an optional embodiment, the embodiment of the present application uses the auxiliary branch to perform classification loss constraints on the sample image and the sample label to obtain the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function, which may specifically include: according to the formula Determine the auxiliary branch category loss function; according to the formula Determine the auxiliary branch anchor frame regression loss function; where, is the auxiliary branch category loss function, p t is the confidence of the predicted box category, α is a parameter used to control the balance of positive and negative samples, and γ is a modulation factor parameter used to reduce the proportion of easy-to-separate samples in model training. is the auxiliary branch anchor box regression loss function, Pred bbox is the predicted anchor box, GT box To label the anchor box, CIoU loss is used to minimize the deviation between the predicted anchor box and the labeled anchor box. bbox It is obtained by the auxiliary branch through detection and classification based on the input data. The category loss function and anchor frame regression loss function of the auxiliary branch are used to improve the ability of the auxiliary branch to accurately classify the input data.

[0099] Preferably, α=0.25, γ=2.

[0100] Step 250: Determine an auxiliary branch loss function based on the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function.

[0101] In this embodiment, the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function are combined to obtain a loss function for training the auxiliary branch. When each auxiliary branch can stably detect the target frame coordinates and category confidence of the corresponding category, the auxiliary branch training is basically completed.

[0102] Step 260 , based on the potential distribution soft labels, a preset main path detection algorithm is used to perform spatial classification loss constraints on the main path model to obtain a divergence loss function.

[0103] The divergence loss function is used to minimize the deviation between the confidence of the predicted target box of the main path model and the potential distribution soft label.

[0104] Specifically, the divergence loss function can be understood as a classification loss constraint from the sample label space. This implementation uses the potential distribution as the main path detection algorithm to co-distill the learning and training soft labels, and minimizes the deviation between the confidence of the main path prediction target box and the soft label through the KL loss.

[0105] In actual implementation, the main path model can predict the input data and obtain the confidence distribution of the predicted target box category. The confidence distribution is used in combination with the latent distribution soft label to define the KL loss function, thereby minimizing the deviation between the confidence of the main path predicted target box and the soft label.

[0106] Optionally, the above-mentioned method of performing spatial classification loss constraints on the main path model through a preset main path detection algorithm based on the potential distribution soft label to obtain a divergence loss function may specifically include the following sub-steps:

[0107] Sub-step 2601, obtain the branch prediction target box predicted by the auxiliary branch for the confused sample data, and obtain the main path prediction target box predicted by the main path model for the potential distribution soft label and the category confidence corresponding to the main path prediction target box.

[0108] Sub-step 2602: performing intersection-over-union matching based on the first target box and the second target box, and calculating a divergence loss function according to the category confidence.

[0109] Sub-steps 2601 and 2602 are described uniformly:

[0110] In actual implementation, considering that the prediction box output by the target detection is disordered, this embodiment can match the target box predicted by the auxiliary branch with the target box predicted by the main path through IoU (Intersection over Union), and then calculate the KL loss for the category of the box. Through the defined KL loss, the feature extraction capabilities of the main branch and the auxiliary branch are improved respectively.

[0111] In an optional embodiment, the embodiment of the present application performs intersection-over-union matching based on the first target frame and the second target frame, and calculates the divergence loss function according to the category confidence, which may specifically include: according to the formula Calculate the divergence loss function, and according to Perform intersection-and-union matching; where D KL is the KL loss function, p(x) is the potential distribution soft label, and q(x) is the confidence distribution of the target box category predicted by the main path.

[0112] Step 270, based on the branch features and the main path fusion features, the main path model is subjected to loss constraints in the feature space dimension to obtain a distillation loss function.

[0113] In the specific implementation, the above KL loss function has already performed classification loss constraints on the sample label space. For the target detection algorithm, this embodiment also considers the feature space dimension and uses the similarity-preserving knowledge distillation loss to constrain the feature maps of the main path and the auxiliary branch, so that similar semantic features can be extracted from the image on different branches.

[0114] In an optional embodiment, the embodiment of the present application performs loss constraints on the feature space dimension of the main path model based on the branch features and the main path fusion features to obtain a distillation loss function, which may specifically include: according to the formula Perform loss constraints on the feature space dimension to obtain the distillation loss function; where L SP is the distillation loss, G main is the main path fusion feature, G bi For each branch feature, the distillation loss function is used to make the target branch's ability to extract features closer to multiple auxiliary branches.

[0115] Step 280, based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained through preset weight information to obtain a target behavior detection model.

[0116] The weight information is adjusted accordingly based on the training cycle of the initial behavior detection model.

[0117] In an optional embodiment, in the embodiment of the present application, when the weight information includes the auxiliary branch training weight and the main path training weight, the above-mentioned model training of the initial behavior detection model based on the auxiliary branch loss function, the divergence loss function and the distillation loss function through the preset weight information may specifically include: Determine the auxiliary branch loss function; according to Determine the main branch loss function; according to

[0118] Determine the calculation loss function and train the initial behavior detection model; where, is the main branch category classification loss, ε is the auxiliary branch training weight, (1-ε) is the main path training weight, the auxiliary branch training weight is inversely proportional to the main path training weight, and is adjusted based on the training cycle of the initial behavior detection model.

[0119] In a specific implementation, the auxiliary branch training weights and the main path training weights can be adjusted as the training cycle (epoch) increases. In the initial stage of model training, since the auxiliary branch is still exploring the potential distribution of the data, a higher weight can be given to promote its learning and improve the feature extraction capability of the auxiliary branch. At this time, the auxiliary branch training weight can be greater than the main path training weight. As the training progresses, once the performance of the auxiliary branch is significantly improved, the focus of training can gradually shift to the main path to further improve the overall performance of the model. Thus, the embodiment of the present application achieves the reduction of the impact of the original fuzzy labels on the model performance through collaborative distillation learning, effectively improving the accuracy of the behavior detection algorithm.

[0120] Specifically, in the initial stage of overall algorithm training, the value of ε can be raised to mainly train the loss function of the auxiliary branch. When each auxiliary branch can stably detect the target frame coordinates and category confidence of the corresponding category, the auxiliary branch training is basically completed; at this time, the algorithm begins to adjust the value of ε, so that the loss function gradually tends to train the main branch. Among them, the loss on the main branch includes the regression loss of the target frame and the annotation frame. Class loss Loss of intermediate layer feature map and auxiliary branch feature map And the category loss of the target box predicted by the main branch and the auxiliary branch

[0121] It should be noted that when calculating the KL loss, if on a picture, the auxiliary branch predicts two boxes, the categories are A1 and B1, and the main path also predicts two boxes, A2 and B2, respectively. When calculating the category loss, A1 and A2 can be calculated, and B1 and B2 can be calculated. Through the IoU matching method, the boxes of the corresponding targets predicted by the auxiliary and main paths are matched, and then the loss is calculated.

[0122] Step 290: Perform feature recognition and behavior detection processing on the experimental video stream containing the fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

[0123] In summary, in the training stage of the behavior detection model, the embodiment of the present application constructs an initial behavior detection model and obtains confused data samples, inputs the confused data samples into the corresponding auxiliary branches, and classifies and predicts and extracts features of the sample images and sample labels in the confused data samples through the auxiliary branches to obtain the predicted anchor boxes, the predicted categories corresponding to the predicted anchor boxes, the potential distribution soft labels of the confused sample data in the behavior categories, and the branch features output by each auxiliary branch. Then, for the predicted anchor boxes and the predicted categories, the sample images and sample labels are classified and constrained by the auxiliary branches to obtain the auxiliary branch category loss function and the auxiliary branch anchor box regression loss function. Subsequently, according to the potential distribution soft labels, the main path model is spatially classified and constrained by the preset main path detection algorithm to obtain the divergence loss function. The main path model is subjected to loss constraints in the feature space dimension based on the branch features and the main path fusion features to obtain the distillation loss function. Finally, based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained through the preset weight information to obtain a target behavior detection model that can perform feature recognition and behavior detection on a video stream containing fuzzy behaviors with high precision, thereby realizing the model training of the behavior detection model using fuzzy labels, reducing the influence of the original fuzzy labels on the model performance through collaborative distillation learning, effectively improving the accuracy of the behavior detection algorithm, reducing the influence of the original fuzzy labels on the model performance, and training a high-precision behavior detection model for poor quality labeled data sets to meet actual use needs, effectively improving the detection accuracy and landing effect of the behavior detection model for the confused label data set, thereby improving the accuracy of the behavior detection algorithm and the behavior detection model, and solving the problem of insufficient behavior recognition detection accuracy caused by the inability to train a high-precision recognition model in the existing behavior recognition method. In addition, the present application solves the following problems through innovative model training methods: ① The existing behavior detection model training method is limited by the quality of the labeled data set and cannot train a high-precision detection model; ② The existing technology can only repeatedly check the data annotation to reduce the fuzzy labels in order to improve the quality of the data set, which leads to waste of human resources and cannot meet the actual needs.

[0124] It should be noted that, for the purpose of simple description, the method embodiments are expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited to the described order of actions, because according to the embodiments of the present application, certain steps may be performed in other orders or simultaneously.

[0125] like Figure 4 As shown, the embodiment of the present application further provides a behavior recognition device 400 based on a behavior detection model applicable to fuzzy labels, comprising:

[0126] A construction module 410 is used to construct an initial behavior detection model and obtain an obfuscated data sample, wherein the initial behavior detection model includes a main path model and at least one auxiliary branch, and the auxiliary branch corresponds one-to-one to a behavior category in the obfuscated data sample;

[0127] A first training module 420 is used to train the auxiliary branch according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category;

[0128] An extraction module 430 is used to perform feature extraction and feature fusion training on the main path model based on the obfuscated data sample to obtain a main path fusion feature;

[0129] A loss constraint module 440 is used to perform multi-dimensional loss constraints on the main path model according to the branch features, the main path fusion features and the potential distribution soft labels to obtain a divergence loss function and a distillation loss function;

[0130] A second training module 450 is used to perform model training on the initial behavior detection model based on the auxiliary branch loss function, the divergence loss function and the distillation loss function through preset weight information to obtain a target behavior detection model, wherein the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model;

[0131] The behavior recognition module 460 is used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

[0132] Optionally, the first training module 420 includes:

[0133] An extraction submodule, used for extracting a sample image and a sample label corresponding to the sample image from the obfuscated data sample, and inputting the sample image and the sample label into a corresponding auxiliary branch;

[0134] A classification prediction and feature extraction submodule, used to perform classification prediction and feature extraction on the sample image and the sample label through the auxiliary branch, to obtain a predicted anchor frame, a predicted category corresponding to the predicted anchor frame, a potential distribution soft label of the obfuscated sample data on the behavior category, and branch features output by each auxiliary branch;

[0135] A classification loss constraint submodule, used for performing classification loss constraints on the sample image and the sample label through the auxiliary branch for the predicted anchor frame and the predicted category, to obtain an auxiliary branch category loss function and an auxiliary branch anchor frame regression loss function;

[0136] The auxiliary branch loss function determination submodule is used to determine the auxiliary branch loss function based on the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function.

[0137] Optionally, the classification loss constraint submodule is specifically used to: Determine the auxiliary branch category loss function; according to the formula Determine the auxiliary branch anchor frame regression loss function; where, is the auxiliary branch category loss function, p t is the confidence of the predicted box category, α is a parameter used to control the balance of positive and negative samples, and γ is a modulation factor parameter used to reduce the proportion of easy-to-separate samples in model training. is the auxiliary branch anchor box regression loss function, Pred bbox is the predicted anchor box, GT box To label the anchor box, CIoUloss is used to minimize the deviation between the predicted anchor box and the labeled anchor box.

[0138] Optionally, the loss constraint module 440 includes:

[0139] A first loss constraint submodule is used to perform spatial classification loss constraints on the main path model through a preset main path detection algorithm according to the potential distribution soft label to obtain a divergence loss function, wherein the divergence loss function is used to minimize the deviation between the confidence of the predicted target box of the main path model and the potential distribution soft label;

[0140] The second loss constraint submodule is used to perform loss constraints on the main path model in the feature space dimension based on the branch features and the main path fusion features to obtain a distillation loss function.

[0141] Optionally, the first loss constraint submodule includes:

[0142] A prediction unit, used to obtain a branch prediction target frame predicted by the auxiliary branch for the confused sample data, and to obtain a main path prediction target frame predicted by the main path model for the potential distribution soft label and a category confidence corresponding to the main path prediction target frame;

[0143] An intersection-over-union matching unit is used to perform an intersection-over-union matching based on the first target frame and the second target frame, and calculate a divergence loss function according to the category confidence.

[0144] Optionally, the intersection-over-combination matching unit is specifically used to: Calculate the divergence loss function, and according to Perform intersection-and-union matching; where D KL is the KL loss function, p(x) is the potential distribution soft label, and q(x) is the confidence distribution of the target box category predicted by the main path.

[0145] Optionally, the second loss constraint submodule is specifically used to: according to the formula Perform loss constraints on the feature space dimension to obtain the distillation loss function; where L SP is the distillation loss, G main is the main path fusion feature, G bi For each branch feature, the distillation loss function is used to make the target branch's ability to extract features closer to multiple auxiliary branches.

[0146] Optionally, the weight information includes auxiliary branch training weights and main path training weights, and the second training module 450 is specifically used to: Determine the auxiliary branch loss function; according to Determine the main branch loss function; according to

[0147] Determine the calculation loss function and train the initial behavior detection model; where, is the main branch category classification loss, ε is the auxiliary branch training weight, (1-ε) is the main path training weight, the auxiliary branch training weight is inversely proportional to the main path training weight, and is adjusted based on the training cycle of the initial behavior detection model.

[0148] It should be noted that the behavior recognition device based on the behavior detection model applicable to fuzzy labels provided in the embodiment of the present application can execute the behavior recognition method based on the behavior detection model applicable to fuzzy labels provided in any embodiment of the present application, and has the corresponding functions and beneficial effects of the execution method.

[0149] In a specific implementation, the above-mentioned behavior recognition device based on the behavior detection model suitable for fuzzy labels can be integrated into the device, so that the device can be based on the existing target detection and anomaly detection algorithms, combined with N auxiliary branches and main paths, and complete the auxiliary branch training and main path collaborative distillation training by controlling the loss function weights. As an electronic device, it is possible to improve the detection accuracy of the basic algorithm for the obfuscated label data set and the landing effect. The electronic device can be composed of two or more physical entities, or it can be composed of one physical entity. For example, the electronic device can be a personal computer (PC), a computer, a server, etc., and the embodiments of the present application do not impose specific restrictions on this.

[0150] like Figure 5 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114; the memory 113 is used to store computer programs; the processor 111 is used to implement the steps of the behavior recognition method based on the behavior detection model suitable for fuzzy labels provided in any of the aforementioned method embodiments when executing the program stored in the memory 113. Exemplarily, the steps of the behavior recognition method based on the behavior detection model applicable to fuzzy labels may include the following steps: constructing an initial behavior detection model, and obtaining a confused data sample, the initial behavior detection model includes a main path model and at least one auxiliary branch, the auxiliary branch corresponding to the behavior category in the confused data sample one by one; training the auxiliary branch based on the confused data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category; performing feature extraction and feature fusion training on the main path model based on the confused data sample to obtain a main path fusion feature; performing multi-dimensional loss constraints on the main path model based on the branch features, the main path fusion features, and the potential distribution soft label to obtain a divergence loss function and a distillation loss function; based on the auxiliary branch loss function, the divergence loss function, and the distillation loss function, model training is performed on the initial behavior detection model through preset weight information to obtain a target behavior detection model, and the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model; feature recognition and behavior detection processing are performed on the experimental video stream containing fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

[0151] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the behavior recognition method based on a behavior detection model applicable to fuzzy labels as provided in any of the aforementioned method embodiments are implemented.

[0152] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0153] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A behavior recognition method based on a behavior detection model suitable for fuzzy labels, characterized in that: include: Constructing an initial behavior detection model and obtaining an obfuscated data sample, wherein the initial behavior detection model comprises a main path model and at least one auxiliary branch, wherein the auxiliary branch corresponds one-to-one to a behavior category in the obfuscated data sample; The auxiliary branch is trained according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category; Based on the obfuscated data sample, feature extraction and feature fusion training are performed on the main path model to obtain main path fusion features; According to the branch features, the main path fusion features and the potential distribution soft labels, multi-dimensional loss constraints are performed on the main path model to obtain a divergence loss function and a distillation loss function; Based on the auxiliary branch loss function, the divergence loss function and the distillation loss function, the initial behavior detection model is trained by using preset weight information to obtain a target behavior detection model, and the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model; The target behavior detection model is used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior to obtain a behavior recognition result.

2. The method according to claim 1, characterized in that The auxiliary branch is trained according to the obfuscated data sample to obtain the auxiliary branch loss function, branch features and potential distribution soft labels corresponding to the behavior category, including: Extracting a sample image and a sample label corresponding to the sample image from the obfuscated data sample, and inputting the sample image and the sample label into a corresponding auxiliary branch; The auxiliary branch performs classification prediction and feature extraction on the sample image and the sample label to obtain a predicted anchor frame, a predicted category corresponding to the predicted anchor frame, a potential distribution soft label of the obfuscated sample data on the behavior category, and branch features output by each auxiliary branch; For the predicted anchor box and the predicted category, the sample image and the sample label are subjected to classification loss constraints by the auxiliary branch to obtain an auxiliary branch category loss function and an auxiliary branch anchor box regression loss function; An auxiliary branch loss function is determined based on the auxiliary branch category loss function and the auxiliary branch anchor frame regression loss function.

3. The method according to claim 2, characterized in that The step of performing classification loss constraints on the sample image and the sample label through the auxiliary branch to obtain an auxiliary branch category loss function and an auxiliary branch anchor frame regression loss function includes: According to the formula Determine the auxiliary branch category loss function; According to the formula Determine the auxiliary branch anchor frame regression loss function; in, is the auxiliary branch category loss function, p t is the confidence of the predicted box category, α is a parameter used to control the balance of positive and negative samples, and γ is a modulation factor parameter used to reduce the proportion of easy-to-separate samples in model training. is the auxiliary branch anchor box regression loss function, Pred bbox is the predicted anchor box, GT box To label the anchor box, CIoU loss is used to minimize the deviation between the predicted anchor box and the labeled anchor box.

4. The method according to claim 1, characterized in that: The method of performing multi-dimensional loss constraints on the main path model based on the branch features, the main path fusion features, and the potential distribution soft labels to obtain a divergence loss function and a distillation loss function includes: According to the potential distribution soft label, the main path model is subjected to spatial classification loss constraints by a preset main path detection algorithm to obtain a divergence loss function, where the divergence loss function is used to minimize the deviation between the confidence of the predicted target box of the main path model and the potential distribution soft label; Based on the branch features and the main path fusion features, the main path model is subjected to loss constraints in the feature space dimension to obtain a distillation loss function.

5. The method according to claim 4, characterized in that According to the potential distribution soft label, the main path model is subjected to spatial classification loss constraint by a preset main path detection algorithm to obtain a divergence loss function, including: Obtaining a branch prediction target frame predicted by the auxiliary branch for the confused sample data, and obtaining a main path prediction target frame predicted by the main path model for the potential distribution soft label and a category confidence corresponding to the main path prediction target frame; An intersection-over-union (IoU) matching is performed based on the first target box and the second target box, and a divergence loss function is calculated according to the category confidence.

6. The method according to claim 5, characterized in that The performing intersection-over-union matching based on the first target frame and the second target frame, and calculating a divergence loss function according to the category confidence, includes: According to the formula Calculate the divergence loss function, and according to Perform intersection and union matching; Among them, D KL is the KL loss function, p(x) is the potential distribution soft label, and q(x) is the confidence distribution of the target box category predicted by the main path.

7. The method according to claim 6, characterized in that The step of performing loss constraints on the main path model in the feature space dimension based on the branch features and the main path fusion features to obtain a distillation loss function includes: According to the formula Perform loss constraints on the feature space dimension to obtain the distillation loss function; Among them, L SP is the distillation loss, G main is the main path fusion feature, G bi For each branch feature, the distillation loss function is used to make the target branch's ability to extract features closer to multiple auxiliary branches.

8. The method according to claim 7, characterized in that The weight information includes auxiliary branch training weights and main path training weights, and the initial behavior detection model is trained based on the auxiliary branch loss function, the divergence loss function, and the distillation loss function through preset weight information, including: according to Determine the auxiliary branch loss function; according to Determine the main branch loss function; according to Determine the calculation loss function and perform model training on the initial behavior detection model; in, is the main branch category classification loss, ε is the auxiliary branch training weight, (1-ε is the main path training weight, the auxiliary branch training weight is inversely proportional to the main path training weight, and is adjusted based on the training cycle of the initial behavior detection model.

9. A behavior recognition device based on a behavior detection model suitable for fuzzy labels, characterized in that: include: A construction module, used to construct an initial behavior detection model and obtain an obfuscated data sample, wherein the initial behavior detection model includes a main path model and at least one auxiliary branch, and the auxiliary branch corresponds one-to-one to the behavior category in the obfuscated data sample; A first training module is used to train the auxiliary branch according to the obfuscated data sample to obtain an auxiliary branch loss function, branch features, and a potential distribution soft label corresponding to the behavior category; An extraction module, used for performing feature extraction and feature fusion training on the main path model based on the obfuscated data sample to obtain a main path fusion feature; A loss constraint module, used to perform multi-dimensional loss constraints on the main path model according to the branch features, the main path fusion features and the potential distribution soft labels, to obtain a divergence loss function and a distillation loss function; A second training module is used to perform model training on the initial behavior detection model based on the auxiliary branch loss function, the divergence loss function and the distillation loss function through preset weight information to obtain a target behavior detection model, wherein the weight information is adjusted accordingly based on the training cycle of the initial behavior detection model; The behavior recognition module is used to perform feature recognition and behavior detection processing on the experimental video stream containing fuzzy behavior through the target behavior detection model to obtain a behavior recognition result.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the behavior recognition method based on the behavior detection model applicable to fuzzy tags as described in any one of claims 1 to 8 when executing the program stored in the memory.

Images