Target behavior detection model construction method, system and device and storage medium

Abstract

The invention discloses a target behavior detection model construction method, system and device, and a storage medium, and the method comprises the steps: constructing a training data set based on a target behavior, and the training data set comprises pictures marking behavior types; inputting the training data set into a backbone network to obtain a preliminary feature map; obtaining a target feature map by using a multi-scale attention network and a feature enhancement network according to the preliminary feature map; according to the target behavior detection model construction method, a training data set is constructed and labeled, a primary feature map is extracted by using a backbone network, expression and integration of features are deepened in combination with a multi-scale attention network and a feature enhancement network, and a target behavior is detected based on the target feature map by using a detection network, and a target weight is determined. And the target behavior is accurately identified and the weight of the target behavior is determined by the detection network, so that the detection system which is efficient, flexible and high in robustness is realized, and the method is suitable for various application occasions needing to accurately identify specific behaviors in complex scenes.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method, system, device and storage medium for building a target behavior detection model. Background Art

[0002] Object detection is one of the classic problems in computer vision and an important research direction for vision-based pattern recognition applications. It requires searching any image, effectively labeling the objects in the image, and extracting the bounding rectangle. Since it is necessary to effectively detect objects with diverse postures in the image, the feature representation of the object needs to have strong expressiveness and discrimination. However, the traditional hand-designed features based on the expression of local statistical characteristics of the image are difficult to distinguish the diversity and complexity of the object, and it is difficult to achieve good results in general object detection problems.

[0003] At present, the YOLOv8 algorithm is widely used in deep learning-based target behavior detection. It constructs a data set by collecting and annotating target behavior images and identifies target behaviors in surveillance videos. The existing technology introduces the CBAM attention mechanism in the Neck part of the YOLOv8 model to increase the focus on target information, but this method has deficiencies in improving the model structure and detecting small targets. In addition, the existing process mainly focuses on the detection of a single target and lacks auxiliary discrimination combined with human posture information, resulting in low recognition accuracy of target behaviors. Summary of the invention

[0004] The main purpose of the present invention is to provide a target behavior detection model construction method, system, device and storage medium. The target behavior detection model construction method constructs and annotates a training data set, uses a backbone network to extract preliminary feature maps, combines a multi-scale attention network and a feature enhancement network to deepen the expression and integration of features, and accurately identifies and weights the target behavior through the detection network, thereby realizing an efficient, flexible and robust detection system suitable for a variety of application scenarios that require accurate identification of specific behaviors in complex scenes.

[0005] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, a method for constructing a target behavior detection model is provided, the method comprising:

[0007] Constructing a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories;

[0008] Input the training data set into the backbone network to obtain a preliminary feature map;

[0009] Obtaining a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map;

[0010] The detection network is used to detect the target behavior based on the target feature map and determine the target weight.

[0011] Optionally, obtaining a target feature map by using a multi-scale attention network and a feature enhancement network according to the preliminary feature map includes:

[0012] Inputting the preliminary feature map into the multi-scale attention network to obtain a global feature map and a local feature map;

[0013] Inputting the global feature map and the local feature map into the feature enhancement network to obtain an enhanced feature map;

[0014] The enhanced feature map is input into the multi-scale attention network to obtain the target feature map.

[0015] Optionally, inputting the preliminary feature map into the multi-scale attention network to obtain global feature information includes:

[0016] Performing serialization processing on the preliminary feature graph to obtain sequence data;

[0017] Processing the sequence data in four different directions based on four long short-term memory networks to obtain global context information in the four directions;

[0018] Integrate the global context information in the four directions to obtain a global feature vector for each pixel;

[0019] The convolution operator is used to calculate the attention weight vector of each pixel;

[0020] Constructing a two-dimensional matrix based on the attention weight vector of each pixel;

[0021] The preliminary feature map is weighted averaged based on a two-dimensional matrix of all pixels to obtain the global feature map.

[0022] Optionally, inputting the preliminary feature map into the multi-scale attention network to obtain local feature information includes:

[0023] Convert the number of channels of the preliminary feature map to the number of channels of the local dimension;

[0024] Calculate the local attention weight vector for each pixel based on the activation function;

[0025] The preliminary feature map is weighted based on the local attention weight vectors of all pixels to obtain a layout feature map.

[0026] Optionally, inputting the global feature map and the local feature map into the feature enhancement network to obtain an enhanced feature map includes:

[0027] Inputting the global feature map and the local feature map into the feature enhancement network for a first upsampling;

[0028] Splicing the global feature map and the local feature map after the first upsampling with the feature map of the same dimension output by the corresponding layer in the backbone network to obtain a spliced ​​feature map;

[0029] The spliced ​​feature map is upsampled for a second time, and the feature map obtained by the second upsampling is spliced ​​with the feature map output by another layer in the backbone network to obtain the enhanced feature map.

[0030] Optionally, inputting the enhanced feature map into the multi-scale attention network to obtain the target feature map includes:

[0031] Inputting the enhanced feature map into the multi-scale attention network, and using the local attention mechanism and the global attention mechanism in the multi-scale attention network to obtain a fused feature map;

[0032] Processing the fused feature map through a convolution layer, and splicing the features output by the convolution layer with the preliminary feature map to obtain spliced ​​features;

[0033] Perform feature conversion and feature enhancement on the spliced ​​features to obtain a key feature map;

[0034] The key feature map is input into the multi-scale attention network, and the above steps are repeated until the set conditions are met, and the target feature map is output.

[0035] Optionally, inputting the training data set into a backbone network to obtain a preliminary feature map includes:

[0036] Inputting the training data set into the backbone network, performing a depth-wise separable convolution operation through a depth-wise separable convolution module, and obtaining an initial feature map;

[0037] The initial feature map is subjected to feature conversion by a feature conversion module to obtain a converted feature map;

[0038] Processing the converted feature map again through the depthwise separable convolution module and the feature conversion module to obtain a pooled feature map;

[0039] The pooling feature map is subjected to a pooling operation to obtain the preliminary feature map.

[0040] According to a second aspect of an embodiment of the present application, a target behavior detection model construction system is provided, the system comprising:

[0041] A training set construction module, used to construct a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories;

[0042] A backbone network module, used to input the training data set into the backbone network to obtain a preliminary feature map;

[0043] A target feature module, used to obtain a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map;

[0044] The training module is used to detect the target behavior based on the target feature map using the detection network and determine the target weight.

[0045] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0046] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0047] In summary, the embodiments of the present application provide a method, system, device and storage medium for constructing a target behavior detection model, by constructing a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories; the training data set is input into a backbone network to obtain a preliminary feature map; a target feature map is obtained based on the preliminary feature map using a multi-scale attention network and a feature enhancement network; the target behavior is detected based on the target feature map using a detection network, and the target weight is determined. The target behavior detection model construction method realizes an efficient, flexible and robust detection system by constructing and annotating a training data set, extracting a preliminary feature map using a backbone network, combining a multi-scale attention network and a feature enhancement network to deepen the expression and integration of features, and accurately identifying and determining the weight of the target behavior by the detection network, which is suitable for a variety of application scenarios that require accurate identification of specific behaviors in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0049] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0050] Figure 1 A flow chart of a method for constructing a target behavior detection model provided in an embodiment of the present application;

[0051] Figure 2 The overall flow chart of target behavior detection provided by the embodiment of the present application;

[0052] Figure 3 An improved backbone network diagram provided in an embodiment of the present application;

[0053] Figure 4 A structural diagram of the attention mechanism module provided in the embodiment of the present application;

[0054] Figure 5 An improved Neck network structure diagram provided for an embodiment of the present application;

[0055] Figure 6 The overall structure of the network model provided in the embodiment of the present application;

[0056] Figure 7 A schematic diagram of the training process provided in an embodiment of the present application;

[0057] Figure 8 A system block diagram for constructing a target behavior detection model provided in an embodiment of the present application;

[0058] Fig. 9 A structural diagram of an electronic device provided in an embodiment of the present application is shown;

[0059] Fig.10 A diagram showing a computer-readable storage medium provided by an embodiment of the present application.

[0060] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0063] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0064] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0066] At present, deep learning-based methods are commonly used to detect target behaviors, mainly using the YOLOv8 algorithm for object recognition and positioning based on deep neural networks. The detection method mainly includes: first, collecting target behavior images to form a data set, using Labelmg to annotate images, dividing the data set, and monitoring the surveillance video in the monitoring environment to detect and annotate the target behavior images; secondly, introducing the attention mechanism CBAM into the Neck of the YOLOv8 model. The CBAM module sequentially calculates the attention map from two different dimensions, channel and space. After the attention mechanism CBAM is introduced into the Neck, it focuses on the target information and ignores other useless information; finally, the network model is trained on the annotated data set to obtain a trained model for target behavior detection.

[0067] In summary, the existing data set annotation only annotates the target. Compared with the picture of the entire monitoring scene, the target size is too small and the features are not obvious enough. The attention mechanism CBAM is only introduced in the Neck of the YOLOv8 model, which is not enough to improve the structure of the model. In addition, the existing traditional attention mechanism is used, which has poor detection effect for small targets. The designed target behavior recognition process is relatively simple, focusing only on a single target, without combining the human posture information of the target behavior for auxiliary judgment, and the recognition accuracy is low.

[0068] The embodiment of the present application improves the YOLOv8 network model based on the multi-scale attention mechanism module (MSAM), the depth-separable convolution module and the WiseIoU loss function to obtain an improved YOLOv8 detection network, which realizes comprehensive target behavior detection that combines global target behavior detection and local target object detection, greatly improving the detection accuracy.

[0069] Figure 1 A method for constructing a target behavior detection model provided in an embodiment of the present application is shown, and the method includes:

[0070] Step 101: constructing a training data set based on the target behavior, wherein the training data set includes pictures labeled with behavior categories;

[0071] Step 102: input the training data set into the backbone network to obtain a preliminary feature map;

[0072] Step 103: Obtain a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map;

[0073] Step 104: Detect the target behavior based on the target feature map using the detection network and determine the target weight.

[0074] In a possible implementation, in step 103, obtaining a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map includes:

[0075] The preliminary feature map is input into the multi-scale attention network to obtain a global feature map and a local feature map; the global feature map and the local feature map are input into the feature enhancement network to obtain an enhanced feature map; the enhanced feature map is input into the multi-scale attention network to obtain the target feature map.

[0076] In a possible implementation, inputting the preliminary feature map into the multi-scale attention network to obtain global feature information includes:

[0077] The preliminary feature map is serialized to obtain sequence data; the sequence data is processed in four different directions based on four long short-term memory networks to obtain global context information in the four directions; the global context information in the four directions is integrated to obtain a global feature vector for each pixel; a convolution operator is used to calculate an attention weight vector for each pixel; a two-dimensional matrix is ​​constructed based on the attention weight vector for each pixel; and the preliminary feature map is weighted averaged based on the two-dimensional matrix of all pixels to obtain the global feature map.

[0078] In a possible implementation, inputting the preliminary feature map into the multi-scale attention network to obtain local feature information includes:

[0079] The number of channels of the preliminary feature map is converted into the number of channels of the local dimension; the local attention weight vector of each pixel is calculated based on the activation function; the preliminary feature map is weighted based on the local attention weight vectors of all pixels to obtain a layout feature map.

[0080] In a possible implementation, the step of inputting the global feature map and the local feature map into the feature enhancement network to obtain an enhanced feature map includes:

[0081] The global feature map and the local feature map are input into the feature enhancement network for the first upsampling; the global feature map and the local feature map after the first upsampling are spliced ​​with the feature map of the same dimension output by the corresponding layer in the backbone network to obtain a spliced ​​feature map; the spliced ​​feature map is upsampled for a second time, and the feature map obtained by the second upsampling is spliced ​​with the feature map output by another layer in the backbone network to obtain the enhanced feature map.

[0082] In a possible implementation, inputting the enhanced feature map into the multi-scale attention network to obtain the target feature map includes:

[0083] The enhanced feature map is input into the multi-scale attention network, and the local attention mechanism and the global attention mechanism in the multi-scale attention network are used to obtain a fused feature map; the fused feature map is processed through a convolution layer, and the features output by the convolution layer are spliced ​​with the preliminary feature map to obtain spliced ​​features; the spliced ​​features are subjected to feature conversion and feature enhancement processing to obtain a key feature map; the key feature map is input into the multi-scale attention network, and the above steps are repeated until the set conditions are met and the target feature map is output.

[0084] In a possible implementation, inputting the training data set into a backbone network to obtain a preliminary feature map includes:

[0085] The training data set is input into the backbone network, and a depthwise separable convolution operation is performed through a depthwise separable convolution module to obtain an initial feature map; the initial feature map is subjected to feature conversion through a feature conversion module to obtain a conversion feature map; the conversion feature map is processed again through the depthwise separable convolution module and the feature conversion module to obtain a pooled feature map; the pooled feature map is subjected to a pooling operation to obtain the preliminary feature map.

[0086] The target behavior detection method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings. Figure 2 It is shown that if applied to the detection of a specific smoking behavior, for example, the target behavior detection method provided in the embodiment of the present application may specifically include the following stages:

[0087] Phase 1: Collect smoking images under indoor scene monitoring and annotate them using tools to create a smoking behavior dataset.

[0088] Step 1: Collect a set number of pictures in different indoor monitoring scenes, including smokers and non-smokers.

[0089] Step 2: Label the collected data set. You can label two categories, namely the smoker's action category and the cigarette category. The labeled categories provide accurate supervision signals for model training. The labeling tool can use the labelimg tool.

[0090] Through the first stage, a richer smoking behavior dataset was constructed, surveillance images in indoor scenes were collected, and two categories, cigarettes and smoking behavior, were annotated.

[0091] The second stage: building a network model that integrates the improved YOLO8 and multi-scale attention mechanism.

[0092] The following are the detailed steps to improve the YOLOv8 network model to integrate the multi-scale attention mechanism:

[0093] Step 1: Build an improved backbone network;

[0094] The traditional convolution (Conv) module in the backbone network of the YOLOv8 network model is replaced with a depthwise separable convolution (DSConv) module to reduce the amount of computation and the number of parameters. Figure 3 The improved backbone network is shown.

[0095] It should be noted that the Conv module in the backbone network of the YOLOv8 network model does not need to be replaced with the deep separable convolution module (DSConv). Still using the original Conv module can achieve higher detection accuracy.

[0096] Step 2: Feature extraction and enhancement based on the backbone network;

[0097] Step 2-1: Input the feature map of size H×W×3 into the improved backbone network.

[0098] Step 2-2: The feature map passes through two depth-wise separable convolution DSConv modules, performs two depth-wise separable convolution operations, and extracts preliminary features.

[0099] Step 2-3: The feature map undergoes three C2f operations through three consecutive C2f modules to further refine the features.

[0100] Step 2-4: Use the DSConv module again to perform a depth-wise separable convolution operation to enhance the expressiveness of the features.

[0101] Step 2-5: Perform the final C2f operation through a C2f module to complete the preliminary feature extraction.

[0102] Step 2-6: Finally, the feature map is pooled through an SPPF (Spatial Pyramid Pooling with Feature Fusion) module and output to the feature enhancement network.

[0103] Step 3: Construct a multi-scale attention mechanism (MSAM) that combines local attention mechanism and global attention mechanism. This attention mechanism can combine local feature information with global feature information.

[0104] The local attention mechanism enhances the feature recognition and detection of small targets (such as cigarettes) through continuous convolutional layers. The global attention mechanism uses the LSTM network to process the flattened sequence data, integrates the global context information, enhances the recognition of human motion features, and focuses on the motion features of the human body in the image, that is, the smoking action of the entire upper body. Figure 4The constructed attention mechanism module structure is shown, which specifically includes the following processes:

[0105] Step 3-1: Construct a global attention mechanism;

[0106] (1) Receive the output feature map from the backbone network, which has the number of channels C, width W and height H.

[0107] (2) Apply flattening operators in the vertical and horizontal directions of the image to convert the feature map into sequence data.

[0108] (3) Four long short-term memory (LSTM) networks are used to process the flattened sequence data respectively to obtain global context information in four directions. Each LSTM network is responsible for processing one direction of the sequence: one LSTM processes the sequence data from left to right to capture the dependencies in the horizontal direction. The second LSTM processes the sequence data from right to left to capture the horizontal information in the reverse direction. The third LSTM processes the sequence data from top to bottom to obtain the dependencies in the vertical direction. The fourth LSTM processes the sequence data from bottom to top to capture the vertical information in the reverse direction.

[0109] (4) The features obtained from the four directions are integrated so that each pixel can utilize the global context of the entire feature map. The feature map obtained at this time has a channel of C, a width of W, and a height of H. w,h Refers to the feature vector obtained at each pixel position (w,h).

[0110] (5) For each pixel position, use the convolution operator to convert the number of channels from C to a learnable dimension and convert the feature map channels to D = W × H; then use the softmax function to calculate the attention weight vector of each pixel

[0111]

[0112] where i∈{1,...,D}.

[0113] Calculate local and global attention weights, and get the attention weight vector for each pixel through the softmax function.

[0114] (6) Reshape the one-dimensional attention weight vector of each pixel into a two-dimensional matrix W×H.

[0115] (7) Use the reshaped attention matrix to perform weighted averaging on the original feature map to obtain the global attention map:

[0116]

[0117] Step 3-2: Construct a local attention mechanism. The local attention module consists of two consecutive convolutional layers to extract local features.

[0118] (1) Convert the number of channels of the feature map from C to a smaller dimension To focus on local features. and Represent the local width and height respectively.

[0119] (2) Calculate the local attention weight vector for each position through the softmax activation function.

[0120] (3) Use the calculated local attention weight to weight the feature map to obtain the local attention feature map:

[0121]

[0122] Step 3-3: Fusion of local and global attention mechanisms.

[0123] (1) The local attention feature map and the global attention feature map are fused to combine local details and global context information.

[0124] (2) The fused feature map can be further enhanced through convolutional layers and activation functions to improve the expressiveness of the features.

[0125] (3) Integrate multi-scale features at different stages of the network to capture the characteristics of objects of different sizes.

[0126] A new multi-scale attention mechanism module (MSAM) is constructed, which can combine local feature information with global feature information. Local attention focuses on the feature recognition and detection of small target objects such as cigarettes, while global attention focuses on the smoking action of the entire upper body.

[0127] Step 4: Incorporate the Multi-Scale Attention Mechanism Module (MSAM) into the Neck network of the YOLOv8 network model.

[0128] Step 4-1: Extract the feature map of the input image through the improved backbone network. Pass the feature map output by the backbone network to the Neck network as the basis for further feature enhancement and fusion. The improved Neck network structure is as follows Figure 5 shown.

[0129] Step 4-2: The backbone network outputs the feature map to the Neck network. The feature map is first upsampled twice, and the feature map obtained by each upsampling is concatenated with the feature map of the same dimension in the backbone network to combine global and local feature information and avoid the loss of feature information.

[0130] (1) The feature map output by the backbone network is upsampled for the first time to restore the spatial resolution.

[0131] (2) Concatenate the upsampled feature map with the feature map of the same dimension output by the corresponding layer in the backbone network.

[0132] (3) The concatenated feature map is upsampled for the second time to further restore the spatial size of the feature map.

[0133] (4) The feature map obtained by the second upsampling is concatenated with the feature map output by another layer in the backbone network, thereby combining global and local feature information and avoiding the loss of feature information.

[0134] The feature map output by the backbone network is upsampled and concatenated with the feature map of the same dimension to combine global and local feature information. The multi-scale attention mechanism module (MSAM) is integrated into the Neck network to extract key features and suppress noise.

[0135] Step 4-3: The upsampled restored feature map is integrated with a multi-scale attention mechanism module (MSAM) to extract the key features of the feature map and suppress noise.

[0136] (1) The local attention mechanism in MSAM is applied to the concatenated feature maps to enhance the feature recognition of small target objects.

[0137] (2) Following the local attention, the global attention mechanism in MSAM is applied to integrate the contextual information of the entire image.

[0138] (3) The local and global attention weights of each pixel position are calculated through the softmax function.

[0139] (4) The feature map is weighted using the calculated attention weights to highlight key features and suppress noise.

[0140] Step 4-4: Feature enhancement and refinement are performed through the convolution module, feature splicing module and C2f module.

[0141] (1) The weighted feature map is further refined through the convolutional layer.

[0142] (2) The features output by the convolutional layer are concatenated with the original feature map to integrate multi-scale information.

[0143] (3) The concatenated features are further transformed and enhanced through the C2f module.

[0144] After the convolution module, feature concatenation module and C2f module, the features are further fused and refined.

[0145] Step 4-5: Finally, the added attention mechanism module outputs the enhanced feature map.

[0146] (1) Apply MSAM again to the feature map after the C2f module to continuously extract and emphasize key features.

[0147] (2) Repeat convolution operations and feature concatenation to further enhance the feature map.

[0148] (3) Another feature transformation and enhancement is performed through the C2f module.

[0149] After being processed by multiple attention mechanism modules, the enhanced feature map is finally output.

[0150] Through these steps, the Neck network can effectively fuse feature information from different levels, while using the Multi-Scale Attention Mechanism Module (MSAM) to extract and emphasize key features, thereby improving the model's detection performance for objects in complex scenes.

[0151] Step 5: Optimize the detection head:

[0152] The DFL loss function and CIOU loss function in the Bbox Loss module in the YOLOv8 network model detection head are replaced with the Wise IoU loss function to obtain the improved detection head; the calculation formula of the Wise IoU loss function is:

[0153]

[0154] Among them, W i ,H i Respectively represent the width and height of the overlapping part, w gt ,h gt Respectively represent the width and height of the target box, w and h represent the width and height of the anchor box. The advantage of the Wise IoU loss function is that it not only considers the relative position and size differences between targets, but also introduces an intelligent weight adjustment mechanism that can adaptively adjust the weight coefficient to improve the flexibility and robustness of target detection.

[0155] It should be noted that the DFL loss function and CIOU loss function in the Bbox Loss module in the detection head of the YOLOv8 network model do not need to be replaced by the Wise IoU loss function, and a high detection accuracy can still be achieved.

[0156] Through these steps, we can build a Figure 6The improved YOLOv8 network model shown in the figure integrates a multi-scale attention mechanism. The improved backbone network is used to extract the features of the input image, the feature enhancement network is used to fuse the extracted features, and the improved detection head is used to identify the features after feature fusion, thereby improving the detection accuracy and robustness of smoking behavior.

[0157] The third stage: train the improved network model to obtain the optimal weight file best.pt.

[0158] like Figure 7 As shown in the figure, the model is trained on the labeled data set, and the number of training rounds is 100. The model is trained on the labeled data set, the hyperparameters are adjusted, and the model performance is verified. The optimal weight file best.pt obtained by the training is applied.

[0159] The fourth stage: Use the optimal weight file (best.pt) obtained through training to detect smoking behavior in actual scenarios.

[0160] The experimental results show that the constructed multi-scale fusion attention mechanism MSAM module and Wise IoU loss function improve the accuracy and robustness of the model. The deep separable convolution (DSConv) module replaces the original convolution layer, which greatly compresses the model's calculation amount and parameter amount, reduces the detection time, reduces the dependence on hardware conditions, and improves the recognition ability of smoking target detection. The method of the present invention has achieved good detection results in all scenarios, and has shown the generalization ability of enhanced targets under different lighting, monitoring angles, and oil smoke interference conditions.

[0161] The CIOU loss function in the Bbox Loss module in the detection head of the YOLOv8 network model is replaced with the WiseIoU loss function to obtain the improved detection head.

[0162] It should be noted that the method of the present invention is not only applicable to smoking behavior detection, but also applicable to other visual computer target detection tasks, and can also be extended to other visual computer target detection tasks. Through this improved YOLOv8 network model, the accuracy and robustness of behavior detection can be effectively improved, while reducing the demand for hardware resources, and has broad application prospects.

[0163] In summary, an embodiment of the present application provides a method for constructing a target behavior detection model, by constructing a training data set based on the target behavior, the training data set includes pictures with labeled behavior categories; the training data set is input into a backbone network to obtain a preliminary feature map; a target feature map is obtained based on the preliminary feature map using a multi-scale attention network and a feature enhancement network; the target behavior is detected based on the target feature map using a detection network, and the target weight is determined. The method for constructing a target behavior detection model realizes an efficient, flexible and robust detection system by constructing and annotating a training data set, extracting a preliminary feature map using a backbone network, combining a multi-scale attention network and a feature enhancement network to deepen the expression and integration of features, and accurately identifying and determining the weight of the target behavior by the detection network, which is suitable for a variety of applications that require accurate identification of specific behaviors in complex scenes.

[0164] Based on the same technical concept, the present application embodiment also provides a target behavior detection model construction system, such as Figure 8 As shown, the system comprises:

[0165] A training set construction module 801 is used to construct a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories;

[0166] A backbone network module 802 is used to input the training data set into the backbone network to obtain a preliminary feature map;

[0167] A target feature module 803 is used to obtain a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map;

[0168] The training module 804 is used to detect the target behavior based on the target feature map using the detection network and determine the target weight.

[0169] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Fig. 9 , which shows an electronic device diagram provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.

[0170] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0171] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0172] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0173] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0174] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Fig.10The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.

[0175] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0176] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0177] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for constructing a target behavior detection model, characterized in that: The method comprises: Constructing a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories; Input the training data set into the backbone network to obtain a preliminary feature map; Obtaining a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map; The detection network is used to detect the target behavior based on the target feature map and determine the target weight.

2. The method according to claim 1, characterized in that The method of obtaining a target feature map by using a multi-scale attention network and a feature enhancement network according to the preliminary feature map includes: Inputting the preliminary feature map into the multi-scale attention network to obtain a global feature map and a local feature map; Inputting the global feature map and the local feature map into the feature enhancement network to obtain an enhanced feature map; The enhanced feature map is input into the multi-scale attention network to obtain the target feature map.

3. The method according to claim 2, characterized in that The step of inputting the preliminary feature map into the multi-scale attention network to obtain global feature information includes: Performing serialization processing on the preliminary feature graph to obtain sequence data; Processing the sequence data in four different directions based on four long short-term memory networks to obtain global context information in the four directions; Integrate the global context information in the four directions to obtain a global feature vector for each pixel; The convolution operator is used to calculate the attention weight vector of each pixel; Constructing a two-dimensional matrix based on the attention weight vector of each pixel; The preliminary feature map is weighted averaged based on a two-dimensional matrix of all pixels to obtain the global feature map.

4. The method according to claim 2, characterized in that The step of inputting the preliminary feature map into the multi-scale attention network to obtain local feature information includes: Convert the number of channels of the preliminary feature map to the number of channels of the local dimension; Calculate the local attention weight vector for each pixel based on the activation function; The preliminary feature map is weighted based on the local attention weight vectors of all pixels to obtain a layout feature map.

5. The method according to claim 2, characterized in that The step of inputting the global feature map and the local feature map into the feature enhancement network to obtain an enhanced feature map comprises: Inputting the global feature map and the local feature map into the feature enhancement network for a first upsampling; Splicing the global feature map and the local feature map after the first upsampling with the feature map of the same dimension output by the corresponding layer in the backbone network to obtain a spliced ​​feature map; The spliced ​​feature map is upsampled for a second time, and the feature map obtained by the second upsampling is spliced ​​with the feature map output by another layer in the backbone network to obtain the enhanced feature map.

6. The method according to claim 2, characterized in that The step of inputting the enhanced feature map into the multi-scale attention network to obtain the target feature map comprises: Inputting the enhanced feature map into the multi-scale attention network, and using the local attention mechanism and the global attention mechanism in the multi-scale attention network to obtain a fused feature map; Processing the fused feature map through a convolution layer, and splicing the features output by the convolution layer with the preliminary feature map to obtain spliced ​​features; Perform feature conversion and feature enhancement on the spliced ​​features to obtain a key feature map; The key feature map is input into the multi-scale attention network, and the above steps are repeated until the set conditions are met, and the target feature map is output.

7. The method according to claim 1, characterized in that The step of inputting the training data set into the backbone network to obtain a preliminary feature map includes: Inputting the training data set into the backbone network, performing a depth-wise separable convolution operation through a depth-wise separable convolution module, and obtaining an initial feature map; Performing feature conversion on the initial feature map through a feature conversion module to obtain a converted feature map; Processing the converted feature map again through the depthwise separable convolution module and the feature conversion module to obtain a pooled feature map; The pooling feature map is subjected to a pooling operation to obtain the preliminary feature map.

8. A target behavior detection model construction system, characterized in that: The system comprises: A training set construction module, used to construct a training data set based on the target behavior, wherein the training data set includes pictures with labeled behavior categories; A backbone network module, used to input the training data set into the backbone network to obtain a preliminary feature map; A target feature module, used to obtain a target feature map using a multi-scale attention network and a feature enhancement network according to the preliminary feature map; The training module is used to detect the target behavior based on the target feature map using the detection network and determine the target weight.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-7.

Images