Target object detection method and device, electronic equipment and storage medium

Abstract

The target object detection method, device, electronic equipment and storage medium provided in the embodiments of the present application comprise the following steps: first, a plurality of continuous detection images of a target object are obtained, and detection feature maps corresponding to the plurality of continuous detection images are obtained; then, the detection feature maps are subjected to attention analysis to obtain an analysis graph, and the analysis graph comprises at least one part feature of the target object; then, an object contour graph corresponding to each detection image is obtained, and the object contour graph and the analysis graph are fused to obtain a fusion data graph; finally, the fusion data graph is input into a gait recognition network for gait recognition, and a gait recognition result of the target object is obtained, so that the fusion data graph with more key information is obtained, and the accuracy of the gait recognition result is effectively improved when the gait recognition is performed based on the fusion data graph.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to target object detection methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Gait recognition is a long-range biometric identification technology that authenticates an individual by capturing and analyzing their unique walking patterns. Compared to other biometric technologies such as facial recognition and fingerprint recognition, gait recognition has several unique advantages. For example, gait data can be acquired from a distance without needing to be near the data collection device, and data can be collected discretely and unobtrusively. Due to these advantages, gait recognition has gradually found widespread application in fields such as security, identity verification, and social security.

[0003] In related technologies, gait recognition and detection for objects typically uses the object's contour or skeleton information as model input for gait recognition. However, since the contour or skeleton information contains limited key information, the detection results obtained through this gait recognition method have low accuracy. Summary of the Invention

[0004] This application provides a target object detection method, apparatus, electronic device, and storage medium, which can improve the detection accuracy of gait recognition of target objects.

[0005] To achieve the above objectives, a first aspect of this application proposes a target object detection method, the method comprising:

[0006] Obtain detection feature maps corresponding to multiple consecutive detection images of the target object, and perform attention parsing on the detection feature maps to obtain an analytical map, wherein the analytical map includes at least one part feature of the target object;

[0007] Obtain the object contour map corresponding to each of the detected images, and fuse the object contour map and the analytical map to obtain a fused data map;

[0008] The fused data map is input into the gait recognition network for gait recognition to obtain the gait recognition result of the target object.

[0009] In some embodiments, the step of performing attention parsing on the detected feature map to obtain an analytical map includes:

[0010] The detection feature map is flattened to obtain flattened detection features;

[0011] Add position encoding to the flattening detection features to obtain position detection features;

[0012] The analytical map is obtained by performing attention parsing on the location detection features.

[0013] In some embodiments, adding position encoding to the flattened detection features to obtain position detection features includes:

[0014] Obtain the location parameters, index parameters, and feature dimension parameters;

[0015] The feature parameters are obtained by performing exponential processing based on the ratio of the index parameter to the feature dimension parameter and based on preset parameters.

[0016] Based on the sine value corresponding to the ratio of the position parameter and the feature parameter, the position encoding feature is obtained;

[0017] The location detection feature is obtained by superimposing the location encoding feature and the flattening detection feature.

[0018] In some embodiments, the step of performing attention parsing on the location detection features to obtain the parsed map includes:

[0019] Obtain the query vector and the mask matrix, and perform probability distribution processing based on the query vector, the location detection features, and the mask matrix to obtain the mask attention parameters;

[0020] The mask attention parameters are parsed to obtain the location mask and category prediction corresponding to each location detection feature;

[0021] The location mask that matches the part of the target object predicted by the category is selected from the multiple location masks as the target location mask, and the analytical map corresponding to the target location mask is generated based on matrix multiplication.

[0022] In some embodiments, the location detection features include a first location detection feature and a second location detection feature. The step of performing probability distribution processing based on the query vector, the location detection features, and the mask matrix to obtain mask attention parameters includes:

[0023] Obtain the feature vector dimension of the location detection features;

[0024] The initial mask parameters are obtained by multiplying the first position detection feature and the query vector, dividing by the square root of the feature vector dimension, and adding the mask matrix.

[0025] The mask position parameters are obtained by multiplying the initial mask parameters and the second position detection features.

[0026] The mask position parameters are processed by probability distribution to obtain the mask attention parameters.

[0027] In some embodiments, fusing the object contour map and the analytical map to obtain a fused data map includes:

[0028] Obtain multiple contour pixel values ​​from the object contour map, and obtain multiple parsed pixel values ​​from the parsed map;

[0029] The pixels of the object contour map are updated one by one based on the contour pixel value and the parsed pixel value of each pixel.

[0030] The fused data map is obtained based on the updated object outline map.

[0031] In some embodiments, updating the pixels of the object contour map based on the contour pixel value and the parsed pixel value of each pixel includes:

[0032] When the outline pixel value of the pixel is one and the parsed pixel value of the pixel is not zero, the object part corresponding to the pixel in the parsed image is obtained;

[0033] Obtain the part color of the object, and color the pixels of the object outline map based on the part color.

[0034] In some embodiments, inputting the fused data map into a gait recognition network for gait recognition to obtain the gait recognition result of the target object includes:

[0035] The fused data graph is sequentially subjected to two-dimensional convolution, batch normalization, and nonlinearization to obtain fused data features.

[0036] The fused data features are then subjected to global pooling to obtain dimensionality-reduced fused features.

[0037] The dimensionality reduction and fusion features are used for gait recognition to obtain the gait recognition result.

[0038] To achieve the above objectives, a second aspect of this application provides a target object detection device, the module comprising:

[0039] The analytical image acquisition module is used to acquire detection feature maps corresponding to multiple consecutive detection images of the target object, and to perform attention parsing on the detection feature maps to obtain an analytical image, wherein the analytical image includes at least one part feature of the target object;

[0040] The data fusion module is used to acquire an object contour map corresponding to each of the detected images, and to fuse the object contour map and the analytical map to obtain a fused data map.

[0041] The gait recognition module is used to input the fused data map into the gait recognition network for gait recognition, and obtain the gait recognition result of the target object.

[0042] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the target object detection method as described in the first aspect.

[0043] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the target object detection method described in the first aspect.

[0044] The target object detection method, apparatus, electronic device, and storage medium proposed in this application include: First, acquiring detection feature maps corresponding to multiple consecutive detection images of the target object, and performing attention parsing on the detection feature maps to obtain an analytical map, the analytical map including at least one part feature of the target object; then, acquiring an object contour map corresponding to each detection image, and fusing the object contour map and the analytical map to obtain a fused data map; finally, inputting the fused data map into a gait recognition network for gait recognition to obtain the gait recognition result of the target object. This application utilizes the acquisition of analytical maps containing part features of the target object to improve the input key information for gait detection, and further fuses the analytical map and the object contour map to retain the part features of the target object while further recovering and enhancing these important shape information, thereby obtaining a fused data map with more key information. This effectively improves the accuracy of the gait recognition result when performing gait recognition based on the fused data map.

[0045] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0046] Figure 1 This is a flowchart of a target object detection method provided in an embodiment of this application.

[0047] Figure 2 yes Figure 1 The flowchart for step 101.

[0048] Figure 3 yes Figure 2The flowchart for step 202.

[0049] Figure 4 yes Figure 2 The flowchart for step 203.

[0050] Figure 5 yes Figure 4 The flowchart for step 401.

[0051] Figure 6 yes Figure 1 The flowchart for step 102.

[0052] Figure 7 yes Figure 6 The flowchart for step 602.

[0053] Figure 8 yes Figure 1 The flowchart for step 103.

[0054] Figure 9 This is a schematic diagram of the Resnet9 network architecture provided in another embodiment of this application.

[0055] Figure 10 This is a schematic diagram of a gait recognition process for a target object provided in another embodiment of this application.

[0056] Figure 11 This is a schematic diagram of the target object detection device provided in another embodiment of this application.

[0057] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0061] First, let's analyze some of the terms used in this application:

[0062] Transformer-based detectors are computer vision models that utilize the Transformer architecture to perform object detection tasks. This type of detector improves detection speed and efficiency without sacrificing accuracy by combining traditional convolutional neural networks (CNNs) with self-attention mechanisms. These detectors typically employ an encoder-decoder structure, where the encoder extracts features from the image, while the decoder predicts the location and class of the target. Due to their parallel processing capabilities and efficient capture of long-range dependencies, Transformer-based detectors demonstrate superior performance in multi-object detection in complex scenes.

[0063] RGB is an additive color model commonly used for displaying colors on screens. "RGB" represents three colors: red, green, and blue. Different combinations of these three colors can produce a wide range of colors. In the RGB color mode, each color (red, green, blue) has an intensity value, which is typically an integer from 0 to 255. When the intensity of red, green, and blue is all set to the maximum value (255), white is obtained; when all intensity values ​​are set to 0, black is obtained. Other colors are created by combining red, green, and blue with different intensities.

[0064] ResNet-50 (Residual Network 50 layers) is a deep learning convolutional neural network model proposed by Facebook AI Research Labs (FAIR) in their 2015 paper, "Deep Residual Learning for Image Recognition." ResNet-50 is renowned for its outstanding performance on image recognition tasks, particularly its excellent results in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). The core innovation of ResNet-50 lies in its residual block design, which addresses the vanishing and exploding gradient problems in deep network training by introducing skip connections (also called short-circuit connections). Traditional deep neural networks become increasingly difficult to train with increasing layer count and are prone to degradation (i.e., accuracy no longer improves or even decreases as the network deepens). ResNet, through its residual learning framework, makes information and gradients easier to propagate forward and backward, thus allowing for efficient training of deeper networks.

[0065] Gait recognition is a long-range biometric identification technology that authenticates an individual by capturing and analyzing their unique walking patterns. Compared to other biometric technologies such as facial recognition and fingerprint recognition, gait recognition has several unique advantages. For example, gait data can be acquired from a distance without needing to be near the data collection device, and data can be collected discretely and unobtrusively. Due to these advantages, gait recognition has gradually found widespread application in fields such as security, identity verification, and social security.

[0066] In related technologies, gait recognition and detection for objects typically uses the object's contour or skeleton information as model input for gait recognition. However, since the contour or skeleton information contains limited key information, the detection results obtained through this gait recognition method have low accuracy.

[0067] To improve the detection accuracy of gait recognition for target objects, embodiments of this application utilize analytical maps containing particulate features of the target object to enhance the key input information for gait detection. Furthermore, the analytical maps and object contour maps are fused to preserve the particulate features of the target object while further recovering and enhancing these important shape information, thereby obtaining a fused data map with more key information. This effectively improves the accuracy of gait recognition results when performing gait recognition based on the fused data map.

[0068] The target object detection method, apparatus, electronic device, and storage medium provided in the embodiments of this application will be further described below. The target object detection method provided in the embodiments of this application is applied to a server, processor, or smart terminal.

[0069] First, the target object detection method in the embodiments of this application is described in detail. (Refer to...) Figure 1 This is an optional flowchart of the target object detection method provided in the embodiments of this application. Figure 1 The method described may include, but is not limited to, steps 101 to 103. It is also understood that this embodiment... Figure 1 The order of steps 101 to 103 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0070] Step 101: Obtain the detection feature maps corresponding to multiple consecutive detection images of the target object, and perform attention parsing on the detection feature maps to obtain the analytical map.

[0071] Step 101 will be described in detail below.

[0072] In some embodiments, in response to a gait recognition request for a sequence of detection images consisting of multiple consecutive detection images of a target object, the detection image sequence is input into an encoder for feature extraction. The extracted features at different scales are then input into a backbone network (typically a ResNet-50 network) for feature fusion as shown in the following formula (1).

[0073] F fused =fuse({Backbone i (I)}) (1)

[0074] Where, I∈R B×H×W×3 Represents the input RGB detection image sequence, Backbone i This refers to the operation of the i-th layer backbone network, while fuse is a multi-layer fusion operation. Furthermore, B represents the number of data samples, H represents the image height (usually in pixels), W represents the image width (usually in pixels), and 3 refers to the number of image channels.

[0075] Next, the fused features are upsampled to recover a higher resolution detection feature map as shown in the following formula (2).

[0076] F decoded =BilinearUpsample(F fused (2)

[0077] Bilinear Upsampling is a commonly used image processing technique in deep learning, primarily used to enlarge image size while maintaining image quality. In Convolutional Neural Networks (CNNs), especially in tasks requiring image reconstruction (such as semantic segmentation and image inpainting), the upsampling step is crucial because it helps recover detailed information.

[0078] In this embodiment, a pre-trained model based on M2FP is used. This pre-trained model is based on the Mask2Former architecture and has been improved to adapt to human body analysis. It can segment various features from the input RGB image, including body parts such as the head and arms, as well as objects such as watches and backpacks. However, for gait recognition tasks, accessories such as watches and backpacks should be regarded as noise information.

[0079] Next, in order to obtain accurate gait recognition results for the target object, attention parsing is first required on the detection feature map to obtain an analytical map containing key information of the target object (i.e., at least one part feature of the target object, such as the head, body, arms, legs and feet of the human body), as described below.

[0080] Reference Figure 2 The detection feature map is subjected to attention parsing to obtain the parsed map, including the following steps 201 to 203.

[0081] Step 201: Flatten the detection feature map to obtain flattened detection features.

[0082] Step 202: Add position encoding to the flattened detection features to obtain position detection features.

[0083] Steps 201 to 202 are described in detail below.

[0084] In some embodiments, after obtaining the detection feature map F corresponding to the detection image sequence... decoded Next, the detection feature map F is first processed. decoded After flattening, the flattened detection features are obtained as shown in the following formula (3).

[0085] F flattened =Flatten(F decoded (3)

[0086] Flatten is a flattening operation, primarily used to transform multidimensional data into one-dimensional data. Furthermore,

[0087] F flattened ∈R B×(H'×W')×C' C is the feature dimension parameter.

[0088] Next, the flattening detection feature F flattened Adding positional encoding to obtain positional detection features improves the positional information of the features, which facilitates the subsequent fusion of the analytical image and the object contour image, as described below.

[0089] Reference Figure 3 The flattened detection features are added with position encoding to obtain position detection features, including the following steps 301 to 304.

[0090] Step 301: Obtain the location parameters, index parameters, and feature dimension parameters.

[0091] Step 302: Based on the ratio of the index parameter to the feature dimension parameter, and based on the preset parameters, perform exponential processing to obtain the feature parameters.

[0092] Step 303: Obtain the position coding feature based on the sine value corresponding to the ratio of the position parameter and the feature parameter.

[0093] Step 304: Overlay the location coding features and flattening detection features to obtain the location detection features.

[0094] Steps 301 to 304 are described in detail below.

[0095] In some embodiments, based on the set position parameters (including x and y), the channel index parameter i, and the feature dimension parameter C, the feature parameter 10000 is obtained by performing exponential processing based on the ratio C of the index parameter i to the feature dimension parameter and a preset parameter (i.e., 10000). 2i / C Furthermore, based on position parameters (including x and y) and feature parameters 10000 2i / C The sine value corresponding to the ratio is used to obtain the position coding feature as shown in the following formula (4).

[0096]

[0097] Next, based on the positional encoding feature P pos (x,y) and flattening detection features F flattened The location detection feature F is obtained by overlay processing. pos As shown in the following formula (5).

[0098] F pos =F flattened +P pos (5)

[0099] Through steps 301 to 304 above, the position coding features obtained by sinusoidal processing of index parameters, feature dimension parameters, and preset parameters are used to perform position coding superposition processing on the flattened detection features to obtain position detection features that effectively improve position information, thereby facilitating the subsequent fusion of the obtained analytical map and object contour map.

[0100] Step 203: Perform attention parsing on the location detection features to obtain the analytical map.

[0101] Step 203 will be described in detail below.

[0102] In some embodiments, when flattening detection feature F flattened Position encoding is performed to obtain the position detection feature F. pos Next, attention analysis will be performed on the unknown detection features to obtain an analytical map containing key information of the target object (i.e., at least one part feature of the target object, such as the head, body, arms, legs and feet of a human body), as described below.

[0103] Reference Figure 4 The location detection features are analyzed by attention to obtain an analytical map, including the following steps 401 to 403.

[0104] Step 401: Obtain the query vector and mask matrix, and perform probability distribution processing based on the query vector, position detection features and mask matrix to obtain the mask attention parameters.

[0105] Step 401 will be described in detail below.

[0106] In some embodiments, based on the defined query vector Q∈R M×C And the mask matrix M, further combined with the query vector Q∈R M×C Location detection feature F pos The mask matrix M is then processed using probability distribution to obtain the attention parameters needed for attention analysis in the attention processing. The query vector Q∈R M×C These represent different parsing objectives (including background query, component query, and human body query). In this embodiment, more emphasis will be placed on querying the feature characteristics of the target object (such as head, body, arms, legs, and feet) to improve the key information for gait recognition of the target object. The following will further describe how to obtain the mask attention parameters.

[0107] Reference Figure 5 Based on the query vector, location detection features, and mask matrix, probability distribution processing is performed to obtain mask attention parameters, including the following steps 501 to 504.

[0108] Step 501: Obtain the feature vector dimension of the location detection features.

[0109] Step 502: Based on the product of the first position detection feature and the query vector, divide by the square root of the feature vector dimension, and add the mask matrix to obtain the initial mask parameters.

[0110] Step 503: Obtain the mask position parameters based on the product of the initial mask parameters and the second position detection features.

[0111] Step 504: Perform probability distribution processing on the mask position parameters to obtain the mask attention parameters.

[0112] Steps 501 to 504 are described in detail below.

[0113] In some embodiments, based on the determined location detection features F pos Feature vector dimension d k and location detection feature F pos The first position detection feature K and the second position detection feature V are used; further, the product of the transpose of the first position detection feature K and the query vector Q is divided by the root of the feature vector dimension. Adding the mask matrix M, we obtain the initial mask parameters. Next, based on the product of the initial mask parameters and the second position detection feature V, the mask position parameters are obtained. Finally, the mask position parameters are processed by probability distribution to obtain the mask attention parameters as shown in the following formula (6).

[0114]

[0115] Step 402: Parse the mask attention parameters to obtain the location mask and category prediction corresponding to each location detection feature.

[0116] Step 403: Select the location mask from multiple location masks that matches the part of the target object in the category prediction as the target location mask, and generate the analytical map corresponding to the target location mask based on matrix multiplication.

[0117] Steps 402 to 403 are described in detail below.

[0118] In some embodiments, the Transformer Decoder is then used to fuse and interact with the input features and query vector through a series of attention mechanisms to parse the mask attention parameters, obtain the location mask and class prediction corresponding to each location detection feature, and select the location mask from multiple location masks whose class prediction matches the part of the target object (i.e., head, body, arms, legs and feet for human gait recognition) as the target location mask. Finally, the analytical map corresponding to the target location mask is generated based on matrix multiplication, which is the semantic segmentation result.

[0119] Through steps 401 to 403 and steps 501 to 504 above, the mask attention parameters obtained by the calculation of the position detection features, query vector and mask matrix as shown in formula (6) can be used to obtain the position mask and category prediction corresponding to different position detection features more accurately. The position mask that matches the part of the target object is selected from multiple position masks to generate an analytical map, so that only the analytical features corresponding to the parts of the target object related to gait recognition are retained, so as to exclude other interference information unrelated to gait recognition, and further improve the accuracy and reliability of subsequent gait recognition.

[0120] Step 102: Obtain the object contour map corresponding to each detection image, and fuse the object contour map and the analytical map to obtain the fused data map.

[0121] Step 102 is described in detail below.

[0122] In some embodiments, the analytical map obtained in step 101 focuses on segmenting the various parts of the target object. However, during the generation process, information such as the object's outline, edges, and shape may be lost. But the unique body shape and posture of the target object are key information in gait recognition. In this embodiment, by combining the analytical map with the object outline map, not only can the features of various parts of the human body be preserved, but also these important shape information can be restored and enhanced, so that the overall gait features of the target object can be captured more accurately in the subsequent gait recognition process.

[0123] Therefore, in this embodiment, the object contour map corresponding to the target object in the detection image is first obtained, and then the object contour map and the analytical map are fused to obtain a fused data map for subsequent improvement of gait recognition accuracy, as described in detail below.

[0124] Reference Figure 6 The object outline map and the analytical map are fused together to obtain a fused data map, including the following steps 601 to 603.

[0125] Step 601: Obtain multiple contour pixel values ​​from the object contour map and multiple parsed pixel values ​​from the parsed map.

[0126] Step 602: Update the pixels of the object outline map one by one according to the outline pixel value and the parsed pixel value of each pixel.

[0127] Steps 601 to 602 are described in detail below.

[0128] In some embodiments, based on the obtained object contour map and analytical map, the contour pixel value W(x,y) corresponding to each pixel point (i.e., coordinate (x,y)) in the object contour map is first obtained, and the analytical pixel value P(x,y) corresponding to each pixel point (i.e., coordinate (x,y)) in the analytical map is obtained.

[0129] Understandably, in the object outline drawing, except for the body parts of the target object which are white, the rest of the background is black.

[0130] Then, the pixels of the object contour map are updated one by one according to the contour pixel value W(x,y) and the parsed pixel value P(x,y) of each pixel point (i.e., coordinates (x,y)), as described in the following description.

[0131] Reference Figure 7 The object contour map is updated based on the contour pixel value and the parsed pixel value of each pixel, including the following steps 701 to 702.

[0132] Step 701: When the outline pixel value of a pixel is one and the parsed pixel value of the pixel is not zero, obtain the object part corresponding to the pixel in the parsed image.

[0133] Step 702: Obtain the part color of the object and color the pixels of the object outline map based on the part color.

[0134] Steps 701 to 702 are described in detail below.

[0135] In some embodiments, for each pixel (i.e., coordinates (x, y)), when the outline pixel value of the pixel is one (i.e., W(x, y) = 1) and the parsed pixel value of the pixel is not zero (i.e., P(x, y) ≠ 0), the object part k corresponding to the pixel in the parsed image is obtained, and the part color C of the object part is further determined. k Then, the pixels of the object outline are colored based on the part color, as shown in the following formula (7).

[0136]

[0137] Where F(x,y) represents the pixel value at (x,y) of the final fused image.

[0138] Step 603: Obtain the fused data map based on the updated object outline map.

[0139] Step 603 will be described in detail below.

[0140] In some embodiments, after traversing all pixels and coloring the object contour map according to the matching results, the updated object contour map is used as the fused data map.

[0141] Through steps 601 to 603 and steps 701 to 702 above, the pixel values ​​corresponding to each pixel in the object contour map and the analytical map are matched, and the object contour map is colored according to the matching results to complete the fusion of the object contour map and the analytical map. This can preserve the features of each part of the human body and restore and enhance these important shape information, so that the overall gait features of the target object can be captured more accurately in the subsequent gait recognition process, so as to obtain more accurate gait recognition results for the target object.

[0142] Step 103: Input the fused data map into the gait recognition network for gait recognition to obtain the gait recognition result of the target object.

[0143] Step 103 will be described in detail below.

[0144] In some embodiments, the gait recognition network uses the Gaitbase network to train and recognize the fused images. Gaitbase is a neural network structure designed specifically for gait recognition tasks. It has powerful spatiotemporal feature extraction capabilities by introducing multi-level convolutional neural networks and adaptive feature aggregation modules.

[0145] To further improve the accuracy of gait recognition for target objects in practical applications, it is necessary to train the gait recognition network.

[0146] In this embodiment, the generated fused data maps are used as input to the Gaitbase network for feature learning during training. Because these fused data maps combine the advantages of contour maps and analytical maps, the Gaitbase network can better capture the motion features of different parts of an object, avoiding interference from external objects. Simultaneously, the multi-layered feature extraction mechanism of the Gaitbase network demonstrates strong robustness when handling complex scenes and different perspectives.

[0147] Based on this gait recognition network, after obtaining the fused data map of the target object in practical applications, the fused data map is further input into the gait recognition network for gait recognition to obtain the gait recognition result of the target object, as described in detail below.

[0148] Reference Figure 8 The fused data map is input into the gait recognition network for gait recognition to obtain the gait recognition result of the target object, including the following steps 801 to 803.

[0149] Step 801: Perform two-dimensional convolution, batch normalization, and nonlinearization on the fused data graph in sequence to obtain the fused data features.

[0150] Step 802: Perform global pooling on the fused data features to obtain dimensionality-reduced fused features.

[0151] Step 803: Perform gait recognition on the dimensionality-reduced fusion features to obtain the gait recognition results.

[0152] Steps 801 to 803 are described in detail below.

[0153] In some embodiments, the gait recognition network uses the ResNet9 network architecture, referring to... Figure 9 This is a schematic diagram of a ResNet9 network architecture provided in an embodiment of this application. Figure 9 As shown, the ResNet9 network architecture used in this embodiment consists of multiple convolutional layers and residual blocks. Each residual block contains two convolutional layers and a skip connection to prevent the gradient vanishing problem.

[0154] After the fused data map is input into the gait recognition network, it is sequentially processed by two-dimensional convolution, batch normalization, and nonlinearization to obtain fused data features. Then, the fused data features are processed by global pooling (i.e., first, temporal pooling is used to integrate the time dimension into global features of the sequence, and then the feature map is divided into multiple horizontal regions for pooling to capture local features in the image) to obtain dimensionality-reduced fused features. Finally, gait recognition is performed on the dimensionality-reduced fused features through a fully connected layer to obtain gait recognition results (i.e., classification and feature mapping).

[0155] Through steps 801 to 803 above, the Gaitbase network, which has powerful spatiotemporal feature extraction capabilities, and the ResNet9 network architecture, which maintains model efficiency while possessing powerful feature extraction capabilities, are used to perform gait recognition of the target object on the fused data map obtained from the contour map and the analytical map. This results in more accurate gait recognition of the target object, thereby greatly improving the accuracy of target object detection.

[0156] Reference Figure 10 This is a schematic diagram illustrating the gait recognition process for a target object provided in an embodiment of this application. Figure 10 As shown, gait recognition of the target object is divided into three stages. In the first stage, the detection image is transformed into a human body parsing image through a pre-trained Mask2Former module to eliminate noise information. In the second stage, the human body parsing image generated in the first stage is fused with a binarized contour map. The two complement each other, which can simultaneously preserve the detailed features and global shape information of the human body, improving the accuracy and robustness of gait recognition. In the third stage, the spatiotemporal features extracted from the fused image are input into the gait recognition network to accurately identify the individual's gait pattern and perform identity verification.

[0157] The target object detection method, apparatus, electronic device, and storage medium proposed in this application include: First, acquiring detection feature maps corresponding to multiple consecutive detection images of the target object, and flattening the detection feature maps to obtain flattened detection features; acquiring position parameters, index parameters, and feature dimension parameters; obtaining feature parameters based on the ratio of the index parameters to the feature dimension parameters and performing exponential processing based on preset parameters; obtaining position encoding features based on the sine value corresponding to the ratio of the position parameters and the feature parameters; obtaining position detection features by superimposing the position encoding features and the flattened detection features; acquiring the feature vector dimension of the position detection features; obtaining initial mask parameters by multiplying the first position detection feature and the query vector, dividing by the square root of the feature vector dimension, and adding a mask matrix; obtaining mask position parameters by multiplying the initial mask parameters and the second position detection features; obtaining mask attention parameters by performing probability distribution processing on the mask position parameters; and parsing the mask attention parameters to obtain each The system generates a target location mask based on the location masks and category predictions corresponding to each location detection feature. It selects the location mask whose category prediction matches the part of the target object from multiple location masks and generates an analytical map corresponding to the target location mask based on matrix multiplication. The analytical map includes at least one part feature of the target object. Then, it acquires the object contour map corresponding to each detection image, and obtains multiple contour pixel values ​​from the object contour map and multiple analytical pixel values ​​from the analytical map. For each pixel whose contour pixel value is one and whose analytical pixel value is not zero, it acquires the object part corresponding to the pixel in the analytical map, obtains the part color of the object part, and colors the pixels in the object contour map based on the part color. A fused data map is obtained based on the updated object contour map. Finally, the fused data map is sequentially processed by two-dimensional convolution, batch normalization, and nonlinearization to obtain fused data features. These fused data features are then subjected to global pooling to obtain dimensionality-reduced fused features. Gait recognition is then performed on these dimensionality-reduced fused features to obtain the gait recognition result of the target object.

[0158] This application embodiment utilizes the mask attention parameter obtained by calculating the position detection features, query vector, and mask matrix as shown in formula (6), thereby more accurately obtaining the position mask and category prediction corresponding to different position detection features. It also selects the position mask matching the target object's part from multiple position masks to generate an analytical map, ensuring that only the analytical features corresponding to the target object's part related to gait recognition are retained, thus eliminating other interference information unrelated to gait recognition and further improving the accuracy and reliability of subsequent gait recognition. Furthermore, it uses the pixel values ​​corresponding to each pixel in the object contour map and the analytical map for matching, and adjusts the object contour map according to the matching results based on the corresponding part color. The process involves coloring the contour map and analytical map to fuse them, preserving the features of different parts of the human body and restoring and enhancing this important shape information. This allows for more accurate capture of the overall gait characteristics of the target object during subsequent gait recognition, resulting in more precise gait recognition results. Furthermore, the Gaitbase network, with its powerful spatiotemporal feature extraction capabilities, and the ResNet9 network architecture, which maintains model efficiency while possessing strong feature extraction capabilities, are used to perform gait recognition on the fused data map obtained from the contour map and analytical map. This yields even more accurate gait recognition results, significantly improving the accuracy of target object detection.

[0159] This application also provides a target object detection device that can implement the above-described target object detection method, see reference. Figure 11 The device 1100 includes:

[0160] The analytical image acquisition module 1110 is used to acquire the detection feature maps corresponding to multiple consecutive detection images of the target object, and to perform attention parsing on the detection feature maps to obtain an analytical image, which includes at least one part feature of the target object;

[0161] The data fusion module 1120 is used to acquire the object contour map corresponding to each detection image, and to fuse the object contour map and the analytical map to obtain the fused data map.

[0162] The gait recognition module 1130 is used to input the fused data map into the gait recognition network for gait recognition and obtain the gait recognition result of the target object.

[0163] In some embodiments, the parsing graph acquisition module 1110 is further configured to:

[0164] Flatten the detection feature map to obtain flattened detection features;

[0165] Add positional encoding to the flattened detection features to obtain the positional detection features;

[0166] An analytical graph is obtained by performing attention parsing on the location detection features.

[0167] In some embodiments, the parsing graph acquisition module 1110 is further configured to:

[0168] Obtain the location parameters, index parameters, and feature dimension parameters;

[0169] The feature parameters are obtained by performing exponential processing based on the ratio of the index parameter to the feature dimension parameter and based on preset parameters.

[0170] The location coding features are obtained based on the sine value corresponding to the ratio of the location parameters and the feature parameters;

[0171] The location detection features are obtained by overlaying the location encoding features and the flattening detection features.

[0172] In some embodiments, the parsing graph acquisition module 1110 is further configured to:

[0173] Obtain the query vector and mask matrix, and perform probability distribution processing based on the query vector, position detection features, and mask matrix to obtain the mask attention parameters;

[0174] The mask attention parameters are parsed to obtain the location mask and category prediction corresponding to each location detection feature;

[0175] Select the location mask that matches the part of the target object in the category prediction from multiple location masks as the target location mask, and generate the analytical map corresponding to the target location mask based on matrix multiplication.

[0176] In some embodiments, the parsing graph acquisition module 1110 is further configured to:

[0177] Obtain the feature vector dimension of the location detection features;

[0178] The initial mask parameters are obtained by multiplying the first position detection feature and the query vector, dividing by the square root of the feature vector dimension, and adding the mask matrix.

[0179] The mask position parameters are obtained by multiplying the initial mask parameters and the second position detection features.

[0180] The mask position parameters are processed by probability distribution to obtain the mask attention parameters.

[0181] In some embodiments, the data fusion module 1120 is further configured to:

[0182] Get multiple outline pixel values ​​from the object outline map, and get multiple parsed pixel values ​​from the parsed map;

[0183] Update the pixels of the object outline map one by one based on the outline pixel value and the parsed pixel value of each pixel;

[0184] The merged data map is obtained based on the updated object outline map.

[0185] In some embodiments, the data fusion module 1120 is further configured to:

[0186] When the outline pixel value of a pixel is one and the parsed pixel value of the pixel is not zero, obtain the object part corresponding to the pixel in the parsed image;

[0187] Get the part color of the object and color the pixels of the object outline map based on the part color.

[0188] In some embodiments, the gait recognition module 1130 is further configured to:

[0189] The fused data graph is sequentially subjected to two-dimensional convolution, batch normalization, and nonlinearization to obtain the fused data features.

[0190] The fused data features are subjected to global pooling to obtain dimensionality-reduced fused features;

[0191] The dimensionality-reduced and fused features are used for gait recognition to obtain the gait recognition results.

[0192] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, the specific implementation of the target object detection device is basically the same as the specific implementation of the target object detection method described above, and will not be repeated here.

[0193] In this embodiment, the target object detection device utilizes the mask attention parameter calculated as shown in formula (6) using position detection features, query vectors, and mask matrices. This allows for more accurate acquisition of position masks and category predictions corresponding to different position detection features. Furthermore, it selects a position mask matching the target object's part from multiple position masks to generate an analytical map. This ensures that only analytical features corresponding to the target object's part related to gait recognition are retained, eliminating other interference information unrelated to gait recognition and further improving the accuracy and reliability of subsequent gait recognition. Additionally, it matches the pixel values ​​corresponding to each pixel in the object contour map and the analytical map, and matches the corresponding part colors based on the matching results. Coloring the contour map completes the fusion of the object's contour map and analytical map, preserving the features of various parts of the human body and restoring and enhancing this important shape information. This allows for more accurate capture of the overall gait features of the target object during subsequent gait recognition, resulting in more precise gait recognition results. Furthermore, the Gaitbase network, with its powerful spatiotemporal feature extraction capabilities, and the ResNet9 network architecture, which maintains model efficiency while possessing strong feature extraction capabilities, are used to perform gait recognition on the fused data map obtained from the contour map and analytical map. This yields more accurate gait recognition results, significantly improving the accuracy of target object detection.

[0194] This application also provides an electronic device, including:

[0195] At least one memory;

[0196] At least one processor;

[0197] At least one program;

[0198] The program is stored in a memory, and the processor executes the at least one program to implement the target object detection method described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0199] Please see Figure 12 , Figure 12 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0200] The processor 1201 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0201] The memory 1202 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1202 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1202 and is called and executed by the processor 1201 to execute the target object detection method of the embodiments of this application.

[0202] The input / output interface 1203 is used to implement information input and output;

[0203] The communication interface 1204 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0204] Bus 1205 transmits information between various components of the device (e.g., processor 1201, memory 1202, input / output interface 1203, and communication interface 1204);

[0205] The processor 1201, memory 1202, input / output interface 1203 and communication interface 1204 are connected to each other within the device via bus 1205.

[0206] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described target object detection method.

[0207] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0208] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0209] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0210] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0212] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0213] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0215] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0217] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for detecting a target object, characterized in that, include: Obtain detection feature maps corresponding to multiple consecutive detection images of the target object, and perform attention parsing on the detection feature maps to obtain an analytical map, wherein the analytical map includes at least one part feature of the target object; Obtain the object contour map corresponding to each of the detected images, and fuse the object contour map and the analytical map to obtain a fused data map; The fused data map is input into the gait recognition network for gait recognition to obtain the gait recognition result of the target object; The step of performing attention parsing on the detected feature map to obtain the parsed map includes: The detection feature map is flattened to obtain flattened detection features; Add position encoding to the flattening detection features to obtain position detection features; The analytical map is obtained by performing attention parsing on the location detection features. The step of performing attention parsing on the location detection features to obtain the parsed map includes: Obtain the query vector and the mask matrix, and perform probability distribution processing based on the query vector, the location detection features, and the mask matrix to obtain the mask attention parameters; The mask attention parameters are parsed to obtain the location mask and category prediction corresponding to each location detection feature; The location mask that matches the part of the target object predicted by the category is selected from the multiple location masks as the target location mask, and the analytical map corresponding to the target location mask is generated based on matrix multiplication.

2. The target object detection method according to claim 1, characterized in that, The step of adding position encoding to the flattened detection features to obtain position detection features includes: Obtain the location parameters, index parameters, and feature dimension parameters; The feature parameters are obtained by performing exponential processing based on the ratio of the index parameter to the feature dimension parameter and based on preset parameters. Based on the sine value corresponding to the ratio of the position parameter and the feature parameter, the position encoding feature is obtained; The location detection feature is obtained by superimposing the location encoding feature and the flattening detection feature.

3. The target object detection method according to claim 1, characterized in that, The location detection features include a first location detection feature and a second location detection feature. The process of performing probability distribution processing based on the query vector, the location detection features, and the mask matrix to obtain mask attention parameters includes: Obtain the feature vector dimension of the location detection features; The initial mask parameters are obtained by multiplying the first position detection feature and the query vector, dividing by the square root of the feature vector dimension, and adding the mask matrix. The mask position parameters are obtained by multiplying the initial mask parameters and the second position detection features. The mask position parameters are processed by probability distribution to obtain the mask attention parameters.

4. The target object detection method according to claim 1, characterized in that, The process of fusing the object contour map and the analytical map to obtain a fused data map includes: Obtain multiple contour pixel values ​​from the object contour map, and obtain multiple parsed pixel values ​​from the parsed map; The pixels of the object contour map are updated one by one based on the contour pixel value and the parsed pixel value of each pixel. The fused data map is obtained based on the updated object outline map.

5. The target object detection method according to claim 4, characterized in that, The step of updating the pixels of the object contour map based on the contour pixel value and the parsed pixel value of each pixel includes: When the outline pixel value of the pixel is one and the parsed pixel value of the pixel is not zero, the object part corresponding to the pixel in the parsed image is obtained; Obtain the part color of the object, and color the pixels of the object outline map based on the part color.

6. The target object detection method according to claim 1, characterized in that, The step of inputting the fused data map into the gait recognition network for gait recognition to obtain the gait recognition result of the target object includes: The fused data graph is sequentially subjected to two-dimensional convolution, batch normalization, and nonlinearization to obtain fused data features. The fused data features are then subjected to global pooling to obtain dimensionality-reduced fused features. The dimensionality reduction and fusion features are used for gait recognition to obtain the gait recognition result.

7. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the target object detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target object detection method as described in any one of claims 1 to 6.

Images