A target recognition method based on multi-stage deep feature extraction

Abstract

This application relates to the field of computer vision technology, and in particular to a target recognition method based on multi-stage deep feature extraction. The method includes: performing target detection on an input image to obtain bounding box information of the target object in the input image; processing the bounding box information of the target object using a pose estimation model to extract a set of key point information and their confidence scores; adaptively determining an optimized region of interest within the bounding box information of the input image based on the key point information; extracting deep features from the optimized region of interest using a deep feature extraction model to obtain a fixed-dimensional deep feature vector; and performing similarity matching between the feature vector and standard feature vectors in a feature library to determine the category of the target object. This method can solve the technical problems of low recognition accuracy and robustness caused by background interference, inconsistent feature representations, poor cross-category adaptation, and insufficient feature stability in existing technologies.

Description

Technical Field

[0001] The embodiments of this application relate to the technical fields of computer vision, deep learning, edge computing, etc., and in particular to a target recognition method based on multi-stage deep feature extraction. Background Technology

[0002] With the rapid development of computer vision technology, visual recognition has been widely applied in smart factories, industrial quality inspection, traffic management, and human-machine collaboration. Traditional recognition methods typically rely on manually designed color features, texture features, or local image descriptions. However, these features are prone to failure under conditions such as changes in lighting, angle transformation, occlusion interference, and complex backgrounds, making it difficult to meet the stability and generalization requirements of real-world scenarios. Object detection can locate specific objects in complex images, and then classification networks can be used to classify the target category, providing structured basic information for subsequent object representation. Meanwhile, with the widespread adoption of edge computing in front-end camera equipment, industrial terminals, and mobile smart devices, the recognition process not only needs high-precision performance in the cloud but also needs to run with low latency on edge devices with limited computing power. However, in multi-class, multi-pose, and multi-scene video images, relying solely on a single classification result is still insufficient to achieve fine-grained or individual-level recognition. Therefore, it is necessary to further combine deep feature representations, which are more discriminative and suitable for real-time processing at the edge.

[0003] Existing deep feature recognition methods typically extract feature vectors using convolutional neural networks and determine similarity based on vector distance. This is the mainstream technical approach for tasks such as person recognition, pedestrian re-identification, uniform recognition, and object appearance comparison. However, existing methods generally suffer from several shortcomings: First, the lack of target detection before feature extraction leads to significant background interference, affecting the quality of feature representation. Second, some methods directly use the output of classification networks as the basis for recognition, lacking a unified dimension and highly discriminative deep feature vectors. Third, existing solutions lack adaptability to different categories of objects and compatibility with multi-category scenes. Fourth, in complex video environments, feature consistency and cross-scene stability are still difficult to guarantee due to changes in target scale and appearance. Summary of the Invention

[0004] In view of this, embodiments of this application propose a target recognition method based on multi-stage deep feature extraction, aiming to solve the technical problems of low recognition accuracy and robustness caused by background interference, inconsistent feature representation, poor cross-class adaptation and insufficient feature stability in the prior art.

[0005] To achieve the above objectives, embodiments of this application propose a target recognition method based on multi-stage deep feature extraction, the method comprising the following steps: Using an object detection model, object detection is performed on the input image to obtain the bounding box information of the target objects in the input image; Using a pose estimation model, the bounding box information of the target object is processed to extract a set of key point information and their confidence scores. Based on key point information, the region of interest is adaptively determined within the bounding box information of the input image; Using a deep feature extraction model, deep features are extracted from the optimized region of interest to obtain a fixed-dimensional deep feature vector, and the deep feature vector is then normalized. A feature library is obtained, and the feature vectors are matched with the standard feature vectors in the feature library to determine the category of the target object; where each standard feature in the feature library corresponds to a category.

[0006] To achieve the above objectives, embodiments of this application also propose a target recognition system based on multi-stage deep feature extraction, the system comprising: The object detection module is used to perform object detection on the input image using an object detection model, so as to obtain the bounding box information of the target objects in the input image; The key point extraction module is used to process the bounding box information of the target object using the pose estimation model in order to extract a set of key point information and their confidence scores of the target object. The region optimization module is used to adaptively determine the optimized region of interest within the bounding box information of the input image based on key point information. The deep feature extraction module is used to extract deep features from the optimized region of interest using a deep feature extraction model, obtain a fixed-dimensional deep feature vector, and normalize the deep feature vector. The similarity matching module is used to obtain a feature library and perform similarity matching between the feature vectors and the standard feature vectors in the feature library to determine the category of the target object; where each standard feature in the feature library corresponds to a category.

[0007] To achieve the above objectives, embodiments of this application also propose an electronic device, including: a processor and a memory, wherein the memory stores instructions executable by the processor, and the processor is configured to execute the instructions such that the electronic device can implement a target recognition method based on multi-stage deep feature extraction as described above.

[0008] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables a target recognition method based on multi-stage deep feature extraction as described above.

[0009] This application proposes a target recognition method based on multi-stage deep feature extraction. It utilizes a target detection model to detect targets in an input image, obtaining bounding box information of the target object. A pose estimation model is then used to process the bounding box information to extract a set of keypoint information and their confidence scores. Based on the keypoint information, an optimized region of interest (ROI) is adaptively determined within the bounding box information of the input image. A deep feature extraction model is then used to extract deep features from the optimized ROI, obtaining a fixed-dimensional deep feature vector, which is then normalized. Finally, a feature library is acquired, and... The similarity matching between the feature vector and the standard feature vector in the feature library is performed to determine the category of the target object; each standard feature in the feature library corresponds to a category. This scheme adopts a multi-stage cascaded architecture of target detection, pose estimation, adaptive region localization, deep feature extraction and similarity matching. It can combine human key point-based region of interest optimization technology and multi-dimensional feature fusion mechanism to solve the technical problems of low recognition accuracy and robustness caused by background interference, inconsistent feature representation, poor cross-category adaptation and insufficient feature stability in existing technologies. Thus, it can achieve high-precision and high-robust recognition of multiple types of objects in complex scenes.

[0010] Optionally, the object detection model is an improved YOLO series single-stage deep convolutional neural network; the object detection model includes the content-aware feature reorganization upsampling operator CARAFE, the CARAFE upsampling process including: The low-resolution input feature map is convolved to compress its number of channels to the target value, resulting in a compressed feature map; the calculation formula is as follows: ; in, Indicates the input feature map, This represents the compressed feature map. It is a 1×1 convolution matrix. The index of the input feature map; Obtain the upsampling ratio and upsampling size, and predict the upsampling kernel of the compressed feature map through a convolutional layer; The upsampling kernel is normalized to obtain the normalized upsampling kernel; Perform channel and spatial integration operations, and use the normalized upsampling kernel to upsample the compressed feature map to obtain the upsampled feature map; the calculation formula is as follows: ; in, ( ) indicates channel and space integration operations.

[0011] Optionally, a pose estimation model is used to process the bounding box information of the target object to extract a set of key point information and their confidence scores, including: The target region corresponding to the bounding box information is cropped from the input image, and its size is normalized to be adjusted to the input size required by the pose estimation model. Color space conversion and pixel value normalization are then performed to obtain the preprocessed image. The formula for pixel value normalization includes: ; in, This represents the color channels, including the R, G, and B channels. It is the mean vector. The standard deviation vector, mean vector, and standard deviation vector are normalized using the parameters from the ImageNet dataset; The preprocessed image is input into the pose estimation model to obtain a dimension of The heatmap tensor; where... and These represent the height and width in the dimensions of the heatmap, respectively. Indicates the number of key points; The heatmaps corresponding to each key point in the heatmap tensor are processed to determine the location of the maximum response value, which is then used as the coordinates of the key point in the heatmap, and the maximum response value is used as the confidence level of the key point. The coordinates in the heatmap are converted into cropping region coordinates, and the cropping region coordinates are converted into the original coordinates in the input image, thus obtaining a set of key points; the parameters of each key point in the set of key points include coordinate position and confidence level. The conversion formula for converting coordinates in a heatmap to coordinates of the clipping region is as follows: ; ; in, Indicates the coordinates of the clipping region; , Indicates the original width and original height of the cropped area; Indicates the location of the maximum response value; The conversion formula for converting the cropped region coordinates to the original coordinates in the input image is as follows: ; ; in, Represents the x and y coordinates of the original coordinates in the input image; This indicates the coordinates of the top-left corner of the bounding box in the input image.

[0012] Optionally, based on keypoint information, an optimized region of interest is adaptively determined within the bounding box information of the input image, including: performing pose quality assessment on the keypoint set of the target object to detect whether the target object is a valid target object; if the target object is a valid target object, then based on the keypoint set, calculating the bounding rectangle coordinates of its spatial distribution, and adaptively expanding it in the horizontal and vertical directions to obtain the optimized region of interest; wherein the range of the optimized region of interest does not exceed the range corresponding to the bounding box information.

[0013] Optionally, the posture quality assessment includes a clear posture filtering mode; the posture quality assessment of the key point set of the target object, and the detection of whether the target object is a valid target object, includes: determining the number of valid key points in the key point set whose confidence exceeds a key point confidence threshold based on the clear posture filtering mode and the confidence of each key point; when the number of valid key points is lower than a first preset threshold, determining the posture of the target object as an invalid posture and filtering it; when the number of valid key points is greater than or equal to the first preset threshold, determining the target object as a valid target object.

[0014] Optionally, the posture quality assessment includes a standing posture filtering mode; the posture quality assessment of the key point set of the target object to detect whether the target object is a valid target object includes: performing a body uprightness check and a leg posture check on the target object to determine whether the target object's posture is a standing posture; wherein, the body uprightness check requires that the vertical coordinates of the shoulder key points in the key point set are less than the vertical coordinates of the hip key points; the leg posture check requires that the vertical coordinates of the hip key points, knee key points, and ankle key points in the key point set satisfy an increasing relationship from top to bottom; when both the body uprightness check and the leg posture check pass, the target object is determined to be in a standing posture, and thus the target object is determined to be a valid target object.

[0015] Optionally, a feature library is obtained, and the deep feature vector is matched with the standard feature vector in the feature library to determine the category of the target object. This includes: using a cosine similarity-based matching method to calculate the similarity between the deep feature vector and each standard feature vector in the feature library; traversing all similarity calculation results and taking the category corresponding to the highest similarity calculation result as the candidate matching result; if the highest similarity calculation result is greater than or equal to a preset judgment threshold, the target object is determined to belong to the candidate matching result.

[0016] Optionally, the training process of the deep feature extraction model includes: collecting a training dataset containing images of multiple categories, and performing preprocessing on each image in the training dataset to obtain standardized regions of interest as training samples; constructing a ResNet network architecture containing multiple residual blocks, and adding a fully connected layer of a specific dimension at the end of the ResNet network as a feature extraction and classification layer; wherein, the residual block contains a multi-scale feature fusion structure; using the cross-entropy loss function and gradient descent optimization algorithm, using the standardized region of interest image as input and the image's category label as a supervision signal, performing end-to-end training on the ResNet network; after the ResNet network training is completed, removing the classification layer of the ResNet network and retaining the feature extraction layer for outputting deep feature vectors. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. Obviously, the following drawings are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings described herein are only used to explain this application and are not intended to limit this application.

[0018] Figure 1 This is a flowchart of a target recognition method based on multi-stage deep feature extraction provided in one embodiment of this application; Figure 2 This is a detailed flowchart of another target recognition method based on multi-stage deep feature extraction provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a target recognition system based on multi-stage deep feature extraction provided in another embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. Those skilled in the art will understand that many technical details have been presented in the embodiments of this application to facilitate better understanding. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of this application. The following embodiments can be combined with and referenced by each other without contradiction.

[0020] With the rapid development of computer vision technology, visual recognition has been widely applied in smart factories, industrial quality inspection, traffic management, and human-machine collaboration. Traditional recognition methods typically rely on manually designed color features, texture features, or local image descriptions. However, these features are prone to failure under conditions such as changes in lighting, angle transformation, occlusion interference, and complex backgrounds, making it difficult to meet the stability and generalization requirements of real-world scenarios. Object detection can locate specific objects in complex images, and then classification networks can be used to classify the target category, providing structured basic information for subsequent object representation. Meanwhile, with the widespread adoption of edge computing in front-end camera equipment, industrial terminals, and mobile smart devices, the recognition process not only needs high-precision performance in the cloud but also needs to run with low latency on edge devices with limited computing power. However, in multi-class, multi-pose, and multi-scene video images, relying solely on a single classification result is still insufficient to achieve fine-grained or individual-level recognition. Therefore, it is necessary to further combine deep feature representations, which are more discriminative and suitable for real-time processing at the edge.

[0021] Existing deep feature recognition methods typically extract feature vectors using convolutional neural networks and determine similarity based on vector distance. This is the mainstream technical approach for tasks such as person recognition, pedestrian re-identification, uniform recognition, and object appearance comparison. However, existing methods generally suffer from several shortcomings: First, the lack of target detection before feature extraction leads to significant background interference, affecting the quality of feature representation. Second, some methods directly use the output of classification networks as the basis for recognition, lacking a unified dimension and highly discriminative deep feature vectors. Third, existing solutions lack adaptability to different categories of objects and compatibility with multi-category scenes. Fourth, in complex video environments, feature consistency and cross-scene stability are still difficult to guarantee due to changes in target scale and appearance.

[0022] In view of this, embodiments of this application propose a target recognition method based on multi-stage deep feature extraction, aiming to solve the technical problems of low recognition accuracy and robustness caused by background interference, inconsistent feature representation, poor cross-class adaptation and insufficient feature stability in the prior art.

[0023] One embodiment of this application proposes a target recognition method based on multi-stage deep feature extraction, applied to an electronic device, wherein the electronic device can be a terminal or a server. This embodiment and the following embodiments will use a server as an example for description. The implementation details of the target recognition method based on multi-stage deep feature extraction proposed in this embodiment will be described in detail below. The following implementation details are provided for ease of understanding and are not necessary for implementing this solution.

[0024] The specific process of the target recognition method based on multi-stage deep feature extraction proposed in this embodiment can be described as follows: Figure 1 As shown, it includes: Step 101: Use the object detection model to perform object detection on the input image to obtain the bounding box information of the target object in the input image.

[0025] Understandably, this involves locating the bounding box information of all target objects from the input image. In practice, an improved YOLOv11 single-stage deep convolutional neural network is used as the object detection model.

[0026] For example, the input image can be a single frame image decoded from a video stream. The target object can be a human body. The number of target objects in the input image can be one or more, and no specific limitation is made in the embodiments of this application.

[0027] For example, the object detection model could be an improved YOLO family of single-stage deep convolutional neural networks.

[0028] For example, the input image can be preprocessed by using a hardware decoder to decode the input video frames into an image format suitable for direct processing by the AI ​​processor. For instance, the hardware decoder could be the Decoder provided by the SOPHON SDK; the AI ​​processor could be a TPU device; and the image format could be BMImage, which can be processed directly in the TPU device's memory without CPU intervention. Subsequently, a visual computing library is used to resize the image to the input size required by the network (e.g., 640x640 pixels), and color space conversion and pixel value normalization are performed.

[0029] For example, a single-stage deep convolutional neural network can be used to perform object detection on the input image, obtaining the bounding box coordinates, detection confidence, and category information of the target object. The deep convolutional neural network uses an improved YOLOv11 architecture, filtering out non-target categories through a category discriminant function, and outputting a set of detection results. Each detection result includes bounding box coordinates, detection confidence, and category label.

[0030] For example, the preprocessed image data is converted into a Tensor format and used as input to the YOLOv11 model. The SOPHON inference engine is then called to load the quantized YOLOv11 model file, and forward inference is performed to obtain the detection results. The YOLOv11 network introduces a lightweight CARAFE upsampling operator, which improves the efficiency and performance of the upsampling operation through content-aware reconstructed features.

[0031] In one possible embodiment, the object detection model includes a content-aware feature reassembly upsampling operator CARAFE to improve the efficiency and performance of the upsampling operation; the CARAFE upsampling process includes: The low-resolution input feature map is convolved to compress its number of channels to the target value, resulting in a compressed feature map. For example, the target value is The input low-resolution feature map Perform a 1×1 convolution operation to compress its channel number to .

[0032] The calculation formula is as follows: ; in, Indicates the input feature map, This represents the compressed feature map. It is a 1×1 convolution matrix. The index of the input feature map; Obtain the upsampling ratio and upsampling size, and predict the upsampling kernel of the compressed feature map through a convolutional layer; The upsampling kernel is normalized to obtain the normalized upsampling kernel; For example, setting the upsampling factor to be... Upsampling kernel size For the compressed input feature map pass The convolutional layer prediction upsampling kernel has the following input channels: The number of output channels is Expanding the channel dimension in spatial dimensions yields a shape of Predictive upsampling kernel Then, SoftMax is used for normalization so that the sum of the convolution kernel weights is 1, resulting in the calculated upsampling kernel. .

[0033] Perform channel and spatial integration operations, and use the normalized upsampling kernel to upsample the compressed feature map to obtain the upsampled feature map; the calculation formula is as follows: ; in, ( ) indicates channel and space integration operations.

[0034] Understandably, the CARAFE operator dynamically generates upsampling kernels by sensing the content of the feature map, thereby improving the quality and efficiency of feature map reconstruction.

[0035] After model inference, the output includes bounding box coordinates, detection confidence, and category label. A category discrimination function filters out non-human categories, retaining only detection results labeled as "human." A detection confidence threshold of 0.5 is set to filter out detection boxes with confidence scores below this threshold, ultimately outputting the detection result set. The detection results include bounding box coordinates. Confidence level and category information.

[0036] Step 102: Using the pose estimation model, process the bounding box information of the target object to extract a set of key point information and its confidence level of the target object.

[0037] For example, a lightweight pose estimation model can be used to extract the bounding box information detected in the above steps. Coordinates of key points on an individual body and their confidence levels; The value can be 17; the pose estimation model is based on the MobilePose architecture and outputs 17 standard human key points, including nose, eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles.

[0038] In one possible embodiment, step 102 includes: The target region corresponding to the bounding box information is cropped from the input image, and the size of the target region is normalized to adjust it to the input size required by the pose estimation model. Color space conversion and pixel value normalization are then performed to obtain the preprocessed image.

[0039] For example, the region corresponding to the human body bounding box is cropped from the input image. The cropped region is then normalized to the standard input size required by the pose estimation network. The standard input size has a height of 256 pixels and a width of 192 pixels. The normalized image is then subjected to color space conversion, changing the BGR format to RGB_PLANAR format, and pixel value normalization is performed.

[0040] The formula for pixel value normalization includes: ; in, This represents the color channels, including the R, G, and B channels. It is the mean vector. The standard deviation vector, mean vector, and standard deviation vector are normalized using parameters from the ImageNet dataset.

[0041] For example, mean vector Possible values Standard deviation vector It can take the value of .

[0042] The preprocessed image is input into the pose estimation model to obtain a dimension of The heatmap tensor. Among them, and These represent the height and width in the dimensions of the heatmap, respectively. Indicates the number of key points; For example, the heatmap size can be 1 / 4 of the input image size.

[0043] The heatmaps corresponding to each key point in the heatmap tensor are processed to determine the location of the maximum response value, which is then used as the coordinates of the key point in the heatmap, and the maximum response value is used as the confidence level of the key point. The coordinates in the heatmap are converted into cropping region coordinates, and the cropping region coordinates are converted into the original coordinates in the input image, thus obtaining a set of key points; the parameters of each key point in the set of key points include coordinate position and confidence level. For the Heatmap of key points: The conversion formula for converting coordinates in a heatmap to coordinates of the clipping region is as follows: ; ; in, Indicates the coordinates of the clipping region; , Indicates the original width and original height of the cropped area; Indicates the location of the maximum response value; The conversion formula for converting the cropped region coordinates to the original coordinates in the input image is as follows: ; ; in, Represents the x and y coordinates of the original coordinates in the input image; This indicates the coordinates of the top-left corner of the bounding box in the input image.

[0044] Understandable, and They can be used to represent the width and height of an input image, respectively.

[0045] Ultimately, the set of key points for the target object is obtained. The set of key points can contain up to 17 key points. Each key point... Includes original image coordinates and confidence level .

[0046] Step 103: Based on key point information, adaptively determine the optimized region of interest within the bounding box information of the input image.

[0047] Understandably, this step can utilize key point information to optimize the original bounding box, extract the optimized Region of Interest (ROI), and filter out invalid or non-target poses as needed.

[0048] In one possible embodiment, step 103 includes: performing a pose quality assessment on the set of key points of the target object, and detecting whether the target object is a valid target object; If the target object is a valid target object, then based on the set of key points, the coordinates of the bounding rectangle of its spatial distribution are calculated, and adaptive expansion is performed in the horizontal and vertical directions to obtain the optimized region of interest; wherein the range of the optimized region of interest does not exceed the range corresponding to the bounding box information.

[0049] For example, if the target object is a valid target object, adaptive region of interest (ROI) extraction is performed. This is mainly based on core trunk key points such as the shoulders and hips. Valid shoulder and hip key points that meet the confidence requirements are extracted, and the spatial distribution boundary (i.e., minimum bounding rectangle) of these valid key points is calculated. Subsequently, this rectangular region is adaptively expanded in the horizontal and vertical directions (e.g., slightly expanded proportionally) to cover the entire trunk, while ensuring that the expanded ROI does not exceed the range corresponding to the bounding box information of the input image. Compared with fixed region cropping, this embodiment can dynamically adjust the ROI according to the actual posture, effectively reducing interference from the background and extremities.

[0050] In one possible embodiment, the pose quality assessment includes a clear pose filtering mode; the pose quality assessment of the keypoint set of the target object, and the detection of whether the target object is a valid target object, includes: Based on the clear pose filtering mode and the confidence level of each key point, the number of valid key points in the key point set whose confidence level exceeds the key point confidence level threshold is determined; when the number of valid key points is lower than the first preset threshold, the pose of the target object is determined to be an invalid pose and filtered out; when the number of valid key points is greater than or equal to the first preset threshold, the target object is determined to be a valid target object.

[0051] For example, the keypoint confidence threshold can be 0.3. The first preset threshold represents the minimum number of valid keypoints; the first preset threshold can be 5.

[0052] For example, posture quality assessment can be performed based on the number of valid key points. The key point confidence threshold is set to 0.3, and the number of valid key points that meet the confidence requirement is counted. When the number of valid key points is less than the first preset threshold, it is determined to be an invalid posture and the target is marked as a posture filtering state.

[0053] For example, a clear pose filtering mode typically focuses only on the integrity of the pose. For instance, a threshold for the number of valid keypoints is set (i.e., the first preset threshold mentioned above). A target is only considered valid if its number of valid keypoints reaches or exceeds this threshold.

[0054] In one possible embodiment, the posture quality assessment includes a standing posture filtering mode; the posture quality assessment of the key point set of the target object, and the detection of whether the target object is a valid target object, includes: The target object undergoes a body uprightness check and a leg posture check to determine whether the target object is in a standing posture. If both the body uprightness check and the leg posture check are passed, the target object is determined to be in a standing posture, and thus the target object is determined to be a valid target object.

[0055] Among them, the body uprightness check requires that the vertical coordinates of the shoulder key points in the key point set be less than the vertical coordinates of the hip key points; the leg posture check requires that the vertical coordinates of the hip key points, knee key points and ankle key points in the key point set satisfy an increasing relationship from top to bottom.

[0056] For example, the standing posture filtering mode typically requires the target object to be in an upright standing posture. First, a body uprightness check is performed, requiring that the average ordinate of the left / right shoulder keypoints be less than the average ordinate of the left / right hip keypoints. Second, a leg posture check is performed, requiring that the ordinates of keypoints such as the hip, knee, and ankle on the same side increase from top to bottom. Only when both checks pass is the target considered to be in a valid standing posture.

[0057] Optionally, the pose quality assessment may include a full pose acceptance mode. This mode does not perform any pose-based filtering; subsequent processing is performed as long as the initial pose quality assessment is passed.

[0058] Step 104: Using a deep feature extraction model, deep features are extracted from the optimized region of interest to obtain a fixed-dimensional deep feature vector, and the deep feature vector is then normalized.

[0059] Understandably, this step can perform deep feature extraction on the optimized region of interest to obtain a fixed-dimensional, highly discriminative feature vector.

[0060] For example, the input image is cropped based on the optimized region of interest (ROI) coordinates, using the crop function provided by BMCV to extract the ROI image. The width and height of the ROI are calculated, and the crop coordinates are ensured to be valid, i.e., the width is greater than 0 and the height is greater than 0. The ROI is then resized to the standard input size of 224×224 pixels required by the ResNet network using bilinear interpolation.

[0061] Then, the SOPHON inference engine is invoked to load the quantized ResNet model file and perform forward inference to extract deep feature vectors. The ResNet network employs a residual connection architecture, including an initial convolutional layer, multiple residual blocks, and a global average pooling layer. A 256-dimensional fully connected layer is added at the end of the network as a feature extraction layer. The network outputs a 256-dimensional feature vector, which effectively represents the visual appearance features of the target object. The extracted feature vector is then normalized using the L2 norm. The square root of the sum of the squares of each element in the feature vector is calculated as the L2 norm. Each element of the feature vector is divided by this L2 norm, ensuring that the normalized feature vector has a magnitude of 1. It is understood that the L2-normalized feature vector facilitates subsequent cosine similarity calculations and eliminates the influence of differences in feature magnitudes.

[0062] In one possible embodiment, the training process of the deep feature extraction model includes: collecting a training dataset containing images of multiple categories, and performing preprocessing on each image in the training dataset to obtain standardized regions of interest as training samples; constructing a ResNet network architecture containing multiple residual blocks, and adding a fully connected layer of a specific dimension at the end of the ResNet network as a feature extraction and classification layer; wherein the residual blocks contain a multi-scale feature fusion structure; using the cross-entropy loss function and gradient descent optimization algorithm, using standardized region of interest images as input and image category labels as supervision signals, performing end-to-end training of the ResNet network; after the ResNet network training is completed, removing the classification layer of the ResNet network and retaining the feature extraction layer for outputting deep feature vectors.

[0063] For example, during the training phase of the ResNet network, a training dataset containing multiple object categories is collected. Each image in the training dataset undergoes preprocessing steps S101 to S103, including human detection, pose estimation, and region of interest (ROI) optimization, to obtain standardized ROIs as training samples. For instance, multiple object categories could include different styles of work clothes. A ResNet residual network architecture is constructed, with a 256-dimensional fully connected layer added at the end of the network as a feature extraction layer and a classification layer. The cross-entropy loss function is used as the training objective, and the network is trained end-to-end using a stochastic gradient descent optimization algorithm. During training, the gradient of the loss function with respect to the network parameters is calculated using backpropagation, and the network parameters are updated based on the gradient information, enabling the network to learn deep feature representations that distinguish different object categories. After training, the classification layer is removed, and the feature extraction layer is retained as the feature vector output, thus obtaining a deep feature extraction model suitable for similarity matching.

[0064] Step 105: Obtain the feature library and perform similarity matching between the feature vectors and the standard feature vectors in the feature library to determine the category of the target object.

[0065] Each standard feature in the feature library corresponds to a category.

[0066] For example, a feature library can be constructed. For each standard category to be identified (e.g., "Class A work clothes", "Class B safety helmet"), during the system initialization phase, multiple reference images for that category are collected (e.g., stored in the build / image / category name / directory). For each reference image, steps 101 to 104 are performed to extract its 256-dimensional L2-normalized feature vector. Then, the arithmetic mean of the feature vectors of all reference images for that category is calculated to obtain the standard feature vector for that category, and L2 normalization is also performed on it. It can be understood that this process generates a representative standard feature vector for each category.

[0067] In one possible embodiment, step 105 includes: using a cosine similarity-based matching method to calculate the similarity between the deep feature vector and each standard feature vector in the feature library; traversing all similarity calculation results and taking the category corresponding to the highest similarity calculation result as a candidate matching result; if the highest similarity calculation result is greater than or equal to a preset judgment threshold, then the target object is determined to belong to the candidate matching result.

[0068] For example, the decision threshold can represent a similarity decision threshold, such as a value of 0.4.

[0069] For example, for a target object to be identified, its normalized feature vector can be extracted through the above steps. Then calculate the normalized eigenvectors. With each standard feature vector in the feature library Cosine similarity between The calculation formula is as follows: ; Understandably, since the feature vectors have all been L2 normalized, cosine similarity simplifies to a vector dot product. After traversing all categories, the category with the highest similarity is selected. and their similarity values .

[0070] For example, if the category with the highest similarity similarity value If the value is greater than or equal to the judgment threshold, then the target object is determined to belong to the category. And mark it as compliant. If the category with the highest similarity... similarity value If the value is less than the threshold, it is judged as "non-compliant" or an unknown category.

[0071] This application proposes a target recognition method based on multi-stage deep feature extraction. It utilizes a target detection model to detect targets in an input image, obtaining bounding box information of the target object. A pose estimation model is then used to process the bounding box information to extract a set of keypoint information and their confidence scores. Based on the keypoint information, an optimized region of interest (ROI) is adaptively determined within the bounding box information of the input image. Deep feature extraction is performed on the optimized ROI to obtain a fixed-dimensional depth feature vector, which is then normalized. Finally, a feature library is acquired, and the feature vectors are compared with… The standard feature vectors in the feature library are used for similarity matching to determine the category of the target object; each standard feature in the feature library corresponds to a category. This scheme uses a multi-stage cascaded architecture of target detection, pose estimation, adaptive region localization, deep feature extraction and similarity matching. It can combine human key point-based region of interest optimization technology and multi-dimensional feature fusion mechanism to solve the technical problems of low recognition accuracy and robustness caused by background interference, inconsistent feature representation, poor cross-category adaptation and insufficient feature stability in existing technologies. Thus, it can achieve high-precision and high-robust recognition of multiple types of objects in complex scenes.

[0072] like Figure 2 As shown, Figure 2 A detailed flowchart of another target recognition method based on multi-stage deep feature extraction provided for embodiments of this application.

[0073] Step 201: Use an improved YOLOv11 network to perform human detection on the input image on an edge computing device. Introduce a lightweight CARAFE upsampling operator to improve feature extraction efficiency and output human bounding box coordinates, detection confidence and category information.

[0074] Step 202: At the edge, use the MobilePose lightweight pose network to identify key points of the detected target and generate 17 human pose information.

[0075] Step 203: Evaluate the posture quality based on the distribution of key points and eliminate invalid postures. Generate an adaptive ROI through the geometric relationship of areas such as the shoulder and hip to achieve accurate positioning of the target torso or main feature areas. Depending on the application, stand-up filtering, clear filtering or full posture mode can be selected to enhance the system's adaptability.

[0076] Step 204: Input the optimized ROI into the quantized ResNet feature extraction model on the edge device to obtain a stable 256-dimensional deep feature vector; construct a highly discriminative feature representation through local inference, pixel normalization and L2 normalization to ensure high-performance feature encoding can be completed at the edge.

[0077] Step 205: Compare the features of the object to be tested with the pre-built standard feature library in the edge device using cosine similarity, output the most similar category in real time, and complete the compliance or non-compliance determination based on the threshold.

[0078] For a description of steps 201 to 205, please refer to the above embodiments, which will not be repeated here.

[0079] This application provides a target recognition method based on multi-stage deep feature extraction, achieving high-precision recognition of multiple object types in complex scenes through a multi-stage cascaded architecture. First, an improved YOLOv11 network is used to introduce a lightweight CARAFE upsampling operator to enhance feature extraction efficiency, providing high-quality target regions for subsequent processing through accurate target detection. Second, human pose estimation technology is innovatively introduced, achieving intelligent and precise localization of regions of interest based on the spatial distribution information of 17 key points. Compared with traditional fixed-ratio cropping methods, this method can dynamically adjust the region of interest according to the actual pose, significantly reducing the influence of background interference and irrelevant parts. Third, a hierarchical region of interest extraction strategy and a multi-mode pose filtering mechanism are designed. When key point detection is effective, a pose-guided precise localization mode is adopted, ensuring the stability of the system in various complex scenes. Furthermore, a completely consistent processing flow is used in the feature library construction stage and the target detection stage, ensuring that standard features and test features are extracted from homogeneous regions of interest, making feature matching more reliable. Finally, this invention employs a lightweight MobilePose pose estimation network and a self-trained ResNet feature extraction network, achieving real-time processing capabilities while ensuring recognition accuracy. The complete processing time for a single image is typically within 70 milliseconds, meeting the requirements for real-time video stream processing. It is particularly suitable for application scenarios with high requirements for recognition accuracy and real-time performance, such as smart factories and video analytics.

[0080] The steps described above are for clarity only. In implementation, they can be combined into one step, or some steps can be broken down into multiple steps, as long as they involve the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0081] Another embodiment of this application proposes a target recognition system based on multi-stage deep feature extraction. The details of this target recognition system based on multi-stage deep feature extraction are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this example. Figure 3 This is a schematic diagram of the structure of a target recognition system based on multi-stage deep feature extraction proposed in this embodiment, including: The object detection module 310 is used to perform object detection on the input image using an object detection model to obtain the bounding box information of the target object in the input image; The key point extraction module 320 is used to process the bounding box information of the target object using the pose estimation model in order to extract a set of key point information and their confidence scores of the target object. The region optimization module 330 is used to adaptively determine the optimized region of interest within the bounding box information of the input image based on key point information; The deep feature extraction module 340 is used to extract deep features from the optimized region of interest using a deep feature extraction model, obtain a fixed-dimensional deep feature vector, and normalize the deep feature vector. The similarity matching module 350 is used to obtain a feature library and perform similarity matching between the feature vector and the standard feature vector in the feature library to determine the category of the target object; wherein, each standard feature in the feature library corresponds to a category.

[0082] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above method embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.

[0083] It is worth mentioning that all modules and units involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units do not exist in this embodiment.

[0084] Another embodiment of this application provides an electronic device, such as Figure 4 As shown, it includes a processor 41 and a memory 42. The memory 42 stores instructions that the processor 41 can execute. When the processor 41 is configured to execute the instructions, the electronic device can implement a target recognition method based on multi-stage deep feature extraction as described in the above method embodiment.

[0085] The memory and processor are connected via a bus, which includes any number of interconnecting buses and bridges, connecting various circuits of one or more processors and the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0086] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0087] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, can implement a target recognition method based on multi-stage deep feature extraction as described in the above method embodiments.

[0088] That is, those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (such as a microcontroller, chip, etc.) or processor to execute all or part of the steps of the method described in the method embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0089] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application. For those skilled in the art, several improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A target recognition method based on multi-stage deep feature extraction, characterized in that, The method includes: Using an object detection model, object detection is performed on the input image to obtain the bounding box information of the target objects in the input image; Using a pose estimation model, the bounding box information of the target object is processed to extract a set of key point information and their confidence scores. Based on key point information, the region of interest is adaptively determined within the bounding box information of the input image; Using a deep feature extraction model, deep features are extracted from the optimized region of interest to obtain a fixed-dimensional deep feature vector, and the deep feature vector is then normalized. A feature library is obtained, and the feature vectors are matched with the standard feature vectors in the feature library to determine the category of the target object; where each standard feature in the feature library corresponds to a category.

2. The method of claim 1, wherein, The target detection model is an improved YOLO series single-stage deep convolutional neural network; The object detection model includes the content-aware feature reassembly upsampling operator CARAFE, and the CARAFE upsampling process includes: The low-resolution input feature map is convolved to compress its number of channels to the target value, resulting in a compressed feature map; the calculation formula is as follows: ； wherein, represents an input feature map, represents a compressed feature map, is a 1x1 convolution matrix, is an index of the input feature map; Obtain the upsampling ratio and upsampling size, and predict the upsampling kernel of the compressed feature map through a convolutional layer; The upsampling kernel is normalized to obtain the normalized upsampling kernel; Perform channel and spatial integration operations, and use the normalized upsampling kernel to upsample the compressed feature map to obtain the upsampled feature map; the calculation formula is as follows: ； wherein ) represents a channel and space integration operation.​ 3. The method according to claim 1, characterized in that, The process of using a pose estimation model to process the bounding box information of the target object to extract a set of key point information and their confidence scores includes: The target region corresponding to the bounding box information is cropped from the input image, and its size is normalized to be adjusted to the input size required by the pose estimation model. Color space conversion and pixel value normalization are then performed to obtain the preprocessed image. The formula for pixel value normalization includes: ； in, This represents the color channels, including the R, G, and B channels. It is the mean vector. The standard deviation vector, mean vector, and standard deviation vector are normalized using the parameters from the ImageNet dataset; The preprocessed image is input into the pose estimation model to obtain a dimension of The heatmap tensor; where... and These represent the height and width in the dimensions of the heatmap, respectively. Indicates the number of key points; The heatmaps corresponding to each key point in the heatmap tensor are processed to determine the location of the maximum response value, which is then used as the coordinates of the key point in the heatmap, and the maximum response value is used as the confidence level of the key point. The coordinates in the heatmap are converted into cropping region coordinates, and the cropping region coordinates are converted into the original coordinates in the input image, thus obtaining a set of key points; the parameters of each key point in the set of key points include coordinate position and confidence level. The conversion formula for converting coordinates in a heatmap to coordinates of the clipping region is as follows: ； ； in, Indicates the coordinates of the clipping region; , Indicates the original width and original height of the cropped area; Indicates the location of the maximum response value; The conversion formula for converting the cropped region coordinates to the original coordinates in the input image is as follows: ； ； in, Represents the x and y coordinates of the original coordinates in the input image; This indicates the coordinates of the top-left corner of the bounding box in the input image.

4. The method according to claim 3, characterized in that, The step of adaptively determining and optimizing the region of interest within the bounding box information of the input image based on key point information includes: Perform pose quality assessment on the set of key points of the target object to determine whether the target object is a valid target object; If the target object is a valid target object, then based on the set of key points, the coordinates of the bounding rectangle of its spatial distribution are calculated, and adaptive expansion is performed in the horizontal and vertical directions to obtain the optimized region of interest; wherein the range of the optimized region of interest does not exceed the range corresponding to the bounding box information.

5. The method according to claim 4, characterized in that, The posture quality assessment includes a clear posture filtering mode; the posture quality assessment of the key point set of the target object, and the detection of whether the target object is a valid target object, includes: Based on the clear pose filtering pattern and the confidence level of each keypoint, determine the number of valid keypoints in the keypoint set whose confidence level exceeds the keypoint confidence threshold; When the number of valid key points is lower than the first preset threshold, the pose of the target object is determined to be an invalid pose and filtered out. When the number of valid key points is greater than or equal to the first preset threshold, the target object is determined to be a valid target object.

6. The method according to claim 4, characterized in that, The posture quality assessment includes a standing posture filtering mode; the posture quality assessment of the key point set of the target object, and the detection of whether the target object is a valid target object, includes: The target object undergoes body uprightness and leg posture checks to determine whether it is in a standing posture. The body uprightness check requires that the vertical coordinates of the shoulder key points in the key point set are less than the vertical coordinates of the hip key points. The leg posture check requires that the vertical coordinates of the hip key points, knee key points, and ankle key points in the key point set satisfy a top-to-bottom increasing relationship. When both the body uprightness check and the leg posture check are passed, the target object is determined to be in a standing posture, and thus the target object is determined to be a valid target object.

7. The method according to claim 1, characterized in that, The step of acquiring a feature library and performing similarity matching between deep feature vectors and standard feature vectors in the feature library to determine the category of the target object includes: The matching method based on cosine similarity calculates the similarity between the deep feature vector and each standard feature vector in the feature library. Iterate through all similarity calculation results and take the category corresponding to the highest similarity calculation result as the candidate matching result; If the highest similarity calculation result is greater than or equal to the preset judgment threshold, then the target object is determined to belong to the candidate matching result.

8. The method according to claim 1, characterized in that, The training process of a deep feature extraction model includes: A training dataset containing images of various categories is collected, and preprocessing is performed on each image in the training dataset to obtain standardized regions of interest as training samples. A ResNet network architecture containing multiple residual blocks is constructed, and a fully connected layer of a specific dimension is added at the end of the ResNet network as a feature extraction and classification layer; wherein, the residual block contains a multi-scale feature fusion structure; Using the cross-entropy loss function and gradient descent optimization algorithm, the ResNet network is trained end-to-end with standardized region of interest images as input and image category labels as supervision signals. After the ResNet network is trained, the classification layer of the ResNet network is removed, and the feature extraction layer is retained for outputting deep feature vectors.

9. An electronic device, characterized in that, include: The processor and memory, wherein the memory stores instructions executable by the processor, and the processor is configured to, when executing the instructions, enable the electronic device to implement a target recognition method based on multi-stage deep feature extraction as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement a target recognition method based on multi-stage deep feature extraction as described in any one of claims 1 to 8.

Images