A target fine-grained identification method and device

Abstract

This invention provides a method and apparatus for fine-grained target recognition. The method includes: acquiring an original image to be recognized; preprocessing the original image to obtain a preprocessed image; performing target object detection on the preprocessed image to obtain target object detection data; performing local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data; extracting fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data; and recognizing the target object based on the target object detection data and the fine-grained feature data to obtain a target object recognition result. This invention can improve the accuracy of target object recognition in images, accurately identify subtle differences such as different models and versions, and provide high-quality target object recognition results.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and also to a method and apparatus for fine-grained target recognition. Background Technology

[0002] In current machine vision applications, image target recognition technology has been widely applied to many important fields such as security monitoring, industrial inspection, and military equipment identification. As application scenarios continue to expand, the accuracy requirements for target recognition are also increasing, especially in specific fields such as equipment identification. It is not only necessary to accurately identify the category of the target object, but also to precisely distinguish fine-grained features such as different models and versions of the target.

[0003] However, existing image target recognition technologies generally suffer from insufficient accuracy. Traditional target recognition algorithms tend to focus on extracting the overall contour features of the target, with weak capabilities in capturing edge details and subtle local differences. Even when some algorithms incorporate deep learning models, the lack of customized optimization for fine-grained recognition leads to confusion and misjudgment when faced with targets that look similar but have subtle differences, failing to meet the practical application requirements of high-precision recognition. For example, in equipment target recognition, existing technologies struggle to accurately distinguish between different improved models of the same type of equipment, greatly complicating subsequent analysis and decision-making and limiting the further application of machine vision technology in high-precision scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a fine-grained target recognition method and apparatus to improve the accuracy of target object recognition in images.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A first aspect of the present invention provides a method for fine-grained target identification, comprising:

[0007] Obtain the original image to be identified;

[0008] The original image is preprocessed to obtain a preprocessed image;

[0009] Target object detection is performed on the preprocessed image to obtain target object detection data;

[0010] Based on the target object detection data, local feature enhancement processing of the target object is performed to obtain local feature enhancement data;

[0011] Based on the target object detection data and the local feature enhancement data, fine-grained feature extraction of the target object is performed to obtain fine-grained feature data;

[0012] The target object is identified based on the target object detection data and the fine-grained feature data to obtain the target object identification result.

[0013] Optionally, the original image is preprocessed to obtain a preprocessed image, including:

[0014] The original image is denoised to obtain a denoised image;

[0015] The denoised image is then normalized to obtain a normalized image;

[0016] The normalized image is subjected to illumination compensation processing to obtain a preprocessed image.

[0017] Optionally, target object detection is performed on the preprocessed image to obtain target object detection data, including:

[0018] The preprocessed image is divided into multi-scale grids to obtain multi-scale grid data;

[0019] Based on the multi-scale grid data, the bounding box regression offset data is obtained;

[0020] The bounding box regression offset data is subjected to coordinate transformation to obtain coordinate transformed data;

[0021] The coordinate transformation data is filtered to obtain the target object detection data.

[0022] Optionally, based on the target object detection data, local feature enhancement processing of the target object is performed to obtain local feature enhancement data, including:

[0023] The original image is cropped based on the target object detection data to obtain a local image block;

[0024] Edge contours are extracted from the local image blocks to obtain the edge contours;

[0025] The edge contour is subjected to edge feature mapping to obtain the edge feature tensor;

[0026] Feature fusion is performed on the edge feature tensor to obtain local feature enhancement data.

[0027] Optionally, fine-grained feature extraction of the target object is performed based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, including:

[0028] The target object detection data is divided into sub-regions to obtain sub-region data;

[0029] Local fine-grained features are extracted from the sub-region data to obtain local fine-grained features;

[0030] The target object detection data and the local fine-grained features are fused to obtain fine-grained feature data.

[0031] Optionally, local fine-grained feature extraction is performed on the sub-region data to obtain local fine-grained features, including:

[0032] The sub-region data is subjected to a first pooling process to obtain weighted features;

[0033] The weighted features are subjected to a second pooling process to obtain local fine-grained features.

[0034] Optionally, target object identification is performed based on the target object detection data and the fine-grained feature data to obtain target object identification results, including:

[0035] Based on the target object detection data and the fine-grained feature data, a multi-dimensional feature vector is obtained;

[0036] The target object is identified based on the multi-dimensional feature vector, and the target object identification result is obtained.

[0037] A second aspect of the present invention provides a target fine-grained identification device, comprising:

[0038] The acquisition module is used to acquire the original image to be identified;

[0039] The processing module is used to preprocess the original image to obtain a preprocessed image; perform target object detection on the preprocessed image to obtain target object detection data; perform local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data; extract fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data; and perform target object recognition based on the target object detection data and the fine-grained feature data to obtain a target object recognition result.

[0040] A third aspect of the present invention provides a computing device, comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described in the first aspect.

[0041] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect.

[0042] The above-described solution of the present invention has at least the following beneficial effects:

[0043] The above-described solution of the present invention acquires the original image to be identified, preprocesses the image to obtain a preprocessed image, then performs target object detection on the preprocessed image to obtain target object detection data, performs local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data, extracts fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, and finally performs target object recognition based on the target object detection data and the fine-grained feature data to obtain the target object recognition result. This can improve the accuracy of target object recognition in images, accurately identify subtle differences such as different models and versions, and provide high-quality target object recognition results. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the target fine-grained identification method in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the target fine-grained identification device in an embodiment of the present invention. Detailed Implementation

[0046] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0047] like Figure 1 As shown, an embodiment of the present invention proposes a fine-grained target identification method, comprising the following steps:

[0048] Step 101: Obtain the original image to be identified;

[0049] Step 102: Preprocess the image to obtain a preprocessed image;

[0050] Step 103: Perform target object detection on the preprocessed image to obtain target object detection data;

[0051] Step 104: Perform local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data;

[0052] Step 105: Extract fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data;

[0053] Step 106: Based on the target object detection data and the fine-grained feature data, perform target object identification to obtain the target object identification result.

[0054] The fine-grained target recognition method proposed in this invention involves acquiring the original image to be recognized, preprocessing the image to obtain a preprocessed image, performing target object detection on the preprocessed image to obtain target object detection data, performing local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data, extracting fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, and finally recognizing the target object based on the target object detection data and the fine-grained feature data to obtain the target object recognition result. This method can improve the accuracy of target object recognition in images, accurately identify subtle differences such as different models and versions, and provide high-quality target object recognition results.

[0055] In an optional embodiment of the present invention, in step 101, an original image to be identified is obtained, the original image containing the target object to be identified.

[0056] In an optional embodiment of the present invention, step 102, which involves preprocessing the original image to obtain a preprocessed image, may include:

[0057] Step 1021: Denoise the original image to obtain a denoised image;

[0058] Specifically, it can be done through The original image is denoised to obtain a denoised image; among which, To remove pixels from an image Pixel value at that location, This is the median calculation function. pixels in the original image Pixel value at that location, is the window offset, k is the half length of the filter window, and n is the size of the filter window (an odd number, such as 3×3 or 5×5).

[0059] Step 1022: Normalize the denoised image to obtain a normalized image;

[0060] Specifically, the purpose of normalization is to unify denoised images of different sizes into a fixed size. Here, the size of the denoised image is... ( For width, (Height), target size (i.e., the size of the normalized image) is W×H, when the aspect ratio of the denoised image size... Aspect ratio larger than the target size At that time, the width of the denoised image is cropped, retaining the central area, and the width of the denoised image becomes... When the aspect ratio of the denoised image size Smaller than target size When the aspect ratio is set, crop the height direction of the denoised image, retain the central area, and the height becomes... ; Scale the cropped image to W×H to obtain a scaled image; through The normalized image is obtained, where, To normalize the pixels in the image Pixel value at that location, To remove pixels from an image Pixel value at that location, This is a rounding function. The width of the denoised image. The height of the denoised image is given by denoising, W is the target width (i.e., the width of the normalized image), and H is the target width-to-height ratio (i.e., the height of the normalized image).

[0061] Step 1023: Perform illumination compensation processing on the normalized image to obtain a preprocessed image.

[0062] Specifically, illumination compensation processing can solve problems such as uneven brightness, overexposure, and underexposure in images, improving image quality and the accuracy of subsequent target recognition. Here, the pixel values ​​in the normalized image are first normalized to the range of [0,1]; through... Perform a transformation to obtain a transformed image, where... To transform pixels in an image Pixel value at that location, To normalize the pixels in the image Pixel value at that location, The gamma coefficient has a range of values ​​of 100. Finally, the pixel values ​​in the transformed image are restored to the range of [0, 255] to obtain the preprocessed image.

[0063] Here, by performing preprocessing operations such as denoising, size normalization, and illumination compensation on the original image, interference caused by environmental factors during the original image shooting process is eliminated, providing high-quality image data for subsequent feature extraction and object detection.

[0064] In an optional embodiment of the present invention, step 103, which involves performing target object detection on the preprocessed image to obtain target object detection data, may include:

[0065] Step 1031: Divide the preprocessed image into multi-scale grids to obtain multi-scale grid data;

[0066] Specifically, multi-scale grid partitioning involves dividing the preprocessed image into multi-scale grids (such as 8×8, 16×16, 32×32), with each grid predicting K anchor boxes.

[0067] Step 1032: Obtain bounding box regression offset data based on the multi-scale grid data;

[0068] Specifically, the multi-scale grid data is mapped to the convolutional output tensor through the detection head convolutional layer, as follows:

[0069] pass The convolution output tensor is obtained, where, To output the tensor of the convolution, For row and column indices of grids in multi-scale grid data, The number of output channels for the backbone network. For multi-scale grid data, grid The eigenvalue of the c-th channel at position c. For convolution kernel weights, This is the convolution bias term;

[0070] The convolution output tensor is split according to dimensionality to obtain the original offset tensor, specifically: through... , , , Obtain the original offset tensor; , , , All are the original offset tensors. For the convolution output tensor, K is the number of anchor boxes per grid. d represents the k-th anchor box in the i-th row and j-th column of the grid, and d is the convolution output channel index.

[0071] The original offset tensor is activated to obtain the bounding box regression offset data. Specifically: through... , Obtain the center offset from the bounding box regression offset data, through , Obtain the width and height offsets from the bounding box regression offset data; where, This refers to the lateral offset in the bounding box regression offset data. This refers to the vertical offset in the bounding box regression offset data. This refers to the width offset in the bounding box regression offset data. The high offset in the bounding box regression offset data. For activation function, For the exponent term, , , , All are offset original tensors.

[0072] Step 1033: Perform coordinate transformation on the bounding box regression offset data to obtain coordinate transformation data;

[0073] Specifically, the bounding box regression offsets in the bounding box regression offset data By combining predefined anchor boxes and grid coordinates, the coordinates are converted into image pixel-level bounding box coordinates (center + width and height), and then converted into top left / bottom right corner coordinates (the basic form of candidate boxes), thus obtaining the coordinate transformation data.

[0074] Step 1034: Filter the coordinate transformation data to obtain target object detection data.

[0075] Specifically, coordinate transformation data is filtered according to preset filtering conditions, and coordinate transformation data that meets the preset filtering conditions is used as target object detection data. Here, the preset filtering conditions may include: the confidence of anchor boxes in coordinate transformation data is not less than the confidence threshold, and duplicate candidate boxes in coordinate transformation data are removed.

[0076] Here, through the above steps, the initial detection and localization of the target object can be completed efficiently, providing a clear target area for fine-grained recognition.

[0077] In an optional embodiment of the present invention, step 104, performing local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data, may include:

[0078] Step 1041: Crop the original image based on the target object detection data to obtain a local image block;

[0079] Specifically, the original image is cropped according to the coordinates of the target candidate region in the target object detection data to obtain local image blocks, i.e., candidate regions. This avoids calculating the entire image and can improve processing efficiency.

[0080] Step 1042: Extract the edge contour of the local image block to obtain the edge contour;

[0081] Specifically, through This yields the edge contour; where, This represents the edge contour; edge pixels are 255, and non-edge pixels are 0. Pixels in a local image The gradient magnitude after non-maximum suppression is obtained by... We obtained, among which, Pixels in a local image gradient magnitude, The magnitude of the gradient is the value of the adjacent pixels along the gradient direction. To preset a low threshold, This is a preset high threshold.

[0082] Step 1043: Perform edge feature mapping on the edge contour to obtain the edge feature tensor;

[0083] Specifically, through Multi-scale scaling of the edge contour yields a scaled contour; through Edge feature mapping is performed on the scaled contour to obtain edge feature tensors at different scales; where... The scaled contour at the k-th scale. Let k be the pixel coordinates at the k-th scale. It is a bilinear interpolation function. For edge contour, is the scaling factor for the k-th scale (k=1,2,3). For the interpolated coordinates at the k-th scale, Let be the edge feature tensor at the k-th scale, 1 be the feature channel index, and 255 be the normalization coefficient.

[0084] Step 1044: Perform feature fusion on the edge feature tensor to obtain local feature enhancement data.

[0085] Specifically, through Upsampling the edge feature tensor yields the first-scale features; through Downsampling of the edge feature tensor yields the second-scale features; through Feature fusion is performed on the first-scale features and the second-scale features to obtain locally enhanced data. Among them, For the first scale feature, It is a bilinear interpolation function. This refers to the small-scale edge feature tensor within the edge feature tensor. For interpolation mapping coordinates, u 2 ∈[0,111] , v 2 ∈[0,111] , This is a second-scale feature. This refers to the large-scale edge feature tensor in the edge feature tensor. The coordinates are in the edge feature tensor. To enhance data for local features, , , All are weights. coordinates Edge feature tensor at the location, coordinates The first scale feature at the location, coordinates The second-scale feature at that location.

[0086] In an optional embodiment of the present invention, step 105, which involves extracting fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, may include:

[0087] Step 1051: Divide the target object detection data into sub-regions to obtain sub-region data;

[0088] Specifically, for each candidate bounding box in the target object detection data, the corresponding image region is divided into K×K non-overlapping / overlapping regions of interest. The specific division method can be: first, normalize each candidate bounding box in the target object detection data to the range [0,1]; then... , , , Obtain the coordinates of the sub-region; the sub-region data includes the sub-region coordinates. Here, Let x be the x-coordinate of the sub-region in row m and column n. The ordinate of the sub-region in row m and column n is... The width of the sub-region in row m and column n. The height of the sub-region in row m and column n is... To detect the starting coordinates of the image in the target object data, The total width of the image in the target object detection data. The total height of the image in the target object detection data is K, where K is the number of regions of interest (which can be 4 or 8), m is the row index, and n is the column index.

[0089] Step 1052: Extract local fine-grained features from the sub-region data to obtain local fine-grained features;

[0090] In an optional embodiment of the present invention, step 1052 includes:

[0091] Step 10521: Perform a first pooling process on the sub-region data to obtain weighted features;

[0092] Specifically, through Perform a first global average pooling on the sub-region data to obtain the first global average pooled data; through The first global max pooling is performed on the sub-region data to obtain the first global max pooled data; the first global average pooled data and the first global max pooled data are mapped separately and then summed to obtain the weighted feature, specifically through... We obtain the weighted features. Among them, This is the first global average pooling data. This is the first global max-pooled data. As a weighted feature, For sub-region data, The height of the local feature map. Where is the width of the local feature map, C is the number of feature channels, i is the pixel index in the height dimension of the feature map, and j is the pixel index in the width dimension of the feature map. for All channel values ​​at the i-th row and j-th column position. This is a global average pooling operation. This is a global max pooling operation. This is the activation function.

[0093] Step 10522: Perform a second pooling process on the weighted features to obtain local fine-grained features.

[0094] Specifically, through Perform a second global average pooling on the weighted features to obtain the second global average pooled data; through A second global max pooling is performed on the weighted features to obtain the second global max pooling data; the second global average pooling data and the second global max pooling data are mapped separately and then summed to obtain local fine-grained features, specifically through... Local fine-grained features are obtained. Among them, This is the second global average pooling data. This is the second global max-pooled data. It is a local fine-grained feature. For weighted features, u is the pixel index in the height dimension of the feature map, and v is the pixel index in the width dimension of the feature map. for The values ​​of all channels at the i-th row and j-th column, where C is the number of feature channels. For neural network layers, This is a global average pooling operation. This is a global max pooling operation. This is the activation function.

[0095] Step 1053: Perform feature fusion on the target object detection data and the local fine-grained features to obtain fine-grained feature data.

[0096] Specifically, through Global feature vectorization is performed on the target object detection data to obtain a global feature vector; through Fine-grained feature data is obtained. Let be the global feature vector of the i-th candidate box in the target object detection data. This is a global average pooling operation. Align pooling for regions of interest. This is a global feature map in the target object detection data. Let C be the i-th target candidate box, and C be the number of feature channels. For the local fine-grained features of the i-th candidate box, As the first dynamic fusion weight, For dimensionality reduction fully connected layers, Let be the local fine-grained feature vector of the i-th candidate box.

[0097] In an optional embodiment of the present invention, step 106, which involves identifying the target object based on the target object detection data and the fine-grained feature data to obtain the target object identification result, may include:

[0098] Step 1061: Obtain a multi-dimensional feature vector based on the target object detection data and the fine-grained feature data;

[0099] Specifically, target object detection data and fine-grained feature data are fused to construct a multi-dimensional feature vector. Here, through... We obtain multi-dimensional feature vectors, where, For multi-dimensional feature vectors, For the second dynamic fusion weight, Let be the global feature vector of the i-th candidate box in the target object detection data, and be the fine-grained feature data after dimensionality reduction. get, For dimensionality reduction fully connected layers, represents the local fine-grained features of the i-th candidate box.

[0100] Step 1062: Target object recognition is performed based on the multi-dimensional feature vector to obtain the target object recognition result.

[0101] Specifically, through The target object recognition result is obtained, where, For the target object identification results, As a text decoder, it decodes feature vectors into natural language descriptions (such as "Equipment Model: XX-1, Version: V2.0, Core Component: XX Sensor"). This is the fine-grained feature description vector for the i-th target candidate box, obtained through... We obtained, among which, For category embedding vectors, For feature mapping layer, It is a multi-dimensional feature vector.

[0102] A specific embodiment of the target fine-grained recognition method of the present invention includes:

[0103] Step 111: Obtain the original image to be identified;

[0104] The original image to be identified can be provided by the user or downloaded from the internet, and there can be one or more original images.

[0105] Step 112: Preprocess the image to obtain a preprocessed image;

[0106] Step 113: Perform target object detection on the preprocessed image to obtain target object detection data;

[0107] A global scan is performed on the preprocessed image to initially determine the location information of target objects in the image and output target candidate regions. By dividing the image into a grid and directly predicting the bounding box and class probability of the target, the initial detection and localization of the target can be efficiently completed, providing clear target regions for fine-grained recognition.

[0108] Step 114: Perform local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data;

[0109] Local feature enhancement processing is performed on the target candidate region in the target object detection data. By extracting the edge contour details of the target and fusing edge features at different scales, the recognizability of the target edge features is enhanced, providing key feature support for distinguishing subtle differences between targets.

[0110] Step 115: Extract fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data;

[0111] Building upon target candidate regions in target object detection data and enhanced edge features in local feature enhancement data, fine-grained feature extraction is introduced. Specifically, target candidate regions are divided into multiple regions of interest (ROIs), focusing on the local detailed features of the target (such as the shape of specific equipment components and markings). Through in-depth mining and fusion of local features, accurate extraction of fine-grained target features is achieved. Simultaneously, these fine-grained features are fused with global features from the target object detection data to construct a multi-dimensional feature vector, enhancing the richness of feature representation.

[0112] Step 116: Based on the target object detection data and the fine-grained feature data, perform target object identification to obtain the target object identification result.

[0113] The fused multi-dimensional feature vector is input into the classifier, and the Softmax classification function is used to achieve accurate classification of the target category. At the same time, the target's location information, confidence level, and fine-grained feature description (such as model, version, etc.) are output, and finally the high-precision intelligent recognition and detection process is completed.

[0114] The target fine-grained recognition method proposed in this invention effectively overcomes the shortcomings of traditional algorithms in fine-grained feature extraction by enhancing edge feature information and achieving high-precision fine-grained recognition. This enables machine vision systems to capture more accurate details of targets, improving overall recognition accuracy by more than 30% compared to existing technologies. Through deep feature mining and multi-dimensional feature fusion of the target's local region of interest, it can accurately distinguish subtle differences such as different models and versions of target objects. For example, in equipment recognition scenarios, it can accurately identify improved or upgraded versions of the same type of equipment, solving the problem of "difficulty in distinguishing similar objects" in existing technologies. The recognition results output by this invention not only include the target's category and location information but also include fine-grained feature descriptions and high-confidence scores, providing high-quality and reliable basic data support for subsequent decision analysis and data statistics, thus broadening the application scope of machine vision technology in high-precision scenarios.

[0115] like Figure 2 As shown, an embodiment of the present invention provides a target fine-grained identification device 200, comprising:

[0116] The acquisition module 201 is used to acquire the original image to be identified;

[0117] The processing module 202 is used to preprocess the original image to obtain a preprocessed image; perform target object detection on the preprocessed image to obtain target object detection data; perform local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data; perform fine-grained feature extraction on the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data; and perform target object recognition based on the target object detection data and the fine-grained feature data to obtain a target object recognition result.

[0118] Optionally, the original image is preprocessed to obtain a preprocessed image, including:

[0119] The original image is denoised to obtain a denoised image;

[0120] The denoised image is then normalized to obtain a normalized image;

[0121] The normalized image is subjected to illumination compensation processing to obtain a preprocessed image.

[0122] Optionally, target object detection is performed on the preprocessed image to obtain target object detection data, including:

[0123] The preprocessed image is divided into multi-scale grids to obtain multi-scale grid data;

[0124] Based on the multi-scale grid data, the bounding box regression offset data is obtained;

[0125] The bounding box regression offset data is subjected to coordinate transformation to obtain coordinate transformed data;

[0126] The coordinate transformation data is filtered to obtain the target object detection data.

[0127] Optionally, based on the target object detection data, local feature enhancement processing of the target object is performed to obtain local feature enhancement data, including:

[0128] The original image is cropped based on the target object detection data to obtain a local image block;

[0129] Edge contours are extracted from the local image blocks to obtain the edge contours;

[0130] The edge contour is subjected to edge feature mapping to obtain the edge feature tensor;

[0131] Feature fusion is performed on the edge feature tensor to obtain local feature enhancement data.

[0132] Optionally, fine-grained feature extraction of the target object is performed based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, including:

[0133] The target object detection data is divided into sub-regions to obtain sub-region data;

[0134] Local fine-grained features are extracted from the sub-region data to obtain local fine-grained features;

[0135] The target object detection data and the local fine-grained features are fused to obtain fine-grained feature data.

[0136] Optionally, local fine-grained feature extraction is performed on the sub-region data to obtain local fine-grained features, including:

[0137] The sub-region data is subjected to a first pooling process to obtain weighted features;

[0138] The weighted features are subjected to a second pooling process to obtain local fine-grained features.

[0139] Optionally, target object identification is performed based on the target object detection data and the fine-grained feature data to obtain target object identification results, including:

[0140] Based on the target object detection data and the fine-grained feature data, a multi-dimensional feature vector is obtained;

[0141] The target object is identified based on the multi-dimensional feature vector, and the target object identification result is obtained.

[0142] The target fine-grained recognition device proposed in this invention acquires the original image to be recognized, preprocesses the image to obtain a preprocessed image, then performs target object detection on the preprocessed image to obtain target object detection data, performs local feature enhancement processing on the target object based on the target object detection data to obtain local feature enhancement data, extracts fine-grained features of the target object based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, and finally performs target object recognition based on the target object detection data and the fine-grained feature data to obtain the target object recognition result. This device can improve the accuracy of target object recognition in images, accurately identify subtle differences such as different models and versions, and provide high-quality target object recognition results.

[0143] It should be noted that this device corresponds to the method described above, and all implementations in the method embodiments described above are applicable to the embodiments of this device and can achieve the same technical effect. Further details are omitted in this embodiment.

[0144] This invention also provides a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effects. Further details are omitted in this embodiment.

[0145] This invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method as described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effects. Further details are omitted in this embodiment.

[0146] It should be noted that in the apparatus and method of the present invention, the components or steps can obviously be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Furthermore, the steps for performing the above series of processes can naturally be performed in the order described and in chronological order, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel, overlapping, or independently of each other.

[0147] It should be noted that in the above embodiments, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments described above is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0148] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A fine-grained target recognition method, characterized in that, include: Obtain the original image to be identified; The original image is preprocessed to obtain a preprocessed image; Target object detection is performed on the preprocessed image to obtain target object detection data; Based on the target object detection data, local feature enhancement processing of the target object is performed to obtain local feature enhancement data; Based on the target object detection data and the local feature enhancement data, fine-grained feature extraction of the target object is performed to obtain fine-grained feature data; Target object identification is performed based on the target object detection data and the fine-grained feature data to obtain the target object identification result; Specifically, target object detection is performed on the preprocessed image to obtain target object detection data, including: The preprocessed image is divided into multi-scale grids to obtain multi-scale grid data; Based on the multi-scale grid data, the bounding box regression offset data is obtained; The bounding box regression offset data is subjected to coordinate transformation to obtain coordinate transformed data; The coordinate transformation data is filtered to obtain target object detection data; Specifically, the local feature enhancement processing of the target object is performed based on the target object detection data to obtain local feature enhancement data, including: The original image is cropped based on the target object detection data to obtain a local image block; Edge contours are extracted from the local image blocks to obtain the edge contours; The edge contour is subjected to edge feature mapping to obtain the edge feature tensor; Feature fusion is performed on the edge feature tensor to obtain local feature enhancement data; Specifically, fine-grained feature extraction of the target object is performed based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, including: The target object detection data is divided into sub-regions to obtain sub-region data; Local fine-grained features are extracted from the sub-region data to obtain local fine-grained features; The target object detection data and the local fine-grained features are fused to obtain fine-grained feature data; Specifically, local fine-grained feature extraction is performed on the sub-region data to obtain local fine-grained features, including: pass Perform a first global average pooling on the sub-region data to obtain the first global average pooled data; through Perform the first global max pooling on the sub-region data to obtain the first global max pooling data; through We obtain the weighted features; among them, This is the first global average pooling data. This is the first global max-pooled data. As a weighted feature, For sub-region data, The height of the local feature map. Where is the width of the local feature map, C is the number of feature channels, i is the pixel index in the height dimension of the feature map, and j is the pixel index in the width dimension of the feature map. for All channel values ​​at the i-th row and j-th column position. This is a global average pooling operation. This is a global max pooling operation. For neural network layers, For activation functions; pass Perform a second global average pooling on the weighted features to obtain the second global average pooled data; through Perform a second global max pooling on the weighted features to obtain the second global max pooling data; through Local fine-grained features are obtained; among them, This is the second global average pooling data. This is the second global max-pooled data. It is a local fine-grained feature. For weighted features, u is the pixel index in the height dimension of the feature map, and v is the pixel index in the width dimension of the feature map. for All channel values ​​at the u-th row and v-th column position.

2. The target fine-grained recognition method according to claim 1, characterized in that, The original image is preprocessed to obtain a preprocessed image, including: The original image is denoised to obtain a denoised image; The denoised image is then normalized to obtain a normalized image; The normalized image is subjected to illumination compensation processing to obtain a preprocessed image.

3. The target fine-grained recognition method according to claim 1, characterized in that, Based on the target object detection data and the fine-grained feature data, target object identification is performed to obtain target object identification results, including: Based on the target object detection data and the fine-grained feature data, a multi-dimensional feature vector is obtained; The target object is identified based on the multi-dimensional feature vector, and the target object identification result is obtained.

4. A fine-grained target recognition device, characterized in that, include: The acquisition module is used to acquire the original image to be identified; The processing module is used to preprocess the original image to obtain a preprocessed image; Target object detection is performed on the preprocessed image to obtain target object detection data; local feature enhancement processing of the target object is performed based on the target object detection data to obtain local feature enhancement data; Based on the target object detection data and the local feature enhancement data, fine-grained features of the target object are extracted to obtain fine-grained feature data; based on the target object detection data and the fine-grained feature data, target object recognition is performed to obtain target object recognition results. Specifically, target object detection is performed on the preprocessed image to obtain target object detection data, including: The preprocessed image is divided into multi-scale grids to obtain multi-scale grid data; Based on the multi-scale grid data, the bounding box regression offset data is obtained; The bounding box regression offset data is subjected to coordinate transformation to obtain coordinate transformed data; The coordinate transformation data is filtered to obtain target object detection data; Specifically, the local feature enhancement processing of the target object is performed based on the target object detection data to obtain local feature enhancement data, including: The original image is cropped based on the target object detection data to obtain a local image block; Edge contours are extracted from the local image blocks to obtain the edge contours; The edge contour is subjected to edge feature mapping to obtain the edge feature tensor; Feature fusion is performed on the edge feature tensor to obtain local feature enhancement data; Specifically, fine-grained feature extraction of the target object is performed based on the target object detection data and the local feature enhancement data to obtain fine-grained feature data, including: The target object detection data is divided into sub-regions to obtain sub-region data; Local fine-grained features are extracted from the sub-region data to obtain local fine-grained features; The target object detection data and the local fine-grained features are fused to obtain fine-grained feature data; Specifically, local fine-grained feature extraction is performed on the sub-region data to obtain local fine-grained features, including: pass Perform a first global average pooling on the sub-region data to obtain the first global average pooled data; through Perform the first global max pooling on the sub-region data to obtain the first global max pooling data; through We obtain the weighted features; among them, This is the first global average pooling data. This is the first global max-pooled data. As a weighted feature, For sub-region data, The height of the local feature map. Where is the width of the local feature map, C is the number of feature channels, i is the pixel index in the height dimension of the feature map, and j is the pixel index in the width dimension of the feature map. for All channel values ​​at the i-th row and j-th column position. This is a global average pooling operation. This is a global max pooling operation. For neural network layers, For activation functions; pass Perform a second global average pooling on the weighted features to obtain the second global average pooled data; through Perform a second global max pooling on the weighted features to obtain the second global max pooling data; through Local fine-grained features are obtained; among them, This is the second global average pooling data. This is the second global max-pooled data. It is a local fine-grained feature. For weighted features, u is the pixel index in the height dimension of the feature map, and v is the pixel index in the width dimension of the feature map. for All channel values ​​at the u-th row and v-th column position.

5. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 3.

Images