A fine-grained image recognition method and system

Abstract

The application discloses a fine-grained image recognition method and system, the method comprising: obtaining a target object image to be recognized; performing polar coordinate transformation on the target object image to map the target region distributed along the circumference into a horizontally flattened image; inputting the horizontally flattened image into a visual feature extraction flow and a semantic feature extraction flow respectively to generate a visual feature vector and a semantic text feature vector; calculating the similarity of the two feature vectors with each candidate target in a feature library respectively, and determining a visual first ranking and a semantic first ranking according to the similarity ranking; using an inverse fusion function to calculate the final fusion score of each candidate target, and outputting a recognition result according to the final fusion score. The application solves the problems of multi-modal feature dimension incommensurability leading to fusion failure, single-modal being susceptible to noise interference and lacking cross-validation means, and realizes high-precision fine-grained recognition in a single-sample scene without retraining the network.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a fine-grained image recognition method and system. Background Technology

[0002] In the fields of industrial visual inspection and public security evidence identification driven by computer vision and artificial intelligence, the accurate identification and retrieval of tiny target objects with complex geometric topological structures and special material optical properties is a core prerequisite for building intelligent identification systems. For example, targets such as bullet base stamping inscriptions and precision industrial stamped parts have surface morphology that carries decisive identification characteristics. However, due to the highly uncontrollable imaging environment, the scarcity of samples, and the extremely subtle differences between classes, this task has significant technical barriers compared to conventional macroscopic identification.

[0003] Currently, the mainstream approaches for recognizing radially arranged and metallic targets fall into two categories: traditional image processing and deep learning. The former relies on manually designed local feature descriptors combined with template matching for measurement; the latter uses convolutional neural networks for end-to-end classification, or borrows from optical character recognition, utilizing recurrent neural networks and attention mechanisms to decode character sequences in the target region. Furthermore, to address the directional differences encountered in actual data acquisition, these approaches generally rely on large-scale data augmentation strategies such as random rotation and affine transformations to expand the limited sample size and improve the model's generalization ability.

[0004] However, the above-mentioned solutions still have significant limitations in practical deployment. Metallic reflective surfaces are prone to random highlights and shadows, and wear on stamped characters leads to fluctuations in semantic confidence. Furthermore, visual and semantic features belong to different dimensions and numerical distribution spaces; direct linear weighting causes score skew, resulting in the fusion result essentially reflecting only a single modality. Simultaneously, in extreme scenarios with single samples and thousands of categories, the supervised network is highly susceptible to overfitting and collapse, and single-modality systems lack cross-domain prior cross-validation. This results in ambiguous retrieval confidence and a high false alarm rate when faced with highly similar models distinguished solely by microscopic texture or character spacing, making it difficult to meet the reliability standards required for practical identification. Summary of the Invention

[0005] This invention provides a fine-grained image recognition method and system that solves the problems of inability to directly fuse visual and semantic features in extreme scenarios with single samples and thousands of categories, and the lack of cross-domain cross-validation methods due to the susceptibility of single modalities to interference from metal surface highlights, shadows, or character wear, leading to ambiguous recognition confidence and high false alarm rates. It achieves the goal of flattening circumferentially distributed image features into a horizontal sequence without requiring any parameter retraining for the target category. Visual morphology and semantic text features are extracted in parallel through dual streams, and multimodal comprehensive decision-making is performed within the ranking space using a reciprocal fusion function. This effectively suppresses the interference of random noise from metal reflective surfaces on retrieval accuracy, improves the accuracy and robustness of fine-grained image recognition under single-sample conditions, and provides a highly reliable and interpretable automated recognition method for scenarios such as evidence tracing, industrial inspection, and ancient coin identification.

[0006] This invention provides a fine-grained image recognition method, the method comprising: Acquire an image of the target object to be identified; The target object image is subjected to polar coordinate transformation, and the target region distributed along the circumference in the target object image is mapped to a horizontally flattened image. The horizontally flattened image is input into a visual feature extraction stream to generate a visual feature vector; and the horizontally flattened image is input into a semantic feature extraction stream to generate a semantic text feature vector. The similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library is calculated respectively, and the visual first ranking and semantic first ranking of each candidate target are determined according to the similarity ranking. Based on the visual first ranking and the semantic first ranking of each candidate target, the final fusion score of each candidate target is calculated using the inverse fusion function, and the recognition result is output according to the final fusion score.

[0007] Optionally, the step of performing polar coordinate transformation on the target object image to map the target region distributed along the circumference in the target object image into a horizontally flattened image includes: The image of the target object is subjected to Gaussian smoothing to suppress specular highlight noise on the metal surface; The geometric center coordinates and maximum effective outer diameter of the target object image are automatically calculated based on the connected component analysis algorithm. A nonlinear polar coordinate mapping matrix is ​​constructed, and the geometric center coordinates and the maximum effective outer diameter are used as mapping parameters to remap the pixels arranged along the circumference in the target object image to a sequence of pixels in the horizontal direction, thereby generating the horizontally flattened image.

[0008] Optionally, the step of inputting the horizontally flattened image into a visual feature extraction stream to generate a visual feature vector includes: The horizontally flattened image is divided into multiple image patches; The multiple image patches are input into a deep neural network whose backbone network parameters are frozen; The deep neural network extracts global spatial correlation features between the image patches, and normalizes the extracted features to generate the visual feature vector.

[0009] Optionally, the step of inputting the horizontally flattened image into a semantic feature extraction stream to generate a semantic text feature vector includes: A pre-trained multimodal reasoning model is used to perform context inference and recognition of character information in the horizontally flattened image to generate a plain text sequence; The plain text sequence is processed into text vectorization, which transforms the plain text sequence into a high-dimensional dense vector, thereby generating the semantic text feature vector.

[0010] Optionally, the step of calculating the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, and determining the visual first ranking and semantic first ranking of each candidate target based on the similarity ranking, includes: Calculate the cosine similarity between the visual feature vector and the visual feature vector of each candidate target in the feature library, and sort them from high to low according to the cosine similarity to generate the visual first ranking corresponding to each candidate target; Calculate the similarity between the semantic text feature vector and the semantic feature vector of each candidate target in the feature library, and sort them according to the similarity results to generate the semantic first ranking corresponding to each candidate target.

[0011] Optionally, the step of calculating the final fusion score of each candidate target using the reciprocal fusion function includes: The visual first ranking and semantic first ranking are used as the core variables of the denominator and substituted into the reciprocal fusion function to calculate the sub-scores of the candidate target in the visual modality and semantic modality. The final fusion score is obtained by weighted summation of the individual scores.

[0012] Optionally, the step of using the visual first ranking and the semantic first ranking as core variables in the denominator includes: The visual first ranking and the semantic first ranking are summed with a smoothing penalty constant, and the summation results are used as the core variables of the denominator in the corresponding modality sub-item score calculation formula.

[0013] Optionally, the step of weighted summing of the individual scores includes: Obtain the image quality signal-to-noise ratio of the horizontally flattened image; The weighting coefficients for visual and semantic modalities are set based on the image quality signal-to-noise ratio, so that the modality with a higher signal-to-noise ratio receives a larger weighting coefficient. The scores of the sub-items are weighted and summed using the weighting coefficients.

[0014] Optionally, the step of setting the weight coefficients for the visual modality and the semantic modality based on the image quality signal-to-noise ratio includes: Obtain the ambient light intensity of the target object image acquisition environment; If the ambient light intensity is lower than a preset light intensity threshold, then based on the weight coefficient set for the signal-to-noise ratio, the weight coefficient corresponding to the semantic feature extraction stream is further increased, and the weight coefficient corresponding to the visual feature extraction stream is decreased.

[0015] Furthermore, to achieve the above objectives, embodiments of the present invention also provide a fine-grained image recognition system, the system comprising: The polar coordinate transformation module is used to acquire the target object image to be identified and perform polar coordinate transformation on the target object image to map the target area distributed along the circumference in the target object image into a horizontally flattened image. The visual feature extraction stream module is used to input the horizontally flattened image into the visual feature extraction stream to generate a visual feature vector. The semantic feature extraction stream module is used to input the horizontally flattened image into the semantic feature extraction stream to generate a semantic text feature vector. The heterogeneous metric fusion module is used to calculate the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, respectively. Based on the similarity ranking, the first visual ranking and the first semantic ranking of each candidate target are determined. Based on the first visual ranking and the first semantic ranking of each candidate target, the final fusion score of each candidate target is calculated using the reciprocal fusion function, and the recognition result is output based on the final fusion score.

[0016] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: 1. The reciprocal fusion mechanism based on ranking space uses the ranking position of each candidate target in the visual and semantic modal rather than the similarity value as the input variable for fusion. This eliminates the obstacle that visual features and semantic features cannot be directly weighted due to their different dimensions, and enables the two heterogeneous signals to participate fairly in the fusion decision under a unified measurement framework. This avoids the score skew effect caused by the numerical distribution dominance of a single modality.

[0017] 2. Through a dual-stream parallel heterogeneous feature extraction architecture, the visual feature extraction stream and the semantic feature extraction stream independently and without interference capture the microscopic topological morphology and stamped character semantic information of the target surface. Combined with the cross-validation function naturally achieved by ranking-based reciprocal fusion, when one modality is misjudged due to interference from specular highlights, shadows or character wear, the other modality is usually not affected by the same interference. It can automatically correct the ranking deviation of the former at the ranking level, so that the system can maintain stable recognition performance in environments with strong reflection noise.

[0018] 3. Based on the image signal-to-noise ratio and the ambient light intensity, the modal weight coefficients are dynamically adjusted. When specular highlights, shadows, or low-light interference are detected, the visual modal weights are automatically reduced and the semantic modal weights are correspondingly increased. This allows the fusion decision to adaptively tilt towards the modality with less interference. At the same time, the backbone network parameters are frozen and no parameter retraining is required for the target category. This avoids the risk of overfitting in scenarios with a large number of categories and single samples, giving the system good versatility and scalability. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the fine-grained image recognition method of the present invention. Figure 2 This is a schematic diagram of the framework of the fine-grained image recognition system of the present invention; Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present invention. Detailed Implementation

[0020] For tiny, fine-grained targets with radial topological structures and strong metallic reflectivity, in extreme retrieval scenarios involving single samples and thousands of categories, visual and semantic features cannot be directly fused due to dimensional incommensurability. Furthermore, single-modality fusion is susceptible to interference from metallic surface highlights, shadows, or character wear, lacking cross-domain cross-validation methods, leading to problems such as ambiguous recognition confidence and high false alarm rates. This embodiment maps the target region distributed along the circumference into a horizontally flattened image through polar coordinate transformation, eliminating the rotation-invariant learning burden of subsequent feature extraction. Based on this, a dual-stream parallel architecture is used to extract visual morphological features and semantic textual features separately, and each candidate target is independently ranked in its respective modality to generate a visual first ranking and a semantic first ranking. Finally, a reciprocal fusion function is used, with ranking rather than similarity values ​​as input variables, for multimodal fusion decision-making, fundamentally avoiding the constraint of dimensional differences on fusion effectiveness. This scheme achieves cross-validation and automatic bias correction of vision and semantics without requiring any parameter retraining for the target category. It effectively suppresses the interference of random noise from metallic reflective surfaces on retrieval accuracy and improves the accuracy and robustness of fine-grained image recognition under single-sample conditions.

[0021] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] Example 1 In this embodiment, a fine-grained image recognition method is provided.

[0024] Reference Figure 1 The fine-grained image recognition method in this embodiment includes the following steps: Step S1: Obtain the image of the target object to be identified; In this embodiment, the target object refers to a tiny, fine-grained object with a radial topological structure, whose surface contains unique identifiers. The radial topological structure refers to the key semantic information of the target object, such as stamped text or ring-shaped markings, arranged radially or in a ring shape along the circumference. Examples include stamped inscriptions on bullet bases, ring-shaped characters on the surface of precision industrial stamped parts, and ring-shaped inscriptions on ancient coins. The metallic surfaces of such target objects have bidirectional reflectance distribution function characteristics, which can produce strong specular highlights and shadows under random lighting conditions, interfering with subsequent visual feature extraction.

[0025] Optionally, the original image of the target object can be acquired using an industrial camera, microscope, or portable image acquisition device. Since lighting conditions, shooting angle, and device resolution can all affect the quality of the original image during the actual acquisition process, the acquired target object image needs to be processed through subsequent steps S2 to S5.

[0026] Optionally, the target object image may be preprocessed, including but not limited to one or more combinations of grayscale processing, size normalization processing, and histogram equalization processing, for subsequent processing.

[0027] Step S2: Perform polar coordinate transformation on the target object image to map the target area distributed along the circumference in the target object image into a horizontally flattened image; In this embodiment, through polar coordinate transformation, the pixels distributed along the circumference in the Cartesian coordinate system are unfolded into a horizontal sequence of pixels by angle, generating a horizontally flattened image. In the horizontally flattened image, the characters that were originally arranged in an arc are straightened into a horizontal arrangement, and the relative order and spatial relationship between the characters are completely preserved.

[0028] As an optional implementation, when mapping the target object image to a horizontally flattened image, the target object image is first subjected to Gaussian smoothing to suppress specular highlight noise on the metal surface; the geometric center coordinates of the target object image are then automatically calculated based on a connected component analysis algorithm. With the maximum effective outer diameter A nonlinear polar coordinate mapping matrix is ​​constructed, and the geometric center coordinates and the maximum effective outer diameter are used as mapping parameters to remap the pixels arranged along the circumference in the target object image to a horizontal sequence of pixels, generating the horizontally flattened image. The maximum effective outer diameter refers to the maximum Euclidean distance from the geometric center to the boundary pixel of the connected region, i.e., the maximum radial span of the target object in the image.

[0029] Specifically, for any pixel (x, y) in the target object image, with the geometric center... Using the origin of polar coordinates, the Cartesian coordinates of this pixel are mapped to polar coordinates. In this mapping relationship, the radial distance *r* is calculated from the horizontal and vertical offsets of the pixel relative to the geometric center, and the rotation angle... It is calculated by taking the arctangent function of the ratio of the longitudinal offset to the lateral offset. That is:

[0030]

[0031] Based on the above mapping relationship, all angles corresponding to each radial distance r will be... The pixels on the image are unfolded into a horizontal sequence of pixels in angular order, and then all radial distance layers are stitched together in radial order to generate a horizontally flattened image.

[0032] Through the above transformation, the pixels originally arranged in a circle in the Cartesian coordinate system are remapped into a sequence of pixels arranged in a horizontal direction. The characters originally distributed in an arc are straightened into a horizontal arrangement. The relative order and spatial position relationship between the characters are completely preserved, providing a structured input basis for subsequent visual feature extraction streams and semantic feature extraction streams.

[0033] Optionally, before the polar coordinate transformation, dynamic region cropping can be performed on the target object image. This involves detecting the effective region boundary of the target object in the image, cropping the target object image based on the effective region boundary to remove the background region, resulting in an effective region image containing only the target object. The cropped effective region image is then used as the input image for the polar coordinate transformation.

[0034] Step S3: Input the horizontally flattened image into the visual feature extraction stream to generate a visual feature vector; and input the horizontally flattened image into the semantic feature extraction stream to generate a semantic text feature vector; In this embodiment, a dual-stream parallel architecture is employed to extract heterogeneous features from the horizontally flattened image from both the visual shape dimension and the semantic text dimension. The two streams operate independently and in parallel, each outputting feature vectors of different granularities. The visual stream outputs dense numerical features, while the semantic stream outputs high-dimensional semantic text features, providing heterogeneous and complementary input for subsequent multimodal ranking fusion.

[0035] As an optional implementation, the horizontally flattened image is segmented into multiple image patches; the multiple image patches are input into a deep neural network with frozen backbone network parameters; the global spatial correlation features between the image patches are extracted through the deep neural network, and the extracted features are normalized to generate the visual feature vector.

[0036] For example, a horizontally flattened image is uniformly divided into several local image regions according to a preset patch size. Each image patch, after flattening, is mapped to a patch embedding vector through a linear projection layer. This embedding vector contains the pixel distribution information of the local region. The deep neural network adopts a network architecture based on a self-attention mechanism. The multiple image patch embedding vectors, along with category labels, are used as the input sequence. The input sequence undergoes layer-by-layer feature transformation through stacked multi-head self-attention layers and feedforward network layers. In each layer, the self-attention mechanism uses the current representation of each patch as the query term and the representations of all patches as key-value pairs. It calculates the attention weight between the patch and all other patches in the image, and performs weighted aggregation of the value representations of all patches based on the attention weight, updating the patch's representation. Through multi-layer stacking, the representation of each patch can aggregate information from all other patches, thereby achieving the extraction of global spatial correlation features.

[0037] After the above processing, the feature vector corresponding to the category label output by the deep neural network is extracted as a global representation of the entire horizontally flattened image. This feature vector has aggregated the global spatial information of all image patches in the flattened image. The extracted feature vector is then subjected to L2 normalization, that is, the feature vector is divided by its L2 norm so that the vector magnitude is 1, generating a visual feature vector.

[0038] Optionally, the deep neural network may employ a visual Transformer architecture based on a self-attention mechanism, whose global receptive field can effectively model the long-range dependencies between character regions in a flattened image, compensating for local pixel distortion that may be introduced by polar coordinate transformation.

[0039] As an optional implementation, a pre-trained multimodal reasoning model is used to perform context inference and recognition of character information in the horizontally flattened image to generate a plain text sequence; the plain text sequence is then processed into text vectorization, transforming it into a high-dimensional dense vector to generate the semantic text feature vector.

[0040] For example, the multimodal inference model adopts a Transformer-based encoder-decoder architecture. At the encoder end, the horizontally flattened image has its visual features extracted by an image coding network, and then the visual features are mapped into a series of visual marker vectors through a cross-attention mechanism. At the decoder end, an autoregressive approach is used to generate recognition results character by character, outputting a plain text sequence. Because the multimodal inference model possesses rich language rules and contextual patterns, it can infer and complete characters that are missing or blurred due to stamping wear or partial occlusion, thus outputting a complete and accurate plain text sequence even under conditions of visual feature degradation. After obtaining the plain text sequence, it undergoes text vectorization processing, mapping discrete strings into continuous high-dimensional dense vectors. Specifically, the plain text sequence is segmented into N-Gram segments to obtain multiple continuous character fragments, which are then hashed. The hash values ​​of all fragments are then added in phase or concatenated and projected in a reduced dimension to obtain a fixed-dimensional dense vector, which serves as the semantic text feature vector.

[0041] Step S4: Calculate the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, and determine the visual first ranking and semantic first ranking of each candidate target according to the similarity ranking. In this embodiment, the dual-stream feature vector of the target to be identified is compared with the corresponding features of each candidate target in the feature library. The feature library pre-stores the visual feature vector and semantic text feature vector for each type of candidate target, with only one template image corresponding to each type of candidate target stored. Since in practical applications, each type of candidate target objectively only provides a single template image, the feature library stores only one template image corresponding to each type of candidate target. Accordingly, this embodiment employs a metric learning paradigm, achieving identification by comparing and sorting the feature vector of the target to be identified with the template features in the feature library one by one.

[0042] Optionally, the feature library is pre-built by acquiring template images of multiple candidate targets in advance, providing at least one template image for each type of candidate target; performing the aforementioned polar coordinate transformation, visual feature extraction, and semantic feature extraction operations on each template image to generate a visual feature vector and a semantic text feature vector for each type of candidate target; and storing the two types of feature vectors of all candidate targets in the feature library, wherein the visual feature vector and the semantic text feature vector are stored in association according to the category identifier.

[0043] Optionally, the feature library adopts a key-value pair storage structure, where the key is the category identifier of the candidate target and the value is a structure containing visual feature vectors and semantic text feature vectors.

[0044] As an optional implementation, the cosine similarity between the visual feature vector and the visual feature vector of each candidate target in the feature library is calculated, and the candidates are sorted from high to low according to the cosine similarity to generate the visual first ranking corresponding to each candidate target.

[0045] Specifically, since the visual feature vectors have already undergone L2 normalization, the cosine similarity is calculated by the inner product of two normalized vectors. That is, the visual feature vector of the target to be identified is multiplied by the dimensions of the visual feature vectors of the candidate targets, and then the sum is obtained. The closer the value is to 1, the more similar the two vectors are. All candidate targets are sorted from high to low according to their cosine similarity to generate the visual first ranking of each candidate target under the visual feature flow.

[0046] As an optional implementation, the similarity between the semantic text feature vector and the semantic feature vector of each candidate target in the feature library is calculated, and the results are sorted according to the similarity to generate the semantic first ranking corresponding to each candidate target.

[0047] Specifically, cosine similarity or the reciprocal of Euclidean distance can be used as the metric. When using cosine similarity, its calculation method is the same as for visual modalities. When using the reciprocal of Euclidean distance, the smaller the distance, the higher the similarity. The similarity of each candidate target in its respective modality is sorted from high to low to generate the first ranking of each candidate target in the semantic feature stream.

[0048] Optionally, each candidate target corresponds to a visual ranking and a semantic ranking. The smaller the ranking value, the more similar the candidate target is to the target to be identified in the corresponding modality.

[0049] Step S5: Based on the visual first ranking and the semantic first ranking of each candidate target, calculate the final fusion score of each candidate target using the inverse fusion function, and output the recognition result according to the final fusion score.

[0050] In this embodiment, the two rankings obtained by each candidate target in the visual and semantic modal modes are fused. Both the visual and semantic rankings are dimensionless integers, with values ​​ranging from 1 to the total number of categories. A reciprocal fusion function maps the rankings to scores: candidate targets with higher rankings receive higher sub-scores; and the rate of score increase decreases as the ranking increases. By configuring weight coefficients for each modality and weighting and summing the sub-scores, the final fused score for each candidate target is obtained.

[0051] As an optional implementation, the visual first ranking and the semantic first ranking are used as the core variables of the denominator and substituted into the reciprocal fusion function to calculate the sub-scores of the candidate target in the visual modality and semantic modality; the sub-scores are then weighted and summed to obtain the final fusion score.

[0052] The expression for the reciprocal fusion function is as follows:

[0053] Where d represents any candidate target. The candidate target is ranked first visually in the visual feature flow. This is the first semantic ranking of the candidate target in the semantic feature stream. For the smoothing penalty constant, and These are the weight coefficients for the visual modality and the semantic modality, respectively. Each candidate target receives a sub-score in both the visual and semantic modalities. Each sub-score is equal to the weight coefficient of that modality divided by the sum of the ranking and smoothing penalty constant in that modality. The final fusion score of the candidate target is obtained by adding the two sub-scores together.

[0054] Optionally, the weighting coefficient and The initial setting is determined based on the modality confidence level. When the image quality is good, it can be set to... =0.5、 =0.5 achieves equal weight fusion; in other application scenarios, a higher initial weight can be assigned to a certain modality based on prior knowledge.

[0055] By using the aforementioned fusion mechanism and ranking as the input variable, the obstacle of not being able to directly weight visual similarity and semantic similarity due to their different scales is eliminated. The ranking itself has relative stability; even if the similarity value of a certain modality fluctuates drastically due to specular highlights or character wear, its relative ranking order usually remains stable, ensuring that the fusion result will not fluctuate drastically due to numerical jumps in a single modality.

[0056] As an optional implementation, the visual first ranking and the semantic first ranking are used as the core variables of the denominator. That is, the visual first ranking and the semantic first ranking are summed with a smoothing penalty constant, and the summation results are used as the core variables of the denominator in the corresponding modality sub-item score calculation formula.

[0057] Optionally, the smoothing penalty constant The preferred value range is [40, 80]. This range ensures that when the ranking is 1, the sub-score is [40, 80]. When the ranking is equal to the total number of categories N, the sub-score is: The gap between the two is limited to a reasonable range, avoiding exponential growth in scores when the ranking is extremely high, and ensuring the stability of the scores of each candidate target in the long-tail data distribution.

[0058] As an optional implementation, the step of weighted summation of the sub-scores includes: obtaining the image quality signal-to-noise ratio of the horizontally flattened image; setting weight coefficients for the visual modality and semantic modality based on the image quality signal-to-noise ratio, so that the modality with a higher signal-to-noise ratio receives a larger weight coefficient; and weighted summation of the sub-scores using the weight coefficients.

[0059] Specifically, when the signal-to-noise ratio is higher than a preset quality threshold, it indicates that the image is clear, specular highlights and shadow interference are small, and the reliability of visual features is high, so the corresponding value is increased. Value; when the signal-to-noise ratio is lower than the preset quality threshold, it indicates that the image is severely affected by highlights or shadows, and the reliability of visual features decreases, so the value should be reduced accordingly. The value of and increase The value of determines how semantic modalities dominate the final fusion score. The signal-to-noise ratio (SNR) can be calculated using the image local variance mean ratio or other image quality assessment methods known in the field.

[0060] As an optional implementation, the step of setting the weight coefficients of the visual modality and the semantic modality based on the image quality signal-to-noise ratio includes: obtaining the ambient light intensity of the target object image; if the ambient light intensity is lower than a preset light intensity threshold, then, based on the weight coefficients set by the signal-to-noise ratio, further increasing the weight coefficient corresponding to the semantic feature extraction stream and decreasing the weight coefficient corresponding to the visual feature extraction stream.

[0061] Specifically, when the light intensity is below a preset light intensity threshold, the overall image contrast decreases, and the discriminative power of visual features diminishes. However, the semantic feature extraction stream, relying on inference and completion of prior knowledge, is relatively less affected by light intensity. Therefore, based on the pre-set weighting of the signal-to-noise ratio, it is further tilted towards the semantic modality. After calculating the final fusion score for all candidate targets, the candidate target with the highest score is output as the recognition result. The output recognition result includes the category identifier of the candidate target and its corresponding final fusion score; and a list of the top K candidate targets can be output for manual review.

[0062] This embodiment achieves high-precision and robust recognition of fine-grained metallic targets under single-sample conditions without requiring any parameter retraining for the target category. Polar coordinate transformation flattens the circumferential topology into a horizontal sequence, eliminating the learning dependency of rotation invariance in subsequent feature extraction. A dual-stream parallel architecture extracts features from two heterogeneous dimensions: visual morphology and semantic text, ensuring that the two information streams are mutually independent and complementary. A reciprocal fusion function makes a comprehensive decision on the two signals in the ranking space, ensuring that the fusion result remains stable even when any single mode is affected by noise. The final output is a reliable and interpretable recognition result, suitable for general microscopic recognition scenarios such as ballistic evidence tracing, precision industrial inspection, and ancient coin identification.

[0063] Based on the same inventive concept, the present invention also provides a system corresponding to the method in Embodiment 1, as shown in Embodiment 2.

[0064] Example 2 Based on Embodiment 1, another embodiment of the present invention is proposed, with reference to... Figure 2 Fine-grained image recognition systems include: The polar coordinate transformation module is used to acquire the target object image to be identified and perform polar coordinate transformation on the target object image to map the target area distributed along the circumference in the target object image into a horizontally flattened image. Optionally, the polar coordinate transformation module includes a Gaussian smoothing unit, a geometric parameter calculation unit, and a mapping unit, which are respectively used to perform Gaussian smoothing processing, calculation of geometric center and maximum effective outer diameter, construction of nonlinear polar coordinate mapping matrix, and pixel remapping operation.

[0065] Optionally, the polar coordinate transformation module further includes a dynamic cropping unit, used to perform effective region cropping on the target object image before polar coordinate transformation to remove background interference.

[0066] The visual feature extraction stream module, connected to the polar coordinate transformation module, is used to input the horizontally flattened image into the visual feature extraction stream to generate a visual feature vector; Optionally, the visual feature extraction module includes an image segmentation unit and a backbone network unit. The image segmentation unit is used to uniformly segment the horizontally flattened image into multiple image patches. The backbone network unit is a deep neural network with frozen backbone network parameters, used to extract global spatial correlation features between image patches. After L2 normalization of the extracted features by the normalization unit, a visual feature vector is output.

[0067] The semantic feature extraction stream module, connected to the polar coordinate transformation module, is used to input the horizontally flattened image into the semantic feature extraction stream to generate a semantic text feature vector; Optionally, the semantic feature extraction module includes a multimodal inference unit and a text vectorization unit. The multimodal inference unit uses a pre-trained multimodal inference model to perform context inference and recognition of character information in the horizontally flattened image to generate a plain text sequence. The text vectorization unit performs text vectorization processing such as N-Gram hashing on the plain text sequence, transforms it into a high-dimensional dense vector, and outputs a semantic text feature vector.

[0068] The heterogeneous metric fusion module is connected to the visual feature extraction stream module and the semantic feature extraction stream module, respectively. It is used to calculate the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, respectively. Based on the similarity ranking, it determines the visual first ranking and semantic first ranking of each candidate target, and calculates the final fusion score of each candidate target using the reciprocal fusion function based on the visual first ranking and the semantic first ranking of each candidate target, and outputs the recognition result based on the final fusion score.

[0069] Optionally, the heterogeneous metric fusion module includes a similarity calculation unit, a ranking generation unit, a reciprocal fusion calculation unit, and a weight adjustment unit, which are respectively used to perform similarity calculation, ranking generation, reciprocal fusion calculation, and dynamic adjustment of modal weight coefficients according to signal-to-noise ratio and illumination intensity.

[0070] Optionally, the system further includes a feature library module connected to the heterogeneous metric fusion module, used to store visual feature vectors and semantic text feature vectors of multiple candidate targets, with each type of candidate target storing only the feature vector corresponding to a template image.

[0071] The specific implementation details of each functional module in this embodiment have been described in detail in Embodiment 1, and will not be repeated here. The fine-grained image recognition system also achieves high-precision and robust recognition of fine-grained metal targets under single-sample conditions without requiring any parameter retraining for the target category. It is suitable for general microscopic recognition scenarios such as ballistic evidence tracing, precision industrial inspection, and ancient coin identification.

[0072] Since the system described in Embodiment 2 of this invention is a system used to implement the method of Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this system based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All systems used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0073] Example 3 In this embodiment of the invention, a fine-grained image recognition device is proposed.

[0074] Reference Figure 3 , Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present invention.

[0075] like Figure 3 As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed RAM or stable non-volatile memory, such as disk storage. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0076] Those skilled in the art will understand that Figure 3 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0077] like Figure 3 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and a fine-grained image recognition program.

[0078] exist Figure 3 In the hardware structure of the fine-grained image recognition device shown, the processor 1001 can call the fine-grained image recognition program stored in the memory 1004 and perform the following operations: Acquire an image of the target object to be identified; The target object image is subjected to polar coordinate transformation, and the target region distributed along the circumference in the target object image is mapped to a horizontally flattened image. The horizontally flattened image is input into a visual feature extraction stream to generate a visual feature vector; and the horizontally flattened image is input into a semantic feature extraction stream to generate a semantic text feature vector. The similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library is calculated respectively, and the visual first ranking and semantic first ranking of each candidate target are determined according to the similarity ranking. Based on the visual first ranking and the semantic first ranking of each candidate target, the final fusion score of each candidate target is calculated using the inverse fusion function, and the recognition result is output according to the final fusion score.

[0079] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: The image of the target object is subjected to Gaussian smoothing to suppress specular highlight noise on the metal surface; The geometric center coordinates and maximum effective outer diameter of the target object image are automatically calculated based on the connected component analysis algorithm. A nonlinear polar coordinate mapping matrix is ​​constructed, and the geometric center coordinates and the maximum effective outer diameter are used as mapping parameters to remap the pixels arranged along the circumference in the target object image to a sequence of pixels in the horizontal direction, thereby generating the horizontally flattened image.

[0080] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: The horizontally flattened image is divided into multiple image patches; The multiple image patches are input into a deep neural network whose backbone network parameters are frozen; The deep neural network extracts global spatial correlation features between the image patches, and normalizes the extracted features to generate the visual feature vector.

[0081] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: A pre-trained multimodal reasoning model is used to perform context inference and recognition of character information in the horizontally flattened image to generate a plain text sequence; The plain text sequence is processed into text vectorization, which transforms the plain text sequence into a high-dimensional dense vector, thereby generating the semantic text feature vector.

[0082] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: Calculate the cosine similarity between the visual feature vector and the visual feature vector of each candidate target in the feature library, and sort them from high to low according to the cosine similarity to generate the visual first ranking corresponding to each candidate target; Calculate the similarity between the semantic text feature vector and the semantic feature vector of each candidate target in the feature library, and sort them according to the similarity results to generate the semantic first ranking corresponding to each candidate target.

[0083] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: The visual first ranking and semantic first ranking are used as the core variables of the denominator and substituted into the reciprocal fusion function to calculate the sub-scores of the candidate target in the visual modality and semantic modality. The final fusion score is obtained by weighted summation of the individual scores.

[0084] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: The visual first ranking and the semantic first ranking are summed with a smoothing penalty constant, and the summation results are used as the core variables of the denominator in the corresponding modality sub-item score calculation formula.

[0085] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: Obtain the image quality signal-to-noise ratio of the horizontally flattened image; The weighting coefficients for visual and semantic modalities are set based on the image quality signal-to-noise ratio, so that the modality with a higher signal-to-noise ratio receives a larger weighting coefficient. The scores of the sub-items are weighted and summed using the weighting coefficients.

[0086] Optionally, the processor 1001 may call a fine-grained image recognition program stored in the memory 1004 and further perform the following operations: Obtain the ambient light intensity of the target object image acquisition environment; If the ambient light intensity is lower than a preset light intensity threshold, then based on the weight coefficient set for the signal-to-noise ratio, the weight coefficient corresponding to the semantic feature extraction stream is further increased, and the weight coefficient corresponding to the visual feature extraction stream is decreased.

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0092] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0093] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A fine-grained image recognition method, characterized in that, The method includes: Acquire an image of the target object to be identified; The target object image is subjected to polar coordinate transformation, and the target region distributed along the circumference in the target object image is mapped to a horizontally flattened image. The horizontally flattened image is input into a visual feature extraction stream to generate a visual feature vector; and the horizontally flattened image is input into a semantic feature extraction stream to generate a semantic text feature vector. The similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library is calculated respectively, and the visual first ranking and semantic first ranking of each candidate target are determined according to the similarity ranking. Based on the visual first ranking and the semantic first ranking of each candidate target, the final fusion score of each candidate target is calculated using the inverse fusion function, and the recognition result is output according to the final fusion score.

2. The method as described in claim 1, characterized in that, The step of performing polar coordinate transformation on the target object image to map the target region distributed along the circumference in the target object image into a horizontally flattened image includes: The image of the target object is subjected to Gaussian smoothing to suppress specular highlight noise on the metal surface; The geometric center coordinates and maximum effective outer diameter of the target object image are automatically calculated based on the connected component analysis algorithm. A nonlinear polar coordinate mapping matrix is ​​constructed, and the geometric center coordinates and the maximum effective outer diameter are used as mapping parameters to remap the pixels arranged along the circumference in the target object image to a sequence of pixels in the horizontal direction, thereby generating the horizontally flattened image.

3. The method as described in claim 1, characterized in that, The step of inputting the horizontally flattened image into a visual feature extraction stream to generate a visual feature vector includes: The horizontally flattened image is divided into multiple image patches; The multiple image patches are input into a deep neural network whose backbone network parameters are frozen; The deep neural network extracts global spatial correlation features between the image patches, and normalizes the extracted features to generate the visual feature vector.

4. The method as described in claim 1, characterized in that, The step of inputting the horizontally flattened image into a semantic feature extraction stream to generate a semantic text feature vector includes: A pre-trained multimodal reasoning model is used to perform context inference and recognition of character information in the horizontally flattened image to generate a plain text sequence; The plain text sequence is processed into text vectorization, which transforms the plain text sequence into a high-dimensional dense vector, thereby generating the semantic text feature vector.

5. The method as described in claim 1, characterized in that, The steps of calculating the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, and determining the visual first ranking and semantic first ranking of each candidate target based on the similarity ranking, include: Calculate the cosine similarity between the visual feature vector and the visual feature vector of each candidate target in the feature library, and sort them from high to low according to the cosine similarity to generate the visual first ranking corresponding to each candidate target; Calculate the similarity between the semantic text feature vector and the semantic feature vector of each candidate target in the feature library, and sort them according to the similarity results to generate the semantic first ranking corresponding to each candidate target.

6. The method as described in claim 1, characterized in that, The calculation of the final fusion score for each candidate target using the reciprocal fusion function includes: The visual first ranking and semantic first ranking are used as the core variables of the denominator and substituted into the reciprocal fusion function to calculate the sub-scores of the candidate target in the visual modality and semantic modality. The final fusion score is obtained by weighted summation of the individual scores.

7. The method as described in claim 6, characterized in that, The step of using the visual first ranking and the semantic first ranking as core variables in the denominator includes: The visual first ranking and the semantic first ranking are summed with a smoothing penalty constant, and the summation results are used as the core variables of the denominator in the corresponding modality sub-item score calculation formula.

8. The method as described in claim 6, characterized in that, The step of weighted summing of the individual scores includes: Obtain the image quality signal-to-noise ratio of the horizontally flattened image; The weighting coefficients for visual and semantic modalities are set based on the image quality signal-to-noise ratio, so that the modality with a higher signal-to-noise ratio receives a larger weighting coefficient. The scores of the sub-items are weighted and summed using the weighting coefficients.

9. The method as described in claim 8, characterized in that, The step of setting the weight coefficients for the visual modality and semantic modality based on the image quality signal-to-noise ratio includes: Obtain the ambient light intensity of the target object image acquisition environment; If the ambient light intensity is lower than a preset light intensity threshold, then based on the weight coefficient set for the signal-to-noise ratio, the weight coefficient corresponding to the semantic feature extraction stream is further increased, and the weight coefficient corresponding to the visual feature extraction stream is decreased.

10. A fine-grained image recognition system, characterized in that, The system includes: The polar coordinate transformation module is used to acquire the target object image to be identified and perform polar coordinate transformation on the target object image to map the target area distributed along the circumference in the target object image into a horizontally flattened image. The visual feature extraction stream module is used to input the horizontally flattened image into the visual feature extraction stream to generate a visual feature vector. The semantic feature extraction stream module is used to input the horizontally flattened image into the semantic feature extraction stream to generate a semantic text feature vector. The heterogeneous metric fusion module is used to calculate the similarity between the visual feature vector and the semantic text feature vector and each candidate target in the feature library, respectively. Based on the similarity ranking, the first visual ranking and the first semantic ranking of each candidate target are determined. Based on the first visual ranking and the first semantic ranking of each candidate target, the final fusion score of each candidate target is calculated using the reciprocal fusion function, and the recognition result is output based on the final fusion score.

Images