Image recognition method and device, electronic equipment and storage medium

Abstract

The invention is suitable for the technical field of image recognition, and provides an image recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized image; inputting the to-be-recognized image into a trained open set recognition model to obtain a coarseness recognition result output by the open set recognition model, the open set recognition model being used for recognizing a coarseness category of a target in the to-be-recognized image; and under the condition that the coarse granularity category of the target in the coarse granularity recognition result belongs to a set target category, inputting the to-be-recognized image into a trained target recognition model to obtain a fine granularity recognition result output by the target recognition model, the target recognition model is used for recognizing the fine-grained category of a target in the to-be-recognized image; and determining a target recognition result of the to-be-recognized image based on the fine-grained recognition result. The method can reduce the limitation of an open set recognition model, and improves the recognition precision of a fine-grained category target.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, and particularly relates to an image recognition method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Image recognition technology refers to a technology of processing, analyzing, and understanding images by using a computer to recognize various different modes of targets and objects. Image recognition technology recognizes the category to which a target in an image belongs by analyzing the features of the image.

[0003] Currently, an open set recognition model is usually used for image recognition to accurately recognize unknown categories of targets that are not seen in the model training process in actual applications. However, the open set recognition model is not sensitive enough to fine-grained features, so it is difficult to achieve high-precision detection when fine-grained category detection is required, and even cannot perform fine-grained category detection. SUMMARY

[0004] Embodiments of the present application provide an image recognition method, device, electronic device, and storage medium, which can reduce the limitations of the open set recognition model and improve the recognition accuracy of fine-grained category targets.

[0005] In a first aspect, an embodiment of the present application provides an image recognition method, comprising:

[0006] obtaining a to-be-recognized image;

[0007] inputting the to-be-recognized image into a trained open set recognition model to obtain a coarse-grained recognition result output by the open set recognition model, the open set recognition model being configured to recognize a coarse-grained category of a target in the to-be-recognized image;

[0008] in a case where the coarse-grained category of the target in the coarse-grained recognition result belongs to a set target category, inputting the to-be-recognized image into a trained target recognition model to obtain a fine-grained recognition result output by the target recognition model, the target recognition model being configured to recognize a fine-grained category of the target in the to-be-recognized image;

[0009] determining a target recognition result of the to-be-recognized image based on the fine-grained recognition result.

[0010] In a possible implementation manner of the first aspect, the inputting the to-be-recognized image into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model comprises:

[0011] extracting features of the to-be-recognized image by using the target recognition model, comparing the extracted to-be-recognized features with pre-stored fine-grained category features, and determining the fine-grained recognition result according to a comparison result.

[0012] In a possible implementation manner of the first aspect, before the inputting the to-be-identified image into the trained target identification model to obtain the fine-grained identification result output by the target identification model, the method further includes:

[0013] For any fine-grained category, the fine-grained category feature corresponding to the fine-grained category is obtained by performing weighted calculation on the respective image features corresponding to the fine-grained category, and is stored.

[0014] In a possible implementation manner of the first aspect, before the inputting the to-be-identified image into the trained target identification model to obtain the fine-grained identification result output by the target identification model, in the case that the coarse-grained category of the target in the coarse-grained identification result belongs to the set target category, the method further includes:

[0015] The target category is determined based on an identification requirement corresponding to the to-be-identified image, and the identification requirement is used to reflect an identification requirement of an application scenario corresponding to the to-be-identified image for the fine-grained category.

[0016] In a possible implementation manner of the first aspect, the determining the target identification result of the to-be-identified image based on the fine-grained identification result includes:

[0017] In the case that the confidence of the fine-grained identification result is greater than or equal to a set confidence threshold, the target identification result is determined based on the fine-grained identification result.

[0018] In a possible implementation manner of the first aspect, the open-set identification model is further used to identify a medium-grained category of the target in the to-be-identified image, and after the inputting the to-be-identified image into the trained open-set identification model to obtain the coarse-grained identification result output by the open-set identification model, the method further includes:

[0019] In the case that the coarse-grained category of the target in the coarse-grained identification result does not belong to the target category, or in the case that the coarse-grained category of the target in the coarse-grained identification result belongs to the target category but the confidence is less than the confidence threshold, the target identification result is determined based on the medium-grained category of the target in the coarse-grained identification result.

[0020] In a possible implementation manner of the first aspect, before the inputting the to-be-identified image into the trained open-set identification model to obtain the coarse-grained identification result output by the open-set identification model, the method further includes:

[0021] The fine-tuning sample is constructed, and one fine-tuning sample includes one training image and a coarse-grained category of a target in the training image.

[0022] Fine-tune a multi-modal large language model based on the fine-tuning samples, and determine a trained open-set recognition model based on the multi-modal large language model after fine-tuning.

[0023] In a second aspect, an image recognition apparatus is provided, and the apparatus comprises:

[0024] An image acquisition module is configured to acquire a to-be-recognized image.

[0025] A coarse-grained recognition module is configured to input the to-be-recognized image into a trained open-set recognition model to obtain a coarse-grained recognition result output by the open-set recognition model, the open-set recognition model being configured to recognize a coarse-grained category of a target in the to-be-recognized image.

[0026] A fine-grained recognition module is configured to input the to-be-recognized image into a trained target recognition model in a case where the coarse-grained category of the target in the coarse-grained recognition result belongs to a set target category, to obtain a fine-grained recognition result output by the target recognition model, the target recognition model being configured to recognize a fine-grained category of the target in the to-be-recognized image.

[0027] A target recognition result determination module is configured to determine a target recognition result of the to-be-recognized image based on the fine-grained recognition result.

[0028] In a third aspect, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the image recognition method of the first aspect when executing the computer program.

[0029] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the image recognition method of the first aspect when executed by a processor.

[0030] In a fifth aspect, a computer program product is provided, which, when executed on an electronic device, causes the electronic device to perform the image recognition method of the first aspect.

[0031] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0032] In the embodiments of the present application, since the open set recognition model can better process seen and unseen classes in the training process, the trained open set recognition model can quickly and comprehensively identify the target in the to-be-recognized image in a coarse-grained class, and can obtain a coarse-grained recognition result with high accuracy. Meanwhile, when the coarse-grained recognition result indicates that the coarse-grained class of the target in the to-be-recognized image belongs to the set target class, the target recognition model is used to identify the target in the to-be-recognized image in a fine-grained class, so that more accurate recognition results can be provided when needed, the target recognition accuracy is improved, and the probability of misrecognition is further reduced. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0034] Figure 1 is a flowchart of an image recognition method provided by an embodiment of the present application;

[0035] Figure 2 is a flowchart of another image recognition method provided by an embodiment of the present application;

[0036] Figure 3 is a flowchart of another image recognition method provided by an embodiment of the present application;

[0037] Figure 4 is a structural schematic diagram of an image recognition device provided by an embodiment of the present application;

[0038] Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0040] It should be understood that when used in the present application and the appended claims, the term "comprising" indicates the presence of the described features, whole, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, whole, steps, operations, elements, components and / or sets thereof.

[0041] It should also be understood that the term "and / or" as used herein refers to any one of the associated listed items, or a combination of any of the associated listed items, and all possible combinations, and includes these combinations.

[0042] Reference throughout this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, however, but can refer to one or more but not all embodiments.

[0043] Embodiment One:

[0044] Figure 1 A flowchart of an image recognition method provided by an embodiment of the application is shown. It should be understood that the image recognition method provided by the embodiment of the application can be applied to devices such as cameras, access control machines, or drones, and can also be applied to systems or platforms such as monitoring systems, autonomous driving systems, or industrial product detection systems, and can be set according to actual application requirements. The above method is described in detail as follows:

[0045] S101, obtaining an image to be recognized.

[0046] It should be understood that the acquisition method of the image to be recognized can be set according to actual application requirements, and the embodiment of the application does not make specific limitations.

[0047] As an example, when the image recognition method is applied to a camera, the image to be recognized can be obtained by the built-in camera of the camera, or can be obtained by a terminal device (such as a mobile phone or a tablet computer) in communication connection with the camera, or can be obtained from a cloud album.

[0048] As another example, when the image recognition method is applied to a traffic system, the image to be recognized can be obtained by a monitoring camera, or can be obtained from a video stream recorded by the monitoring camera.

[0049] S102, inputting the above image to be recognized into a trained open set recognition model to obtain a coarse-grained recognition result output by the open set recognition model, the open set recognition model being used to recognize a coarse-grained category of a target in the image to be recognized.

[0050] It should be understood that the target in the image to be identified generally refers to the object present in the image, and the object has identifiable features, including but not limited to animals, plants, vehicles, or characters, etc. In some embodiments, the target to be identified can be set according to the actual application scene or demand. For example, in an animal detection scene, the target in the image to be identified can refer to an animal.

[0051] The open set identification model described above generally refers to a model that can process unknown class targets that have not been seen in the training stage, that is, the open set identification model can identify targets of coarse-grained classes that have been seen in the training stage and targets of coarse-grained classes that have not been seen in the training stage when performing coarse-grained class identification on the target in the image to be identified, so as to improve the robustness and generalization ability of the open set identification model in an open and dynamic application scene.

[0052] It should be understood that the trained open set identification model can be a general pre-trained open set identification model, or an open set identification model trained by a cloud or locally using specific training data, and the training method of the open set identification model is not specifically limited in the embodiments of the present application.

[0053] It should be understood that the coarse-grained class of the target generally refers to a relatively broad classification of the target in the image, and the classification has a large granularity, mainly focusing on the overall features of the target, and dividing it into a larger class range. The specific granularity of the coarse-grained class classification can be set according to the actual application demand, which is not specifically limited here.

[0054] As an example, when the image recognition method is applied to a general recognition scene (such as a camera), the coarse-grained class of the target can include but is not limited to: animal, plant, vehicle, food, building, or electronic product, etc.

[0055] As another example, when the image recognition method is applied to an animal recognition scene (such as an animal detection system), the coarse-grained class of the target can include but is not limited to: mammal, bird, reptile, fish, amphibian, or insect, etc.

[0056] In some embodiments, the trained open-set recognition model can be a general multi-modal large language model (MLLM), such as GPT-4 with Vision, CogAgent: A Visual Language Model for GUI Agents, or Qwen2-VL-7B-Instruct, or a multi-modal large language model such as a Bao, or the trained multi-modal large language model can be a model fine-tuned from a general multi-modal large language model according to samples corresponding to the recognition task of the coarse-grained category (such as samples including an input image to be recognized and a coarse-grained category label).

[0057] The above-mentioned multi-modal large language model generally refers to the combination of a multi-modal large model and a large language model. Since the multi-modal large language model is based on a large language model and has the ability to process multiple modal data, using the multi-modal large language model as an open-set recognition model can focus on the fusion of language and image modalities, so that the multi-modal large language model can better understand and generate multi-modal content related to language in the image when recognizing the coarse granularity of the target in the input image according to the prompt word, recognize seen and unseen targets, improve the recognition accuracy and reliability of the coarse-grained category of the target in the image to be recognized, and improve the robustness and applicability of the open-set recognition model.

[0058] In other embodiments, to adapt to the computing power or memory configuration or actual application requirements of the device, the open-set recognition model can also be a generative model or a discriminative model constructed based on a multi-modal large model (such as Large Language and Vision Assistant, Pixel Reasoning with Large Multimodal Model, or Least-squares Interpolation Optimizer with Noise) or other network structures (such as a convolutional neural network, a generative adversarial network, or a Vision Transformer). When coarse-grained category recognition is required, the image to be recognized and the corresponding prompt word can be input into the generative open-set recognition model,

[0059] In some embodiments, when the open set recognition model needs to be trained, images of known coarse-grained categories and images of unknown coarse-grained categories can be collected based on the application scenario, and after the corresponding samples are constructed, the sample set can be divided into a known category training set and an unknown category test set. The known category training set can be used for regular training of the open set recognition model, and the unknown category test set can be used to test the recognition ability of the open set recognition model for unknown categories. Among them, images similar in vision to the sample images in the known category sample set can also be mined based on public data sets to construct an unknown category training set for training the open set recognition model to recognize unknown category targets, thereby helping the open set recognition model to distinguish between known categories and unknown categories and more robustly handle unknown category targets in actual applications.

[0060] It should be understood that when training the open set recognition model based on the known category training set and the unknown category training set, the training samples in the training set can generally be input into the open set recognition model; the open set recognition model extracts features of the training samples and recognizes the coarse-grained categories of the targets in the training samples based on the extracted features, updates the model parameters of the open set recognition model according to the recognition results and the corresponding output labels, and obtains an updated open set recognition model; the performance of the current open set recognition model is evaluated using the unknown category test set, and the parameters of the open set recognition model are adjusted in combination with the evaluation results to improve the performance of the open set recognition model, and a trained open set recognition model is obtained.

[0061] In the embodiments of the present application, since the open set recognition model can recognize targets of categories seen and not seen in the training phase, i.e., the open set recognition model can recognize coarse-grained categories of a large number of targets, the open set recognition model with wide generality is used to recognize the coarse-grained categories of the targets in the to-be-recognized image first, which can quickly and accurately recognize various targets in the to-be-recognized image, improve the adaptability of image target recognition, and reduce the probability of misrecognition.

[0062] S103, in the case where the coarse-grained category of the target in the above coarse-grained recognition result belongs to a set target category, inputting the to-be-recognized image into the trained target recognition model to obtain a fine-grained recognition result output by the target recognition model, the target recognition model being used to recognize a fine-grained category of the target in the to-be-recognized image.

[0063] It should be understood that the target category can be determined according to user settings or inputs, or can be determined by intelligent algorithms such as large models, and can be set according to actual application requirements.

[0064] It should be understood that the fine-grained category of the target generally refers to a more accurate division of a sub-category under a certain coarse-grained category, and the classification has a smaller granularity and pays more attention to the local features of the target. The specific granularity of the fine-grained category classification can be set according to the granularity of the coarse-grained category and the actual application requirements, and is not specifically limited here.

[0065] For example, assuming that the coarse-grained category of the target can include birds and fish, the fine-grained category can include but is not limited to specific sub-categories of birds such as peacocks, parrots, and thrushes, and specific sub-categories of fish such as carp, eels, and sea bass.

[0066] It should be understood that the target recognition model generally refers to a model based on a data set of fine-grained category targets, which can identify the fine-grained category of the target in the image to be recognized.

[0067] For example, a data set of fine-grained category targets can be used to construct training samples to supervise the learning of the target recognition model, so that it learns the differences between different fine-grained categories, so that the trained target recognition model can map the target in the input image to the correct fine-grained category, and achieve the recognition of the fine-grained category of the target.

[0068] In some embodiments, the target recognition model can be a CLIP model (Contrastive Language-Image Pre-training). CLIP is a multi-modal model trained based on a contrastive learning method. Since the model can learn the joint representation of text and image in contrastive learning, similar text and images are closer in the embedding space, so that the image content can be better understood, and it can quickly adapt to new categories and tasks without samples, has strong generalization ability and multi-modal understanding ability. Therefore, using a target recognition model based on contrastive learning can achieve more accurate and robust fine-grained category recognition, while significantly reducing the cost of model training and improving the efficiency of deployment.

[0069] Optionally, the CLIP model (i.e., the target recognition model) can generally include an image encoder (such as a ViT encoder) and a text encoder (such as a BERT encoder, Bidirectional Encoder Representations from Transformers). When training the target recognition model, the initial target recognition model can be trained based on a contrastive learning loss using image-text pairs, where an image-text pair can include an image and a corresponding text description. After extracting the features of the image in the image-text pair by the image encoder to obtain image features and extracting the features of the text in the image-text pair by the text encoder to obtain text features, for each image-text pair, the image features and the corresponding text features can be taken as a positive sample pair, and then negative sample pairs can be constructed based on other images or texts. For example, for the image in the same image-text pair, a negative sample pair can be constructed by selecting an unrelated text. By constructing negative sample pairs, the target recognition model can learn to distinguish the irrelevance between images and texts, thereby enhancing the discriminative ability of the target recognition model.

[0070] During the training process, the target recognition model can calculate the similarity (such as dot product similarity) between the positive sample pair and the negative sample pair, measure the difference between the positive sample pair and the negative sample pair through a set contrastive learning loss function, and optimize the model parameters of the target recognition model through a set optimization algorithm (such as a gradient descent algorithm), so that the features of the matching image-text pair are closer, and the features of the non-matching image-text pair are farther apart.

[0071] In some embodiments, to reduce the storage resources occupied by the target recognition model and reduce the requirements on the device, after training, the text encoder in the model can be discarded, the image encoder can be retained, and the target recognition model containing the image encoder can be deployed, thereby reducing unnecessary network structures, reducing the data amount of the target recognition model, and reducing the requirements on the storage capacity of the device.

[0072] In some other embodiments, the target recognition model can also be a model constructed based on a recurrent neural network (RNN), a convolutional neural network, a residual neural network (ResNet), a Transformer, or an autoencoder, and the like. The specific network structure can be set according to actual application requirements.

[0073] In the embodiments of the present application, when the coarse-grained category of the target in the to-be-recognized image belongs to the set target category, the target recognition model can be used to recognize the fine-grained category of the target in the to-be-recognized image, so as to improve the recognition accuracy of the target and further reduce the probability of misrecognition.

[0074] S104, determining a target recognition result of the to-be-recognized image based on the above fine-grained recognition result.

[0075] It should be understood that in the case of fine-grained category recognition of the to-be-recognized image, the final target recognition result of the to-be-recognized image is usually determined based on the obtained fine-grained recognition result.

[0076] In some embodiments, the target recognition result can be determined based on the coarse-grained recognition result and the fine-grained recognition result, so that the user can fully understand the coarse-grained category and fine-grained category of the target in the to-be-recognized image based on the target recognition result, improve the comprehensiveness of image recognition, and improve the user experience.

[0077] In other embodiments, in the case where the to-be-recognized image is not subjected to fine-grained recognition, the target recognition result can be determined according to the coarse-grained recognition result, or the to-be-recognized image can be returned to the step of obtaining the coarse-grained recognition result output by the open set recognition model, which can be set according to actual application requirements.

[0078] In some embodiments, after determining the target recognition result of the to-be-recognized image, the to-be-recognized image and its corresponding target recognition result can be displayed on a set interactive interface.

[0079] In the embodiments of the present application, since the open set recognition model can better handle seen and unseen categories in the training process, the trained open set recognition model can quickly and comprehensively recognize the coarse-grained category of the target in the to-be-recognized image, and can obtain a coarse-grained recognition result with high accuracy. At the same time, when the coarse-grained recognition result indicates that the coarse-grained category of the target in the to-be-recognized image belongs to the set target category, the target recognition model is used to recognize the fine-grained category of the target in the to-be-recognized image, so that more accurate recognition results can be provided when needed, the recognition accuracy of the target is improved, and the probability of misrecognition is further reduced.

[0080] In some embodiments, the above inputting the to-be-recognized image into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model comprises:

[0081] The above target recognition model is used to extract features of the to-be-recognized image, and the extracted to-be-recognized features are compared with pre-stored fine-grained category features, and the fine-grained recognition result is determined according to the comparison result.

[0082] It should be understood that the pre-stored fine-grained category features are usually extracted based on images of targets corresponding to fine-grained categories.

[0083] In some embodiments, the image of the target corresponding to the fine-grained category can be subjected to feature extraction by the target recognition model to determine the fine-grained category feature based on the extracted image feature, i.e., the fine-grained category feature and the to-be-recognized feature are features extracted by the same model, ensuring the consistency of the feature representation, thereby improving the accuracy of subsequent similarity measurement.

[0084] To improve the recognition efficiency, in the embodiments of the present application, the fine-grained category features corresponding to each fine-grained category can be pre-stored. After inputting the to-be-recognized image into the target recognition model, the target recognition model can extract features from the to-be-recognized image to obtain the to-be-recognized features, and compare the extracted to-be-recognized features with the stored fine-grained category features.

[0085] It should be understood that when comparing the to-be-recognized features with the stored fine-grained category features, the similarity between the to-be-recognized features and the fine-grained category features can be calculated based on cosine similarity, Euclidean distance or Hamming distance, and the comparison result can include the similarity between the to-be-recognized features and each fine-grained category feature.

[0086] It should be understood that when determining the fine-grained recognition result according to the comparison result, the fine-grained recognition result can be determined based on the fine-grained category feature with the largest similarity, or the fine-grained recognition result can be determined based on the fine-grained category feature with a similarity greater than or equal to a similarity threshold (such as 0.8), which can be set according to actual application requirements.

[0087] In some embodiments, the image encoder in the target recognition model can be used to extract features from the images corresponding to each fine-grained category in the database to obtain the fine-grained category features corresponding to each fine-grained category and store them.

[0088] In the embodiments of the present application, the target recognition model performs fine-grained category recognition on the target in the to-be-recognized image based on the pre-stored fine-grained category features, i.e., it is equivalent to retrieving the similar features to the to-be-recognized features from the pre-stored fine-grained category features by retrieval, thereby determining the fine-grained recognition result, without the need for complex analysis and mapping of the to-be-recognized features, effectively improving the efficiency of fine-grained recognition.

[0089] In some embodiments, before inputting the to-be-recognized image into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model, the following steps are further included:

[0090] For any fine-grained category, the fine-grained category features corresponding to the fine-grained category can be obtained by weighted calculation of the image features corresponding to the fine-grained category, and stored.

[0091] In the embodiments of the present application, one target of a fine-grained category can correspond to multiple images, that is, multiple images containing the same target of the fine-grained category can be obtained, and the feature of each image is extracted to obtain multiple image features corresponding to the fine-grained category. Alternatively, the feature of each image corresponding to the fine-grained category can be extracted by the target recognition model to obtain the corresponding image features, so as to improve the consistency between the image features.

[0092] In the embodiments of the present application, for any fine-grained category, the weighted calculation can be performed according to the image features corresponding to the fine-grained category, and a more representative fine-grained category feature of the fine-grained category can be obtained by weighted fusion, so as to reduce the influence of the feature changes caused by different shooting conditions and the like, and improve the accuracy of fine-grained category recognition.

[0093] It should be understood that when the weighted calculation is performed according to the image features corresponding to the fine-grained category, the image features can be assigned the weights corresponding to the image features, and the weighted calculation is performed according to the image features and the weights corresponding to the image features; or the image features can be spliced to obtain spliced image features, and then the spliced image features are processed by dimension reduction to obtain the fine-grained category feature. The implementation manner of the weighted calculation according to the image features corresponding to the fine-grained category is not limited in the embodiments of the present application.

[0094] In some embodiments, when the image features are assigned the weights corresponding to the image features, an equal weight can be assigned to each image feature, that is, the average value of the image features is calculated and taken as the fine-grained category feature; or the weight corresponding to the image feature can be determined according to one or more information such as the quality of the image corresponding to the image feature, the acquisition manner, and the shooting condition.

[0095] In the embodiments of the present application, for any fine-grained category, the weighted calculation is performed according to the image features corresponding to the fine-grained category, and a more representative feature is obtained by weighted fusion as the fine-grained category feature corresponding to the fine-grained category, so as to reduce the influence of the feature differences between different images caused by different shooting conditions and the like, and improve the accuracy of subsequent fine-grained category recognition.

[0096] Figure 2 A flowchart of an image recognition method provided by some embodiments is shown. Referring to FIG. 1, the image recognition method includes the following steps. Figure 2 Before the step S103, the method further includes:

[0097] The target category is determined based on a recognition requirement corresponding to the to-be-recognized image, where the recognition requirement reflects a recognition requirement of a fine-grained category corresponding to an application scenario of the to-be-recognized image.

[0098] It should be understood that the recognition requirement corresponding to the to-be-recognized image can be determined in advance or dynamically determined based on the to-be-recognized image, and can be set according to an actual application scenario.

[0099] As an example, when the image recognition method is applied to a learning camera, the target category can be pre-set to include animals and plants, so that the learning camera can generate more accurate knowledge popularization content based on the fine-grained recognition result when generating knowledge popularization based on the image shot by the user, and improve the learning effect of the user using the learning camera for enlightenment learning.

[0100] As another example, when the image recognition method is applied to a general monitoring system, the target category can be determined according to a recognition requirement of a fine-grained category corresponding to an application scenario of the to-be-recognized image.

[0101] For example, it is assumed that the to-be-recognized image A is shot by a visual doorbell of a user. The specific application scenario corresponding to the to-be-recognized image A is security monitoring, and the recognition requirement of a fine-grained category thereof usually includes fine-grained categories such as gender, age group, and occupation of a person. Based on the to-be-recognized image A, it can be determined that the target category can include a person.

[0102] For another example, it is assumed that the to-be-recognized image B is shot by a monitoring camera at a crossroads. The specific application scenario corresponding to the to-be-recognized image B is traffic monitoring, and the recognition requirement of a fine-grained category thereof usually includes fine-grained categories such as a model, a brand, and a color of a vehicle. Based on the to-be-recognized image B, it can be determined that the target category includes a vehicle.

[0103] In the embodiments of the present application, after the coarse-grained category of the target in the to-be-recognized image is determined by the open set recognition model, the target category that needs to be identified in a fine-grained category is determined in combination with the recognition requirement of a fine-grained category corresponding to the application scenario of the to-be-recognized image. If the coarse-grained category of the target in the to-be-recognized image belongs to the target category, the fine-grained category is identified. If it does not belong, the fine-grained category does not need to be identified. Therefore, unnecessary processing steps can be reduced to the greatest extent to improve the efficiency of image recognition and reduce the occupied computing resources under the condition of meeting the requirements of the actual application scenario.

[0104] In some embodiments, the target recognition result of the to-be-recognized image is determined based on the fine-grained recognition result, including:

[0105] In a case where the confidence of the fine-grained recognition result is greater than or equal to a set confidence threshold, the target recognition result is determined based on the fine-grained recognition result.

[0106] It should be understood that the confidence threshold can be set or input by a user, or calculated by an intelligent algorithm such as a large model or in other manners.

[0107] It should be understood that the confidence of the fine-grained recognition result can be used to reflect the reliability of the fine-grained recognition result. The manner of determining the confidence of the fine-grained recognition result can be set according to actual application requirements, and the embodiments of the present application do not make specific limitations thereto.

[0108] As an example, a standard image of the target of each fine-grained category can be set in advance, and when determining the confidence of the fine-grained recognition result, the similarity (such as cosine similarity) between the image to be recognized and the standard image corresponding to the fine-grained category in the fine-grained recognition result can be taken as the confidence of the fine-grained recognition result.

[0109] As another example, the similarity between the fine-grained category feature corresponding to the fine-grained category in the fine-grained recognition result and the to-be-recognized feature corresponding to the image to be recognized can be calculated and taken as the confidence of the fine-grained recognition result. The fine-grained category feature can be a feature calculated by weighting each image feature corresponding to the fine-grained category.

[0110] As another example, the confidence of the fine-grained recognition result can also be determined according to the statistical distribution based on the similarity. For example, it can be assumed that the similarity obeys a target distribution (such as Gaussian distribution), the probability of the similarity corresponding to the fine-grained recognition result under the target distribution is calculated, and the probability is taken as the confidence of the fine-grained recognition result.

[0111] In the embodiments of the present application, after the fine-grained category of the target in the image to be recognized is recognized by the fine-grained recognition model, the confidence of the obtained fine-grained recognition result can be analyzed first, and when the confidence is greater than or equal to the set confidence threshold, the final target recognition result is determined according to the fine-grained recognition result, so as to realize the fine-grained recognition of the target while guaranteeing the reliability and accuracy of the recognition result.

[0112] Figure 3 A flowchart of an image recognition method provided by some embodiments is shown. Referring to FIG. 1, the method includes the following steps. Figure 3 After the step S102, the method further includes:

[0113] In a case where the coarse-grained category of the target in the coarse-grained recognition result does not belong to the target category, or in a case where the coarse-grained category of the target in the coarse-grained recognition result belongs to the target category but the confidence is less than the confidence threshold, the target recognition result is determined based on the medium-grained category of the target in the coarse-grained recognition result.

[0114] It should be noted that, in the embodiments of the present application, the open-set recognition model is used to recognize the medium-grained category of the target in the to-be-recognized image, and correspondingly, the coarse-grained recognition result can include the coarse-grained category and the medium-grained category of the target in the to-be-recognized image.

[0115] It should be understood that the granularity of a category is generally relative, and the medium-grained category can be a category further divided from the coarse-grained category (such as a subcategory under the coarse-grained category), and correspondingly, the fine-grained category can be a category further divided from the medium-grained category. The specific category granularity division can be set according to actual application requirements.

[0116] For example, assuming that the image recognition method is applied to an animal recognition scene, and assuming that the family category of animals is taken as the medium-grained category according to the animal recognition scene, such as the lion, tiger, parrot, and spider family categories; correspondingly, the coarse-grained category can be a class category with larger granularity, such as the mammal, reptile, bird, or fish class category, or a kingdom category such as the animal kingdom; and the fine-grained category can be a subspecies category such as the Siberian tiger, South China tiger, or Bengal tiger.

[0117] For another example, assuming that the image recognition method is applied to a camera, and assuming that the target categories include animals and plants, and the determined medium-grained categories are the family category of animals and the genus category of plants (such as the Malus genus, Triticum genus, or Chrysanthemum genus), then the coarse-grained category (such as the kingdom category including animals and plants) and the fine-grained category (such as the subspecies category of animals, and the species category or subspecies category of plants) can be determined according to the medium-grained categories.

[0118] Alternatively, the target category can be a coarse-grained or medium-grained category, and the specific category can be set according to actual application requirements.

[0119] Alternatively, when the target category is a coarse-grained category, the granularity of the medium-grained category and / or the fine-grained category corresponding to different target categories can be different. For example, when the target categories include plants and animals, the fine-grained category corresponding to plants can be a species category, and the fine-grained category corresponding to animals can be a subspecies category that is finer than the species category.

[0120] When the coarse-grained category of the target in the coarse-grained recognition result does not belong to the target category, it indicates that the current recognition granularity usually meets the recognition requirement of the current recognition scene, and thus the target recognition result of the to-be-recognized image can be directly determined according to the medium-grained category of the target in the coarse-grained recognition result, without further performing fine-grained category recognition, thereby greatly improving the efficiency of image recognition and reducing the occupied computing resources.

[0121] Referring to Figure 3 When the coarse-grained category of the target in the coarse-grained recognition result belongs to the set target category, but the confidence of the fine-grained recognition result is less than the confidence threshold, it indicates that the fine-grained category is of low reliability in the to-be-recognized image, and thus the target recognition result can be determined according to the medium-grained category of the target in the coarse-grained recognition result, to ensure the reliability of the image recognition result.

[0122] In some embodiments, considering that the local detail features of the target in the to-be-recognized image may be affected by reasons such as light or background interference, so that the target recognition model is difficult to accurately recognize the target, when the coarse-grained category of the target in the coarse-grained recognition result belongs to the set target category, but the confidence of the fine-grained recognition result is less than the confidence threshold, the to-be-recognized image can be enhanced (such as background conversion, light adjustment, geometric transformation, etc.), and the enhanced to-be-recognized image is input into the target recognition model to obtain a new fine-grained recognition result. If the confidence of the new fine-grained recognition result is still less than the confidence threshold, the target recognition result is determined according to the medium-grained category of the target in the coarse-grained recognition result.

[0123] In the embodiments of the present application, if the coarse-grained category of the target in the to-be-recognized image indicated by the coarse-grained recognition result does not belong to the target category, the target recognition result can be directly determined according to the medium-grained category of the target in the coarse-grained recognition result, to reduce unnecessary recognition steps, and the determined target recognition result is the medium-grained category of relatively fine granularity, thereby improving the efficiency of image recognition and reducing resource occupation while ensuring the accuracy and practicability of the target recognition result. If the coarse-grained category of the target in the to-be-recognized image belongs to the target category, but the fine-grained recognition result is of low reliability, the target recognition result can also be determined according to the medium-grained category of the target in the coarse-grained recognition result, to ensure the reliability of the finally determined recognition result.

[0124] In some embodiments, before the step S102, the method further includes:

[0125] The fine-tuning sample is constructed, and one fine-tuning sample includes one training image and the coarse-grained category of the target in the training image.

[0126] The multi-modal large language model is fine-tuned based on the fine-tuning sample, and the trained open-set recognition model is determined based on the fine-tuned multi-modal large language model.

[0127] It should be understood that when constructing the fine-tuning sample, the training image and the coarse-grained category of the target in the image can be obtained based on the public data set to construct the required fine-tuning sample; or the image corresponding to the coarse-grained category can also be generated by the generation model, which is used as the training image corresponding to the coarse-grained category, thereby constructing the fine-tuning sample. The implementation of the embodiment of the present application does not make specific restrictions on the implementation of the fine-tuning sample.

[0128] In some other embodiments, one fine-tuning sample can be composed of a training image-coarse-grained category-intermediate-grained category triple.

[0129] It should be understood that when fine-tuning the multi-modal large language model based on the fine-tuning sample, the training image in the fine-tuning sample can generally be used as the input data, and the coarse-grained category in the fine-tuning sample can be used as the output label; during the fine-tuning process, the training image and the corresponding prompt (such as generating the coarse-grained category of the target in the image) can be input into the multi-modal large language model, and the multi-modal large language model can identify and generate the coarse-grained category of the target in the training image based on the prompt; then, the difference between the coarse-grained category generated by the multi-modal large language model and the output label is calculated based on the set loss function (such as the cross-entropy loss function), and the model parameters of the multi-modal large language model are fine-tuned based on the difference and the set optimization algorithm (such as the gradient descent algorithm), thereby obtaining the fine-tuned multi-modal large language model.

[0130] In some embodiments, in the case where the fine-tuned multi-modal large language model does not meet the fine-tuning requirements (such as the number of iterations reaching the number threshold, such as 10), the steps of inputting the training image in the fine-tuning sample and the corresponding prompt into the multi-modal large language model and fine-tuning the model parameters of the multi-modal large language model can be repeated until the fine-tuned multi-modal large language model meets the fine-tuning requirements, and the multi-modal large language model that meets the fine-tuning requirements is used as the open set recognition model.

[0131] In the embodiment of the present application, in order to reduce the training difficulty, the fine-tuning sample can be directly constructed to fine-tune the general multi-modal large language model, and the fine-tuned multi-modal large language model is used as the open set recognition model for the identification of the coarse-grained category of the image target. Since the general multi-modal large language model is usually a model trained based on a large amount of data, it can exhibit certain accuracy in various tasks, therefore, based on the constructed fine-tuning sample, the multi-modal large language model is fine-tuned to obtain the required trained open set recognition model, which improves the efficiency and significantly improves the accuracy and other performances of the multi-modal large language model in the open set coarse-grained category recognition, so that it can better meet the identification requirements of the coarse-grained category of the image target.

[0132] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0133] Embodiment two:

[0134] Corresponding to the image recognition method described in the above embodiment, Figure 4 The structure block diagram of the image recognition device provided by the embodiments of the present application is shown, and only the parts related to the embodiments of the present application are shown for ease of illustration.

[0135] Referring to Figure 4 The device comprises an image acquisition module 41, a coarse-grained recognition module 42, a fine-grained recognition module 43 and a target recognition result determination module 44. Among them,

[0136] The image acquisition module 41 is configured to acquire a to-be-recognized image.

[0137] The coarse-grained recognition module 42 is configured to input the to-be-recognized image into a trained open set recognition model to obtain a coarse-grained recognition result output by the open set recognition model, and the open set recognition model is configured to recognize a coarse-grained category of a target in the to-be-recognized image.

[0138] The fine-grained recognition module 43 is configured to input the to-be-recognized image into a trained target recognition model to obtain a fine-grained recognition result output by the target recognition model in a case where the coarse-grained category of the target in the coarse-grained recognition result belongs to a set target category, and the target recognition model is configured to recognize a fine-grained category of the target in the to-be-recognized image.

[0139] The target recognition result determination module 44 is configured to determine a target recognition result of the to-be-recognized image based on the fine-grained recognition result.

[0140] In the embodiments of the present application, since the open set recognition model can better handle seen and unseen categories in the training process, the trained open set recognition model can quickly and comprehensively recognize the coarse-grained category of the target in the to-be-recognized image, and can obtain a coarse-grained recognition result with high accuracy. At the same time, when the coarse-grained category of the target in the to-be-recognized image in the coarse-grained recognition result belongs to the set target category, the target recognition model is used to recognize the fine-grained category of the target in the to-be-recognized image, so that more accurate recognition results can be provided when needed, and the target recognition accuracy is improved and the probability of misrecognition is further reduced.

[0141] In some embodiments, the fine-grained recognition module 43 comprises:

[0142] The fine-grained recognition unit is configured to perform feature extraction on the to-be-recognized image by using the target recognition model, and compare the extracted to-be-recognized features with pre-stored fine-grained category features, and determine the fine-grained recognition result according to a comparison result.

[0143] In some embodiments, the image recognition device further comprises:

[0144] The feature weighting module is configured to, for any fine-grained category, perform a weighting calculation on each image feature corresponding to the fine-grained category, to obtain a fine-grained category feature corresponding to the fine-grained category and store the fine-grained category feature.

[0145] In some embodiments, the image recognition device further comprises:

[0146] The target category determination module is configured to determine the target category based on a recognition requirement corresponding to the to-be-recognized image, the recognition requirement being used to reflect a recognition requirement of an application scenario corresponding to the to-be-recognized image for the fine-grained category.

[0147] In some embodiments, the target recognition result determination module 44 comprises:

[0148] The result determination unit is configured to, in a case where the confidence of the fine-grained recognition result is greater than or equal to a set confidence threshold, determine the target recognition result based on the fine-grained recognition result.

[0149] In some embodiments, the open-set recognition model is further used to recognize a medium-grained category of the target in the to-be-recognized image, and the image recognition device further comprises:

[0150] The coarse-grained target recognition result determination module is configured to, in a case where the coarse-grained category of the target in the coarse-grained recognition result does not belong to the target category, or in a case where the coarse-grained category of the target in the coarse-grained recognition result belongs to the target category but the confidence is less than the confidence threshold, determine the target recognition result based on the medium-grained category of the target in the coarse-grained recognition result.

[0151] In some embodiments, the image recognition device further comprises:

[0152] The sample construction module is configured to construct a fine-tuning sample, and one fine-tuning sample comprises one training image and a coarse-grained category of a target in the training image.

[0153] The fine-tuning module is configured to fine-tune the multi-modal large language model based on the fine-tuning sample, and determine the trained open-set recognition model based on the fine-tuned multi-modal large language model.

[0154] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0155] Example 3:

[0156] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 The diagram shows only one processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, which, when executed, performs the steps of any of the above method embodiments.

[0157] The electronic device 5 can be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0158] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0159] The memory 51 can be an internal storage unit of the electronic device 5, such as a hard disk or a memory of the electronic device 5 in some embodiments. The memory 51 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5 in some other embodiments. Further, the memory 51 can include both the internal storage unit and the external storage device of the electronic device 5. The memory 51 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, etc. The memory 51 can also be used to temporarily store data that has been output or is to be output.

[0160] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0161] The embodiments of the present application also provide a network device, which comprises at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor implements the steps in any of the method embodiments described above when executing the computer program.

[0162] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps in any of the method embodiments described above.

[0163] The embodiments of the present application provide a computer program product, which, when running on an electronic device, enables the electronic device to implement the steps in any of the method embodiments described above.

[0164] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can at least include any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0165] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0166] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0167] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the above-described apparatus / network device embodiments are only schematic. For example, the division of the modules or units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection between each of the units can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0168] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0169] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An image recognition method, characterized in that, include: Acquire the image to be recognized; The image to be identified is input into a trained open set recognition model to obtain a coarse-grained recognition result output by the open set recognition model. The open set recognition model is used to identify the coarse-grained category of the target in the image to be identified. If the coarse-grained category of the target in the coarse-grained recognition result belongs to the set target category, the image to be recognized is input into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model. The target recognition model is used to identify the fine-grained category of the target in the image to be recognized. The target recognition result of the image to be recognized is determined based on the fine-grained recognition result.

2. The image recognition method as described in claim 1, characterized in that, The step of inputting the image to be recognized into a trained target recognition model to obtain a fine-grained recognition result output by the target recognition model includes: The target recognition model extracts features from the image to be recognized, compares the extracted features with pre-stored fine-grained category features, and determines the fine-grained recognition result based on the comparison result.

3. The image recognition method as described in claim 2, characterized in that, Before inputting the image to be recognized into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model, the method further includes: For any fine-grained category, a weighted calculation is performed based on each image feature corresponding to the fine-grained category to obtain and store the fine-grained category features.

4. The image recognition method as described in claim 1, characterized in that, Before inputting the image to be recognized into the trained target recognition model to obtain the fine-grained recognition result output by the target recognition model, when the coarse-grained category of the target in the coarse-grained recognition result belongs to the set target category, the method further includes: The target category is determined based on the recognition requirements corresponding to the image to be recognized. The recognition requirements are used to reflect the fine-grained category recognition requirements of the application scenario corresponding to the image to be recognized.

5. The image recognition method as described in claim 1, characterized in that, Determining the target recognition result of the image to be recognized based on the fine-grained recognition result includes: If the confidence level of the fine-grained recognition result is greater than or equal to a set confidence level threshold, the target recognition result is determined based on the fine-grained recognition result.

6. The image recognition method as described in claim 5, characterized in that, The open set recognition model is also used to identify the medium-granularity category of the target in the image to be recognized. After inputting the image to be recognized into the trained open set recognition model and obtaining the coarse-granularity recognition result output by the open set recognition model, the model further includes: If the coarse-grained category of the target in the coarse-grained identification result does not belong to the target category, or if the coarse-grained category of the target in the coarse-grained identification result belongs to the target category and the confidence level is less than the confidence level threshold, the target identification result is determined based on the medium-grained category of the target in the coarse-grained identification result.

7. The image recognition method according to any one of claims 1 to 6, characterized in that, Before inputting the image to be recognized into the trained open set recognition model to obtain the coarse-grained recognition result output by the open set recognition model, the method further includes: Construct fine-tuned samples, each fine-tuned sample comprising a training image and a coarse-grained category of the target in the training image; The multimodal large language model is fine-tuned based on the fine-tuned samples, and the trained open set recognition model is determined based on the fine-tuned multimodal large language model.

8. An image recognition device, characterized in that, include: The image acquisition module is used to acquire the image to be recognized; A coarse-grained recognition module is used to input the image to be recognized into a trained open-set recognition model to obtain a coarse-grained recognition result output by the open-set recognition model. The open-set recognition model is used to identify the coarse-grained category of the target in the image to be recognized. A fine-grained recognition module is used to input the image to be recognized into a trained target recognition model when the coarse-grained category of the target in the coarse-grained recognition result belongs to a set target category, and obtain the fine-grained recognition result output by the target recognition model. The target recognition model is used to identify the fine-grained category of the target in the image to be recognized. The target recognition result determination module is used to determine the target recognition result of the image to be recognized based on the fine-grained recognition result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images