Image recognition method and apparatus

Abstract

Embodiments of the present application provide an image recognition method and device, and relate to the technical field of image processing. The method comprises: obtaining a to-be-recognized image; performing image feature extraction on the to-be-recognized image to obtain a first feature vector; searching, in a retrieval library for storing sample images and corresponding sample categories, a sample image whose similarity to the first feature vector meets a similarity condition, the second feature vector being used to represent image features of the sample image; and determining a target category of the to-be-recognized image according to a sample category corresponding to the searched sample image. The embodiments of the present application can solve the problems of low recognition accuracy, instability and poor generalization performance in the related art.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image recognition method and apparatus. Background Technology

[0002] Image recognition is an important research topic in the field of computer vision and has been widely applied in many areas. For example, image recognition of plankton in marine environments is used to achieve long-term, continuous, in-situ observation of these plankton.

[0003] Currently, image recognition typically involves training a convolutional neural network (CNN) model using a training set, and then using this CNN model to predict the category of the image to be recognized. In this image recognition scheme based on image classification, the training set needs to be continuously updated, leading to frequent retraining of the CNN model in order to maintain its recognition performance.

[0004] However, updating the training set relies on a large amount of manual annotation and correction. Therefore, how to improve recognition accuracy and robustness while reducing human intervention, and thus ensure generalization performance, remains an unsolved problem. Summary of the Invention

[0005] This application provides an image recognition method, apparatus, electronic device, and storage medium, which can solve the problems of low recognition accuracy, lack of robustness, and poor generalization performance in related technologies. The technical solutions are as follows:

[0006] According to one aspect of the embodiments of this application, an image recognition method includes: acquiring an image to be recognized; extracting image features from the image to be recognized to obtain a first feature vector; searching in a retrieval library for storing sample images and their corresponding sample categories for finding sample images whose similarity to a second feature vector and the first feature vector satisfies a similarity condition, wherein the second feature vector is used to represent the image features of the sample image; and determining the target category of the image to be recognized based on the sample category corresponding to the found sample image.

[0007] According to one aspect of the embodiments of this application, an image recognition device includes: an image acquisition module for acquiring an image to be recognized; a feature extraction module for extracting image features from the image to be recognized to obtain a first feature vector; an image search module for searching, in a retrieval library for storing sample images and their corresponding sample categories, a sample image whose similarity to a second feature vector and the first feature vector satisfies a similarity condition, wherein the second feature vector is used to represent the image features of the sample image; and an image recognition module for determining the target category of the image to be recognized based on the sample category corresponding to the searched sample image.

[0008] In one exemplary embodiment, the feature extraction module includes an extractor unit for converting the image to be recognized into the first feature vector using a feature extractor that has completed model training.

[0009] In an exemplary embodiment, the apparatus further includes: a model training module, configured to train a base model on image pairs in a training set to obtain the feature extractor, the base model including a first training branch and a second training branch, the first training branch and the second training branch respectively including a feature extraction layer and a dimensionality reduction layer; the model training module includes: an image traversal unit, configured to traverse the image pairs in the training set, the image pairs including positive sample pairs and negative sample pairs, the two sample images in the positive sample pair belonging to the same sample category, and the two sample images in the negative sample pair belonging to different sample categories; the traversal includes: inputting the two sample images in the image pair into the first training branch and the second training branch respectively for processing; calculating the model loss value based on the processing results obtained from the first training branch and the second training branch; and a convergence unit, configured to converge the feature extractor from the feature extraction layer in the base model if the model loss value satisfies the convergence condition.

[0010] In an exemplary embodiment, the apparatus further includes: an image pair construction module; the image pair construction module includes: an amplification unit, configured to perform at least two different image data enhancement processes on one of the sample images in the training set, so that at least a first enhanced image and a second enhanced image are obtained from the sample image; and a pairing unit, configured to perform image pairing processing on the first enhanced image and the second enhanced image obtained from the amplification of each sample image in the training set, to obtain the image pair.

[0011] In an exemplary embodiment, the image search module includes: a similarity calculation unit, configured to calculate the similarity between the second feature vector and the first feature vector for each second feature vector in the feature vector set, wherein the feature vector set is constructed from the second feature vectors of sample images in the retrieval library; and an image search unit, configured to select the sample image with the highest similarity between the second feature vector and the first feature vector as the sample image found from the retrieval library.

[0012] In an exemplary embodiment, the apparatus further includes: a set construction module, configured to construct the feature vector set from the second feature vectors of sample images in the retrieval library; the set construction module includes: a vector addition unit, configured to extract image features from each sample image in the retrieval library, obtain the second feature vector of each sample image in the retrieval library, and add it to the feature vector set; a vector traversal unit, configured to traverse the second feature vectors in the feature vector set, using the traversed second feature vector as a first vector, calculate the similarity between the first vector and the remaining second feature vectors in the feature vector set, and obtain a first similarity; and a vector deletion unit, configured to delete second feature vectors with high redundancy from the feature vector set based on the first similarity, wherein the redundancy is used to indicate the number of similar second feature vectors in the feature vector set.

[0013] In an exemplary embodiment, the vector deletion unit includes: a vector determination subunit, configured to use a second feature vector whose first similarity to the first vector is greater than a first preset threshold as a second vector; a similarity calculation subunit, configured to calculate the similarity between the second vector and the remaining second feature vectors in the feature vector set to obtain a second similarity; a redundancy calculation subunit, configured to determine the redundancy of the first vector based on the number of second feature vectors whose first similarity to the first vector is greater than the first preset threshold, and to determine the redundancy of the second vector based on the number of second feature vectors whose second similarity to the second vector is greater than a second preset threshold; and a deletion subunit, configured to delete the first vector from the feature vector set if the redundancy of the first vector is greater than the redundancy of the second vector.

[0014] In an exemplary embodiment, the image recognition module includes an image recognition unit, configured to, if the second feature vector of the found sample image satisfies the decision condition, use the sample category corresponding to the found sample image as the target category of the image to be recognized.

[0015] In one exemplary embodiment, the apparatus further includes: a new category correction module, configured to correct the target category of the image to be identified in response to a category correction instruction; and a new category addition module, configured to add the image to be identified and its corrected target category to the retrieval library in response to a category addition instruction if the corrected target category of the image to be identified is a new category, wherein the new category refers to the target category of the image to be identified being different from the sample categories in the retrieval library.

[0016] According to one aspect of the embodiments of this application, an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program in the memory through the communication bus; when the computer program is executed by the processor, it implements the image recognition method as described above.

[0017] According to one aspect of the embodiments of this application, a storage medium stores a computer program thereon, which, when executed by a processor, implements the image recognition method as described above.

[0018] According to one aspect of the embodiments of this application, a computer program product includes a computer program stored in a storage medium. The processor of an electronic device reads the computer program from the storage medium and executes the computer program, causing the electronic device to implement the image recognition method as described above when executed.

[0019] The beneficial effects of the technical solution provided in this application are:

[0020] In the above technical solution, based on the first feature vector of the image to be identified, sample images whose similarity to the second feature vector and the first feature vector satisfy the similarity condition are searched in the retrieval library used to store sample images and their corresponding sample categories. Then, according to the sample category corresponding to the found sample image, the target category of the image to be identified is determined. Thus, an image recognition scheme that replaces image classification with image retrieval is realized. Since the recognition accuracy of image retrieval depends on the sample images and their corresponding sample categories in the retrieval library, unlike image classification which depends on frequent changes in the training set and retraining of the convolutional neural network model, the recognition accuracy and robustness can be fully improved with minimal human intervention, and the generalization performance can be fully guaranteed. This effectively solves the problems of low recognition accuracy, lack of robustness, and poor generalization performance in related technologies. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0022] Figure 1 This is a schematic diagram based on the implementation environment involved in this application;

[0023] Figure 2 This is a flowchart illustrating an image recognition method according to an exemplary embodiment;

[0024] Figure 3 This is a schematic diagram illustrating an image to be identified as a Region of Interest (ROI) image according to an exemplary embodiment;

[0025] Figure 4 This is a schematic diagram of the structure of a basic model according to an exemplary embodiment;

[0026] Figure 5 This is a schematic diagram of the structure of a feature extraction layer according to an exemplary embodiment;

[0027] Figure 6 This is a flowchart illustrating the model training process of a feature extraction layer according to an exemplary embodiment;

[0028] Figure 7 This is a schematic diagram illustrating an image pairing process according to an exemplary embodiment;

[0029] Figure 8a This is a flowchart illustrating a method for constructing a set of feature vectors according to an exemplary embodiment;

[0030] Figure 8b yes Figure 8a A flowchart of a method in one embodiment for step 550 of the corresponding embodiment;

[0031] Figure 9 This is a flowchart illustrating another image recognition method according to an exemplary embodiment;

[0032] Figure 10 This is a schematic diagram illustrating an image recognition framework based on image retrieval according to an exemplary embodiment;

[0033] Figure 11 This is a structural block diagram of an image recognition device according to an exemplary embodiment;

[0034] Figure 12 This is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0035] Figure 13 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0036] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0037] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0038] As mentioned earlier, in image recognition schemes based on image classification, the training set needs to be continuously updated, which in turn requires the convolutional neural network model to be retrained more frequently in order to maintain the recognition performance of the image recognition based on the convolutional neural network model.

[0039] Taking image recognition of plankton in the marine environment as an example, the constant changes in the natural marine environment inevitably lead to continuous variations in the types and abundance of plankton. Furthermore, when plankton is sampled at different times in the same sea area, data drift occurs due to the diurnal vertical migration of plankton. Therefore, in the process of plankton image recognition, to ensure a balanced and stable data distribution, frequent changes to the training set and retraining of the convolutional neural network model are necessary to maintain image recognition performance against the backdrop of constantly changing plankton types and abundance. However, updating the training set relies heavily on manual annotation and correction. Moreover, training sets constructed from images sampled at limited spatiotemporal scales and resolutions can never fully and accurately reflect plankton in the real marine environment. All of these factors inevitably affect the accuracy of image recognition and fail to meet the needs of real-time observation of plankton in the marine environment.

[0040] As can be seen from the above, the relevant technologies still have limitations such as low recognition accuracy, lack of robustness, and poor generalization performance.

[0041] Therefore, the image recognition method provided in this application can effectively improve recognition accuracy and robustness, and fully guarantee generalization performance. Accordingly, the image recognition method is applicable to image recognition devices, which can be deployed on electronic devices configured with the von Neumann architecture, such as desktop computers, laptops, servers, etc.

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0043] Figure 1 This is a schematic diagram of an implementation environment involved in an image recognition method. It should be noted that this implementation environment is merely an example adapted to the present invention and should not be considered as providing any limitation on the scope of the invention.

[0044] The implementation environment includes a data acquisition terminal 110 and a server terminal 130.

[0045] Specifically, the acquisition terminal 110 can also be considered an image acquisition device, including but not limited to electronic devices with shooting functions such as cameras, camcorders, and video recorders. For example, the acquisition terminal 110 is an underwater camera.

[0046] Server 130 can be an electronic device such as a desktop computer, laptop computer, or server, or it can be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide backend services, such as, but not limited to, image recognition services.

[0047] The server 130 and the acquisition terminal 110 establish a network communication connection in advance via wired or wireless means, and data transmission between the server 130 and the acquisition terminal 110 is realized through this network communication connection. The transmitted data includes, but is not limited to, the image to be identified.

[0048] In one application scenario, through the interaction between the acquisition terminal 110 and the server 130, the acquisition terminal 110 captures and obtains the image to be recognized, and uploads the image to be recognized to the server 130 to request the server 130 to provide image recognition services.

[0049] For server 130, after receiving the image to be identified uploaded by acquisition terminal 110, it calls the image recognition service to search for sample images similar to the image to be identified in the retrieval library that stores sample images and their corresponding sample categories. Then, based on the sample category corresponding to the found sample image, it determines the target category of the image to be identified, thereby realizing an image recognition scheme that replaces image classification with image retrieval, thus solving the problems of low recognition accuracy, lack of robustness, and poor generalization performance in related technologies.

[0050] Please see Figure 2 This application provides an image recognition method applicable to electronic devices, specifically, such devices. Figure 1 The server 130 in the implementation environment is shown.

[0051] The method is illustrated using an electronic device as the execution subject for each step, but this does not constitute a specific limitation.

[0052] like Figure 2 As shown, the method may include the following steps:

[0053] Step 310: Obtain the image to be recognized.

[0054] The image to be identified is composed of Figure 1 The image acquisition device shown is used to capture and generate images of the environment containing the target object. The target object refers to an object in the captured environment; for example, the target object can be an underwater organism, specifically plankton in a marine environment.

[0055] It is understood that shooting can be a single shot or continuous shooting. For the same target object, continuous shooting yields a video clip, and the image to be identified can be any number of frames from that video. Conversely, multiple shots yield multiple photos, and the image to be identified can be any number of those photos. In other words, the image to be identified in this embodiment can refer to a moving image, such as multiple frames from a video clip or multiple photos, or it can refer to a static image, such as any single frame from a video clip or any single photo from multiple photos. Accordingly, image recognition in this embodiment can be performed on moving images or static images; no limitation is imposed here.

[0056] Regarding the acquisition of the image to be recognized, the image can originate from images captured in real-time by an image acquisition device, or it can be images captured by an image acquisition device within a historical time period that are pre-stored in the electronic device. Therefore, for the electronic device, after the image acquisition device captures and acquires the image to be recognized, it can process the image in real-time, or it can pre-store it for later processing. For example, it can process the image when the CPU of the electronic device is low, or it can process the image according to the instructions of the staff. Thus, the image recognition in this embodiment can be based on images acquired in real-time or images acquired within a historical time period; no specific limitation is made here.

[0057] In one possible implementation, the image to be identified is a ROI (region of interest) image. That is, in the image to be identified, the target object is located within the ROI. This can also be understood as the target object being clearly distinguished from the background area by being identified through the ROI. For example... Figure 3 As shown, in the ROI image, the target object is plankton, located in the region of interest (grayish-white area), which is significantly different from the background area (black area).

[0058] Step 330: Extract image features from the image to be recognized to obtain the first feature vector.

[0059] The first feature vector is used to represent the image features of the sample image. It can also be considered that the first feature vector is an accurate description of the image features of the image to be identified. It should be understood that different images to be identified will have different extracted image features, and correspondingly, the first feature vector will also be different.

[0060] In one possible implementation, image feature extraction can be achieved using feature extraction algorithms such as histogram of oriented gradients, local binary pattern features, and Haar-like features.

[0061] In one possible implementation, image feature extraction is achieved through convolutional kernels. It should be noted that using different numbers and sizes of convolutional kernels will yield first feature vectors of varying lengths, reflecting the image to be identified at different scales.

[0062] In one possible implementation, image feature extraction is achieved through a feature extractor. Specifically, the image to be recognized is converted into a first feature vector using a feature extractor that has completed model training.

[0063] Step 350: In the retrieval library used to store sample images and their corresponding sample categories, find sample images whose similarity between the second feature vector and the first feature vector satisfies the similarity condition.

[0064] First, it should be noted that a retrieval database essentially establishes a correspondence between sample images and their corresponding sample categories. This correspondence allows for the rapid determination of the sample category of a sample object, thus serving as the basis for image retrieval. In one possible implementation, a sample image refers to an image labeled with a sample category; in other words, a sample image is an image carrying a label indicating the sample category.

[0065] Image retrieval is essentially a measurement of the similarity between the image to be identified and the sample images in the search database. However, image recognition based on image retrieval does not directly obtain the target category of the image to be identified. Instead, it indirectly obtains the target category of the image to be identified by comparing the similarity between the image to be identified and the sample images in the search database. That is, it first obtains the sample categories corresponding to the sample images whose similarity with the image to be identified meets the similarity condition, and then obtains the target category of the image to be identified from these categories.

[0066] Secondly, in this embodiment, the comparison of the similarity between the image to be identified and the sample images in the search database is achieved by calculating the similarity between the first feature vector and the second feature vector. The first feature vector represents the image features of the image to be identified, and the second feature vector represents the image features of the sample images in the search database.

[0067] In one possible implementation, similarity calculation schemes include, but are not limited to: cosine similarity, Euclidean distance, Manhattan distance, Jaccard similarity coefficient, Pearson correlation coefficient, etc.

[0068] Let's take cosine similarity as an example to illustrate the process of similarity calculation:

[0069]

[0070] In the calculation formula (1), Similarity(x, y) represents the similarity between x and y, and the value range of this similarity is [0, 1]; x represents the first feature vector of the image to be identified, and y represents the second feature vector of the sample image. It should be understood that the closer the similarity is to 1, the closer the first feature vector and the second feature vector are, that is, the more similar the image to be identified and the sample image are.

[0071] It is worth mentioning that, as mentioned above, the image to be identified is not limited to a static image, such as a photograph or a frame, but can also be a dynamic image. If the image to be identified is a dynamic image, such as multiple photographs or multiple frames, then calculation formula (1) and calculation formula (2) can be combined to calculate multiple similarities at the same time.

[0072] V = Q × G T (2).

[0073] Where V represents the similarity result matrix, Q represents the first feature vector matrix of the image to be identified, and G represents the second feature vector matrix of each sample image in the search database.

[0074] Based on this, in the similarity result matrix V, the values ​​in each column of the i-th row represent the similarity between the first feature vector of the i-th photo or i-th frame in the image to be identified and the second feature vector of each sample image in the retrieval database. This method not only greatly improves the efficiency of similarity calculation, but also enables simultaneous recognition of multiple photos / multiple frames, which is beneficial for batch processing of images to be identified and can effectively improve recognition efficiency.

[0075] In one possible implementation, the similarity condition refers to the highest similarity. Therefore, the sample image with the highest similarity between the second feature vector and the first feature vector is used as the sample image found in the retrieval database. Of course, in other embodiments, the similarity condition can also refer to a similarity exceeding a similarity threshold (e.g., similarity threshold = 0.8), or a similarity ranking exceeding a set ranking (e.g., a set ranking of 10), etc. In this case, sample images with a similarity greater than 0.8 between the second feature vector and the first feature vector, or sample images whose similarity between the second feature vector and the first feature vector ranks in the top 10, are used as the sample images found in the retrieval database.

[0076] In one possible implementation, the second feature vector is pre-calculated and stored in the storage area of ​​the electronic device. In this way, when performing image recognition on different images to be recognized, the pre-calculated second feature vector can be directly read from the storage area of ​​the electronic device, avoiding repeated extraction of the second feature vector in each image recognition process, thereby further improving the recognition efficiency of image recognition.

[0077] In one possible implementation, the second feature vector is stored in the electronic device's memory as a LUT (Look-up Table). In this way, during image recognition, the LUT can be directly loaded into the electronic device's memory, thereby avoiding the repeated extraction of the second feature vector in each image recognition process.

[0078] The above process is particularly applicable to image recognition of out-of-distribution samples. For example, for new categories that did not participate in the training phase, image recognition schemes based on image classification not only affect the accuracy of classification but also lead to inaccurate abundance quantification. In contrast, image recognition schemes based on image retrieval can more accurately exclude out-of-distribution samples through similarity calculation, thereby effectively ensuring the recognition accuracy of image recognition.

[0079] Step 370: Determine the target category of the image to be identified based on the sample category corresponding to the found sample image.

[0080] In other words, the sample category corresponding to the found sample image is the recognition result obtained by performing image recognition on the image to be recognized, that is, the target category of the image to be recognized.

[0081] The inventors realized that the target category of the image to be identified might be a new category, that is, it does not belong to any of the sample categories corresponding to the sample images in the retrieval library. In other words, the target category of the image to be identified is an unknown category. In this case, the target category of the image to be identified cannot actually be correctly obtained based on the sample category corresponding to the found sample image.

[0082] Therefore, in order to avoid image recognition errors, this embodiment proposes a decision condition to reject the recognition of unknown categories, thereby avoiding recognition errors.

[0083] In one possible implementation, the decision condition refers to the similarity between the image to be identified and the found sample image being greater than a similarity threshold. Therefore, the category decision process based on this condition specifically means: if the similarity between the second feature vector of the found sample image and the first feature vector of the image to be identified is greater than the similarity threshold, then the sample category corresponding to the found sample image is taken as the target category of the image to be identified; otherwise, the target category of the image to be identified is determined as a new category.

[0084] Of course, in other embodiments, the decision conditions may also be related to the weights configured for the found sample images, which is not intended to be a specific limitation.

[0085] Through the above process, an image recognition scheme that replaces image classification with image retrieval is realized. Since the recognition accuracy of image retrieval depends on the sample images in the retrieval database and their corresponding sample categories, unlike image classification which depends on frequent changes in the training set and retraining of the convolutional neural network model, the recognition accuracy can be fully guaranteed with minimal human intervention. This effectively solves the problems of low recognition accuracy, lack of robustness, and poor generalization performance in related technologies.

[0086] Figure 4 A schematic diagram of the basic model in one embodiment is shown. Figure 4 In this model, the basic model includes a first training branch and a second training branch, which respectively include a feature extraction layer and a dimensionality reduction layer. The feature extraction layer can be considered as a feature extractor that has not yet completed model training, used for extracting image features. The dimensionality reduction layer consists of two fully connected layers, used to further reduce the dimensionality of the feature vector obtained from the feature extraction layer; for example, converting a feature vector of length 2048 obtained from the feature extraction layer into a feature vector of length 128.

[0087] Figure 5 A schematic diagram of the feature extraction layer in one embodiment is shown. Figure 5 In this model, the feature extraction layer is a convolutional neural network with a depth of 50 layers and no fully connected layers. For example... Figure 5 As shown, in the 50-layer structure, in addition to the convolutional layer (Conv), pooling layer (Pool), activation function layer (ReLU), and based on the ResNeXt module, an SE (Squeeze-and-Excitation) attention module is also introduced. This not only makes the feature vector obtained based on this feature extraction layer have a strong abstract expression ability, but also, with the help of the attention mechanism, it can focus on the parts of the image that play a major role in recognition, such as the region of interest in the ROI image, thus ensuring that image features can be extracted more effectively.

[0088] Now combined Figures 4 to 7 The training process for the feature extraction layer, which involves training the base model using image pairs from the training set to obtain the feature extractor, is explained in detail below:

[0089] Please see Figure 6 In one exemplary embodiment, the model training process may include the following steps:

[0090] Step 410: Traverse the image pairs in the training set.

[0091] The image pairs include positive sample pairs and negative sample pairs. In a positive sample pair, the two sample images belong to the same sample category, while in a negative sample pair, the two sample images belong to different sample categories.

[0092] This section explains the process of constructing image pairs:

[0093] like Figure 7 As shown, for one of the sample images 701 in the training set, at least two different image data augmentation processes are performed, so that at least a first augmented image 7011 and a second augmented image 7012 are obtained from the sample image 701. Among them, image data augmentation processes include, but are not limited to: random cropping, rotation, flipping, grayscale conversion, brightness adjustment, contrast adjustment, saturation adjustment, etc., and are not limited here.

[0094] Image pairing is performed on the first and second enhanced images obtained from the amplification of each sample image in the training set.

[0095] For example, suppose the sample images in the training set include 701 and 702. Accordingly, the first enhanced image and the second enhanced image obtained by amplifying sample image 701 are 7011 and 7012, respectively, and the first enhanced image and the second enhanced image obtained by amplifying sample image 702 are 7021 and 7022, respectively.

[0096] After image pairing processing, the resulting image pairs include {7011, 7012}, {7011, 7021}, {7011, 7022}, {7012, 7021}, {7012, 7022}, and {7021, 7022}. Among these image pairs, {7011, 7012} and {7021, 7022} are positive sample pairs, while {7011, 7021}, {7011, 7022}, {7012, 7021}, and {7012, 7022} are negative sample pairs.

[0097] The traversal process for image pairs in the training set can then include the following steps:

[0098] Step 411: Input the two sample images in the image pair into the first training branch and the second training branch respectively for processing.

[0099] like Figure 4 As shown, in one possible implementation, the processing in the first or second training branch includes at least: extracting image features through a feature extraction layer, and reducing the dimensionality of the feature vector through a dimensionality reduction layer.

[0100] It is worth mentioning that, to avoid distortion, the sample images are preprocessed before being input into the first or second training branch. One possible implementation includes, but is not limited to, padding, scaling, and normalization. This approach, by avoiding distortion, helps to further improve the accuracy of recognition.

[0101] The preprocessing steps, such as padding and scaling, aim to ensure a uniform input size for the first or second training branch. For example, the uniform input size is 224×224.

[0102] Normalization preprocessing refers to the pixel-by-pixel normalization of the sample image after encoding preprocessing, according to the following calculation formula (3).

[0103]

[0104] Among them, I Norm I represents the pixels in the sample image that have undergone normalization, and I represents the pixels in the sample image that are yet to be processed.

[0105] mean and std represent the pixel mean and pixel standard deviation of all pixels in all sample images in the training set, respectively.

[0106] Step 413: Calculate the model loss value based on the processing results obtained from the first training branch and the second training branch.

[0107] In one possible implementation, the formula (4) for calculating the model loss value is as follows:

[0108]

[0109] Among them, L sup This represents the model loss value;

[0110] I represents the set of all sample images in the training set; P(i) represents the set of positive sample pairs in the training set that exclude the i-th sample image; A(i) represents the set of all sample images in the training set that exclude the i-th sample image.

[0111] |P(i)| represents the number of sample images in set P(i);

[0112] z i z is the feature vector of the i-th sample image in set I; p z is the feature vector of the p-th sample image in set P(i); α is the feature vector of the α-th sample image in set A(i); τ is the temperature hyperparameter, used to balance the degree of attention the loss function pays to positive and negative sample pairs respectively.

[0113] If the model loss value satisfies the convergence condition, then proceed to step 430.

[0114] Otherwise, if the model loss value makes the convergence condition not met, then proceed to step 415.

[0115] It should be noted that the convergence condition can refer to the minimum model loss value or the loss value threshold, or it can refer to the number of iterations meeting the iteration threshold. There is no limitation here, and it can be flexibly set according to the actual needs of the application scenario.

[0116] Step 415: Update the parameters of the base model and return to execute step 410.

[0117] Step 430: The feature extractor is obtained by the convergence of the feature extraction layer in the base model.

[0118] Thus, the supervised contrastive learning model training of the feature extraction layer is completed, enabling the feature extractor to bring the two sample images in the positive sample pair closer together and push the two sample images in the negative sample pair further apart in the feature space.

[0119] It is worth mentioning that after the model training is completed, both the dual training branches and the dimensionality reduction layer are discarded. Only one of the feature extraction layers in the dual training branches is retained as a feature extractor for subsequent image recognition. Compared with the convolutional neural network model in image classification, the model structure is greatly simplified, further avoiding reliance on frequent changes in the training set to maintain recognition performance, which is more conducive to improving recognition accuracy.

[0120] In an exemplary embodiment, the above method may further include the following step: constructing a feature vector set from the second feature vector of the sample images in the retrieval library.

[0121] In one possible implementation, the feature vector set is a LUT.

[0122] As mentioned earlier, the second feature vector can be stored in the electronic device's storage area using a LUT (Learning Unit) to avoid repeated extraction of the second feature vector in each image recognition process, thereby improving the recognition efficiency. However, the inventors also realized that as the number of sample images in the search database increases, the number of pre-calculated second feature vectors in the LUT also increases. Since it is necessary to calculate the similarity between the first feature vector and each second feature vector in the LUT, the number of second feature vectors in the LUT will affect the similarity calculation speed, thus affecting the recognition efficiency.

[0123] Therefore, in this embodiment, a feature vector set construction process is proposed to achieve LUT pruning, which can reduce the size of the LUT, i.e., reduce the number of second feature vectors in the LUT, while maintaining the diversity of second feature vectors in the LUT as much as possible.

[0124] Specifically, such as Figure 8a As shown, the process of constructing a feature vector set may include the following steps:

[0125] Step 510: Extract image features from each sample image in the retrieval database to obtain the second feature vector of each sample image in the retrieval database, and add it to the feature vector set.

[0126] Step 530: Traverse the second feature vector in the feature vector set, take the traversed second feature vector as the first vector, and calculate the similarity between the first vector and the other second feature vectors in the feature vector set to obtain the first similarity.

[0127] Step 550: Based on the first similarity, delete the second feature vector with high redundancy from the feature vector set.

[0128] The redundancy of the second feature vector indicates the number of similar second feature vectors in the feature vector set. It should be understood that a higher redundancy indicates a greater number of similar second feature vectors in the feature vector set. Therefore, the sample image corresponding to the second feature vector can be considered redundant in the feature vector set, and thus, the second feature vector can be deleted from the feature vector set.

[0129] In one possible implementation, such as Figure 8b As shown, the LUT pruning process may include the following steps:

[0130] Step 551: The second feature vector whose first similarity to the first vector is greater than the first set threshold is taken as the second vector.

[0131] Step 553: Calculate the similarity between the second vector and the remaining second feature vectors in the feature vector set to obtain the second similarity.

[0132] Step 555: Determine the redundancy of the first vector based on the number of second feature vectors whose first similarity to the first vector is greater than a first set threshold, and determine the redundancy of the second vector based on the number of second feature vectors whose second similarity to the second vector is greater than a second set threshold.

[0133] The redundancy of the first vector indicates the number of similar second feature vectors that exist within the feature vector set. Similarity is defined as a first similarity greater than a first set threshold.

[0134] The redundancy of the second vector indicates the number of similar second feature vectors in the feature vector set. Similarity is defined as a second similarity greater than a first set threshold.

[0135] Step 557: Based on the redundancy of the second feature vector, delete the corresponding second feature vector from the feature vector set.

[0136] Specifically, if the redundancy of the first vector is greater than that of the second vector, the first vector is deleted from the feature vector set; conversely, if the redundancy of the second vector is greater than that of the first vector, the second vector is deleted from the feature vector set.

[0137] For example, the second eigenvectors in the eigenvector set are A, B, C, and D.

[0138] Suppose the second feature vector encountered in the current iteration is A, which is used as the first vector. Then, calculate the similarity between the first vector A and the other second feature vectors B, C, and D respectively, and obtain the first similarity scores: 0.91, 0.95, and 0.97.

[0139] Assuming that the first similarity scores of 0.91, 0.95, and 0.97 are all greater than the first set threshold (0.8), then the second feature vectors B, C, and D are taken as the second vectors. At this time, the similarity scores of the second vector B with the other second feature vectors A, C, and D are calculated respectively, and the second similarity scores are obtained as follows: 0.91, 0.7, and 0.97. The similarity scores of the second vector C with the other second feature vectors A, B, and D are calculated respectively, and the second similarity scores are obtained as follows: 0.95, 0.97, and 0.75. The similarity scores of the second vector D with the other second feature vectors A, B, and C are calculated respectively, and the second similarity scores are obtained as follows: 0.97, 0.75, and 0.77.

[0140] Assuming the second threshold is also 0.8, it can be seen from the above that the number of second feature vectors (B, C, D) with a first similarity greater than 0.8 to the first vector A is 3, the number of second feature vectors (A, D) with a second similarity greater than 0.8 to the second vector B is 2, the number of second feature vectors (A, B) with a second similarity greater than 0.8 to the second vector C is 2, and the number of second feature vectors (A) with a second similarity greater than 0.8 to the second vector D is 1.

[0141] Assuming redundancy is represented by a number of vectors, then the redundancy of the first vector A is 3, the redundancy of the second vector B is 2, the redundancy of the second vector C is 2, and the redundancy of the second vector D is 1. Therefore, the first vector A, with a redundancy of 3, is deleted from the feature vector set. Of course, in other embodiments, redundancy can also be represented in other forms, such as a normalization method based on the number of vectors; this is not a specific limitation.

[0142] The first and second threshold values ​​can be the same or different, and can be flexibly adjusted according to the actual needs of the application scenario to balance recognition efficiency and recognition accuracy. For example, in application scenarios where recognition efficiency is required, a smaller first threshold value can be set.

[0143] With the cooperation of the above embodiments, for different second feature vectors in the LUT, only when the distance between different second feature vectors in the feature space is less than a first set threshold is the second feature vector retained, thereby realizing LUT pruning, which can reduce the size of the LUT and ensure the diversity of the second feature vectors in the LUT as much as possible.

[0144] In a planktonic application scenario, assuming there are 200 planktonic categories, and each planktonic category contains 1,000 sample images, the retrieval database contains 200,000 sample images. Therefore, the LUT can contain a maximum of 200,000 second feature vectors. Taking the LUT in the NVIDIA RTX3090 GPU as an example, the image recognition time for the image to be identified is at most 5.8ms, which can fully meet the needs of real-time observation of planktonic organisms in the marine environment.

[0145] Please see Figure 9 In one exemplary embodiment, the above method may further include the following steps:

[0146] Step 610: In response to the category correction instruction, the target category of the image to be recognized is corrected.

[0147] Step 630: If the target category of the image to be identified after correction is a new category, in response to the category addition instruction, the image to be identified and its corrected target category are added to the retrieval library.

[0148] Here, the new category refers to the target category of the image to be identified after correction, which is different from the sample category in the retrieval library.

[0149] As mentioned earlier, in image recognition schemes based on image classification, the recognition performance of image recognition partly depends on a large amount of manual annotation and manual correction. Due to the large amount of manual work and the short cycle of manual intervention, this type of image recognition scheme is not conducive to reducing the cost of image recognition and cannot achieve a relatively robust, flexible and inexpensive automatic image recognition scheme.

[0150] Therefore, in this embodiment, a human-computer interaction interface is provided in the image recognition scheme based on image retrieval, which helps to detect and correct deviations in image recognition in a timely manner, so as to fully guarantee the recognition performance of image recognition.

[0151] Specifically, Figure 10 A schematic diagram of an image recognition framework based on image retrieval is shown in one embodiment. Figure 10 The image recognition framework includes: a query image module 801 for acquiring images to be recognized; a retrieval library 802 for storing sample images and their corresponding sample categories; a feature extractor 803 for performing image feature extraction; a LUT 804 for storing second feature vectors; a measurement module 805 for calculating the similarity between the first feature vector and the second feature vector; a decision module 806 for determining the target category of the image to be recognized; and a human-computer interaction interface.

[0152] like Figure 10As shown, the human-computer interaction interface includes a correction interface 807 and an addition interface 808. The correction interface 807 is used to generate category correction instructions to correct the target category of the image to be recognized; the addition interface 808 is used to generate category addition instructions to add the image to be recognized and its corrected target category to the retrieval library.

[0153] Assuming the electronic device is a smartphone that provides access to the recognition results, the smartphone displays a browsing page for viewing these results. This browsing page includes a calibration interface and an add interface. It should be noted that the calibration interface and the add interface are essentially controls that enable human-computer interaction; for example, these controls could be input boxes, selection boxes, buttons, switches, progress bars, etc.

[0154] If a user discovers that the target category of the image to be recognized is a new category, they can trigger a corresponding operation at the correction interface. If the correction interface detects this user-triggered operation, it generates a category correction instruction, which instructs the electronic device to correct the target category of the image to be recognized in response to this instruction. For example, the correction interface could be an input box where the user can enter the name of the new category; the user's input is considered a corresponding operation triggered by the user at the correction interface. Similarly, if the target category of the image to be recognized after correction is a new category, the user can also trigger a corresponding operation at the add interface. If the add interface detects this user-triggered operation, it generates a category add instruction, which instructs the electronic device to add the image to be recognized and its corrected target category to the retrieval database in response to this instruction. For example, the add interface could be a "Confirm / Cancel" button that the user can click; the user's click is considered a corresponding operation triggered by the user at the add interface.

[0155] It should be noted that the specific actions triggered by the user will vary depending on the input components configured on the electronic device. For example, if the electronic device is a smartphone with a touchscreen, the triggered actions may be gesture operations such as clicking, touching, or swiping; or if the electronic device is a laptop with a mouse, the triggered actions may be mechanical operations such as single-clicking, double-clicking, or dragging. This embodiment does not constitute a specific limitation in this regard.

[0156] Under the above embodiments, effective supervision is introduced into image recognition. On the one hand, it helps to detect and correct deviations in image recognition in a timely manner, so as to fully guarantee the recognition performance of image recognition. On the other hand, it can effectively enhance the diversity of sample images in the retrieval library. This not only avoids the decline in recognition effect caused by data drift, but also effectively improves the recognition effect of image recognition. Moreover, the diverse retrieval library allows more areas of the feature space to be covered by point groups composed of sample images, so that the target category corresponding to the image to be recognized can be more accurately identified when it falls into the area, which is conducive to improving recognition accuracy.

[0157] Furthermore, the image recognition framework based on image retrieval has the characteristic that "by adding a new category to the retrieval library, the target category of the image to be recognized can be immediately identified as a new category". This makes retraining not always necessary for the image recognition framework, thus helping to delay the need for retraining and reduce the frequency of retraining, providing more convenience and greater flexibility for image recognition.

[0158] The following are embodiments of the apparatus described in this application, which can be used to execute the image recognition method involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the image recognition method involved in this application.

[0159] Please see Figure 11 This application provides an image recognition device 900, including but not limited to: an image acquisition module 910, a feature extraction module 930, an image search module 950, and an image recognition module 970.

[0160] The image acquisition module 910 is used to acquire the image to be recognized.

[0161] The feature extraction module 930 is used to extract image features from the image to be recognized and obtain the first feature vector.

[0162] The image search module 950 is used to search for sample images whose similarity to the second feature vector and the first feature vector satisfies the similarity condition in a retrieval library used to store sample images and their corresponding sample categories. The second feature vector is used to represent the image features of the sample image.

[0163] The image recognition module 970 is used to determine the target category of the image to be recognized based on the sample category corresponding to the found sample image.

[0164] It should be noted that the image recognition device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image processing. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the image recognition device will be divided into different functional modules to complete all or part of the functions described above.

[0165] Furthermore, the image recognition device and image recognition method embodiments provided in the above embodiments belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0166] Figure 12 A schematic diagram of the structure of an electronic device is shown according to an exemplary embodiment. This electronic device is suitable for... Figure 1 The server 130 in the implementation environment is shown.

[0167] It should be noted that this electronic device is merely an example adapted to this application and should not be construed as providing any limitation on the scope of use of this application. Furthermore, this electronic device should not be interpreted as requiring or depending on any specific feature. Figure 12 One or more components of the exemplary electronic device 2000 shown.

[0168] The hardware structure of electronic devices 2000 can vary significantly due to differences in configuration or performance, such as... Figure 12 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0169] Specifically, power supply 210 is used to provide operating voltage for various hardware devices on electronic device 2000.

[0170] Interface 230 includes at least one wired or wireless network interface for interacting with external devices. For example, to perform... Figure 1 The diagram shows the interaction between the acquisition terminal 110 and the server terminal 130 in the implementation environment.

[0171] Of course, in other examples adapted in this application, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 12 As shown, this does not constitute a specific limitation.

[0172] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0173] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0174] Application 253 is a computer program that performs at least one specific task based on operating system 251, and may include at least one module ( Figure 12 (Not shown in the image), each module may contain a computer program for the electronic device 2000. For example, the image recognition device may be considered as an application program 253 deployed on the electronic device 2000.

[0175] Data 255 can be photos, pictures, etc. stored on a disk, or images to be identified, etc., and is stored in memory 250.

[0176] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, an image recognition method may be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0177] Furthermore, this application can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of this application is not limited to any specific hardware circuit, software, or combination thereof.

[0178] Please see Figure 13 This application provides an electronic device 4000, which may include: desktop computer, laptop computer, electronic device, etc.

[0179] exist Figure 13 The electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.

[0180] The processor 4001 and memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0181] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0182] The communication bus 4002 may include a path for transmitting information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0183] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0184] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.

[0185] When the computer program is executed by the processor 4001, it implements the image recognition methods in the above embodiments.

[0186] Furthermore, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the image recognition methods described in the above embodiments.

[0187] This application provides a computer program product comprising a computer program stored in a storage medium. A processor of an electronic device reads the computer program from the storage medium and executes the computer program, causing the electronic device to perform the image recognition methods described in the above embodiments.

[0188] Compared with related technologies, the image recognition framework based on image retrieval, under the premise of good image quality of sample images in the retrieval library, uses the powerful image feature representation brought by supervised contrastive learning to make positive examples of the same category cluster together and negative examples of different categories move away in the feature space. This not only avoids dependence on model retraining and can effectively improve the recognition efficiency of image recognition, but also fully guarantees the recognition accuracy of image recognition.

[0189] Furthermore, the retrieval library within the image recognition framework is not only suitable for retraining but also for user adjustments, facilitating flexible and customized services for recognition tasks with different attributes and spatial dimensions. For example, for recognition tasks targeting diverse organisms, the number of sample images in the retrieval library should be expanded to ensure that image recognition capabilities can accommodate diversity. For recognition tasks targeting plankton in a specific sea area, the sample images in the retrieval library can be limited to specific sample categories, excluding impossible categories. This reduces the computational load of similarity calculations and prevents the image to be recognized from being misidentified as an impossible category, indirectly ensuring the recognition performance. For recognition tasks targeting a limited number of organisms of interest, the retrieval library can be further reduced to include only the sample categories of interest.

[0190] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0191] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image recognition method, characterized in that, The method includes: Acquire the image to be recognized; The image to be identified is converted into a first feature vector using a feature extractor that has completed model training; wherein the feature extractor is obtained by training a base model based on image pairs in the training set, and the base model includes a first training branch and a second training branch, the first training branch and the second training branch respectively including a feature extraction layer and a dimensionality reduction layer; In a retrieval library used to store sample images and their corresponding sample categories, sample images whose similarity to the second feature vector and the first feature vector satisfies the similarity condition are searched. The second feature vector is used to represent the image features of the sample image. The second feature vector stored in the retrieval library is obtained by deleting second feature vectors with higher redundancy than the rest of the second feature vectors from the feature vector set. The redundancy is used to indicate the number of similar second feature vectors in the feature vector set. The target category of the image to be identified is determined based on the sample category corresponding to the found sample image.

2. The method as described in claim 1, characterized in that, The step of training the base model based on image pairs in the training set includes: The image pairs in the training set are traversed, each image pair including positive sample pairs and negative sample pairs. In a positive sample pair, the two sample images belong to the same sample category, and in a negative sample pair, the two sample images belong to different sample categories. The traversal includes: The two sample images in the image pair are respectively input into the first training branch and the second training branch for processing; Calculate the model loss value based on the processing results obtained from the first training branch and the second training branch; If the model loss value satisfies the convergence condition, then the feature extractor is obtained by the convergence of the feature extraction layer in the base model.

3. The method as described in claim 2, characterized in that, The method further includes: constructing the image pairs in the training set; The process of constructing the image pairs in the training set includes: If at least two different image data augmentation processes are performed on one of the sample images in the training set, then at least a first augmented image and a second augmented image can be obtained from the sample image. The first and second enhanced images, obtained by amplifying each sample image in the training set, are subjected to image pairing processing to obtain the image pair.

4. The method as described in claim 1, characterized in that, The step of searching for sample images in a retrieval library used to store sample images and their corresponding sample categories, where the similarity between the second feature vector and the first feature vector satisfies a similarity condition, includes: For each second feature vector in the feature vector set, the similarity between the second feature vector and the first feature vector is calculated respectively. The feature vector set is constructed from the second feature vectors of sample images in the retrieval library. The sample image with the highest similarity between the second feature vector and the first feature vector is used as the sample image found from the retrieval library.

5. The method as described in claim 4, characterized in that, The method further includes: constructing the feature vector set from the second feature vectors of sample images in the retrieval library; The construction of the feature vector set from the second feature vectors of sample images in the retrieval database includes: For each sample image in the retrieval database, image features are extracted to obtain the second feature vector of each sample image in the retrieval database, and then added to the feature vector set. The second feature vector in the feature vector set is traversed, and the traversed second feature vector is used as the first vector. The similarity between the first vector and the remaining second feature vectors in the feature vector set is calculated to obtain the first similarity. Based on the first similarity, a second feature vector with high redundancy is deleted from the feature vector set, where the redundancy is used to indicate the number of similar second feature vectors in the feature vector set.

6. The method as described in claim 5, characterized in that, The step of deleting highly redundant second feature vectors from the feature vector set based on the first similarity includes: The second feature vector whose first similarity to the first vector is greater than the first set threshold is taken as the second vector; Calculate the similarity between the second vector and the remaining second feature vectors in the feature vector set to obtain the second similarity. The redundancy of the first vector is determined based on the number of second feature vectors whose first similarity to the first vector is greater than a first set threshold, and the redundancy of the second vector is determined based on the number of second feature vectors whose second similarity to the second vector is greater than a second set threshold. If the redundancy of the first vector is greater than that of the second vector, then the first vector is deleted from the feature vector set.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the target category of the image to be identified based on the sample category corresponding to the found sample image includes: If the second feature vector of the found sample image satisfies the decision condition, then the sample category corresponding to the found sample image is taken as the target category of the image to be identified.

8. The method according to any one of claims 1 to 6, characterized in that, After determining the target category of the image to be identified based on the sample category corresponding to the found sample image, the method further includes: In response to a category correction command, the target category of the image to be identified is corrected; If the target category of the image to be identified after correction is a new category, in response to the category addition instruction, the image to be identified and its corrected target category are added to the retrieval library, where the new category refers to the target category of the image to be identified after correction that is different from the sample category in the retrieval library.

9. An image recognition device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be recognized; The feature extraction module is used to convert the image to be identified into a first feature vector using a feature extractor that has completed model training; wherein, the feature extractor is obtained by training a basic model based on image pairs in the training set, and the basic model includes a first training branch and a second training branch, the first training branch and the second training branch respectively including a feature extraction layer and a dimensionality reduction layer; The image search module is used to search for sample images whose similarity to the first feature vector and the second feature vector satisfies a similarity condition in a retrieval library used to store sample images and their corresponding sample categories. The second feature vector is used to represent the image features of the sample image. The second feature vector stored in the retrieval library is obtained by deleting second feature vectors with higher redundancy than the rest of the second feature vectors from the feature vector set. The redundancy is used to indicate the number of similar second feature vectors in the feature vector set. The image recognition module is used to determine the target category of the image to be recognized based on the sample category corresponding to the found sample image.

Images