Image recognition method and device, computer device and storage medium

Abstract

The application relates to an image recognition method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a target positive sample image; obtaining a confusion class set corresponding to a positive sample class of the target positive sample image; wherein the confusion class set is obtained by confusing classification of a class set through class recognition data of an image set; taking a confusion class in the confusion class set as a negative sample class, and obtaining a target negative sample image corresponding to the negative sample class; training a to-be-trained image recognition model based on the target positive sample image and the target negative sample image, obtaining a trained image recognition model, and performing image recognition based on the trained image recognition model. The model training process can fully analyze the difference between the positive sample image and the negative sample image in the same confusion class set, can obtain an image recognition model for accurately recognizing images with easily confused classes, and improves the accuracy of image recognition.

Description

Technical Field

[0001] This application relates to the field of image classification technology, and in particular to an image recognition method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of network technology, image recognition technology has emerged. This technology determines the category of an image by analyzing its features. In some cases, it is necessary to distinguish and identify images based on similar categories, such as differentiating between a coffee shop and a Western restaurant, which are easily confused.

[0003] In traditional technologies, during image recognition, different images under similar categories are easily identified as belonging to the same category, resulting in low image recognition accuracy.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] Therefore, it is necessary to provide an image recognition method, apparatus, computer device, and storage medium to address the aforementioned technical problems.

[0006] An image recognition method includes: acquiring a target positive sample image; acquiring a set of confused categories corresponding to the positive sample category corresponding to the target positive sample image; wherein the set of confused categories is obtained by performing category recognition on an image set and performing confused classification on the category set corresponding to the image set based on the category recognition data; using the confused categories in the set of confused categories as negative sample categories, and using the images corresponding to the negative sample categories as target negative sample images corresponding to the target positive sample image; training an image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain a trained image recognition model, and performing image recognition based on the trained image recognition model.

[0007] An image recognition device includes: a positive sample image acquisition module for acquiring a target positive sample image; a confusion category set acquisition module for acquiring a confusion category set corresponding to the positive sample category corresponding to the target positive sample image; wherein the confusion category set is obtained by performing category recognition on an image set and performing confusion classification on the category set corresponding to the image set based on the category recognition data; a negative sample image acquisition module for using the confusion categories in the confusion category set as negative sample categories and the images corresponding to the negative sample categories as target negative sample images corresponding to the target positive sample image; and a model training module for training an image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain a trained image recognition model, and performing image recognition based on the trained image recognition model.

[0008] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring a target positive sample image; acquiring a set of confused categories corresponding to the positive sample category corresponding to the target positive sample image; wherein the set of confused categories is obtained by performing category recognition on an image set and performing confused classification on the category set corresponding to the image set based on the category recognition data; using the confused categories in the set of confused categories as negative sample categories, and using the images corresponding to the negative sample categories as target negative sample images corresponding to the target positive sample image; training an image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain a trained image recognition model, and performing image recognition based on the trained image recognition model.

[0009] A computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, performs the following steps: acquiring a target positive sample image; acquiring a set of confused categories corresponding to the positive sample category corresponding to the target positive sample image; wherein the set of confused categories is obtained by performing category recognition on an image set and performing confused classification on the category set corresponding to the image set based on the category recognition data; using the confused categories in the set of confused categories as negative sample categories, and using the images corresponding to the negative sample categories as target negative sample images corresponding to the target positive sample image; training an image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain a trained image recognition model, and performing image recognition based on the trained image recognition model.

[0010] The aforementioned image recognition method, apparatus, computer equipment, and storage medium acquire target positive sample images and target negative sample images from the same set of confusion categories. The obtained target negative sample images are difficult sample images that are easily confused with target positive sample images. Based on the target positive sample images and target negative sample images, the image recognition model to be trained is trained. The model training process can fully analyze the differences between positive sample images and negative sample images in the same set of confusion categories, thereby improving the ability of the trained model to recognize different images under similar categories. Therefore, an image recognition model that can accurately identify images with easily confused categories can be obtained, thus improving the accuracy of image recognition. Attached Figure Description

[0011] Figure 1 This is an application environment diagram of the image recognition method in one embodiment;

[0012] Figure 2 This is a flowchart illustrating an image recognition method in one embodiment;

[0013] Figure 3 This is a schematic diagram of the structure of a block in one embodiment;

[0014] Figure 4 This is a schematic diagram showing the connections between the modules of the ResNet 101 neural network module in one embodiment;

[0015] Figure 5 This is a flowchart illustrating the image recognition method in another embodiment;

[0016] Figure 6 This is a flowchart illustrating the image recognition method in another embodiment;

[0017] Figure 7 This is a flowchart illustrating the image recognition method in yet another embodiment;

[0018] Figure 8 This is a flowchart illustrating the image recognition method in yet another embodiment;

[0019] Figure 9 This is a structural block diagram of an image recognition device in one embodiment;

[0020] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] In one embodiment, the image recognition method, apparatus, computer equipment, and storage medium provided in this application can be implemented using artificial intelligence (AI) technology. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0023] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0024] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0025] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0026] The solutions provided in this application involve technologies such as machine learning in artificial intelligence, and are specifically illustrated through the following embodiments:

[0027] The image recognition method provided in this application can be applied to, for example... Figure 1The application environment shown includes server 102, terminal 104, and terminal 106. Terminals 104 and 106 communicate with server 102 via a network. Server 102 is configured with database 108. When a target positive sample image is acquired, server 102 retrieves a set of confusion categories from database 108, acquires a target negative sample image based on the confusion category set, and trains an image recognition model using the target positive and negative sample images to obtain the trained image recognition model. The server can acquire the target positive sample image from the terminal. Figure 1 The terminal is not shown in the diagram. When terminal 104 sends an image to be classified to the server, server 102 uses a trained image recognition model to identify the image, obtains the image classification result, and outputs the image classification result to terminal 106. Server 102 can be implemented using a standalone server, a server cluster consisting of multiple servers, or a cloud server. Terminal 104 or terminal 106 can be, but is not limited to, various laptops, smartphones, smart TVs, desktop computers, tablets, and portable wearable devices. Terminal 104 or terminal 106 can be equipped with a client for acquiring images, such as a video client, browser client, instant messaging client, or educational client. These clients can be used to capture or read images from the storage space of terminal 104 or terminal 106, and then send the images to server 102 for processing.

[0028] In one embodiment, such as Figure 2 As shown, an image recognition method is provided. It can be understood that this method can be applied to a terminal or a server, or to a system including both a terminal and a server. It can be executed jointly by the server and the terminal. For example, the target positive sample image can be acquired by the terminal and sent to the server, or the terminal can determine the amount of data to be processed, uploading data with a processing volume exceeding a threshold to the server for processing, and processing data with a processing volume below the threshold on the terminal. This method is applied to... Figure 1 Taking the server in the example, the method includes the following steps:

[0029] S202, Obtain the target positive sample image.

[0030] The images can be from various scenes. A scene can be at least one of the following: a Chinese restaurant, a Western restaurant, a coffee shop, a rainforest, a park, a natural river, or a swamp. The scene of an image can serve as its category; for example, an image from a Western restaurant scene is considered to belong to the category of "Western Restaurant."

[0031] Positive sample images are the sample images corresponding to the category that the model is expected to correctly classify. They determine the category that the model to be trained should identify, and the images corresponding to that category are used as positive sample images. Conversely, negative sample images are the sample images corresponding to categories other than the category that the model is expected to correctly classify. That is, the categories corresponding to negative sample images and positive sample images can be different. For example, the category corresponding to a positive sample image is "coffee shop," while the category corresponding to a negative sample image is "restaurant."

[0032] In one embodiment, the target positive sample image can be a single image, multiple images, or multiple video frames extracted from a video. Furthermore, when there are multiple target positive sample images, these images can be images of the same category or images of different categories. For example, they can be images from multiple scenes uploaded by the terminal, or multiple images from the same scene uploaded by the terminal.

[0033] In one embodiment, the implementation process of S202 can be as follows: the server obtains the video uploaded by the terminal, extracts video frames from the video, and uses the extracted video frames as the target positive sample image. Alternatively, the server can randomly select one image from the images corresponding to each category in the confusion category set as the target positive sample image.

[0034] S204, based on the positive sample category corresponding to the target positive sample image, obtain the confusion category set corresponding to the positive sample category; wherein the confusion category set is obtained by performing category recognition on the image set and performing confusion classification on the category set corresponding to the image set based on the category recognition data.

[0035] In this context, the category corresponding to the target positive sample image is the positive sample category. The category corresponding to the sample image can be pre-labeled. Sample images can be manually or automatically labeled. If an object in a sample image belongs to a certain category, it is labeled with the specific category name. For example, for an image taken in a coffee shop scene, the category of the image would be labeled as "coffee shop".

[0036] The category set corresponding to an image set can be a set constructed from the categories corresponding to the target images in the image set. When the target image corresponds to N categories, the category set can be a set composed of these N categories. The size of N can be determined based on the specific categories to which the target image belongs. The confusion category set is a set composed of categories that are easily confused with each other. A confusion category set can contain at least two similar categories. The confusion degree between categories in the same confusion category set satisfies a confusion degree condition, such as the confusion degree condition including at least one of the following: the confusion degree ranking is before a ranking threshold or the confusion degree is greater than a confusion degree threshold.

[0037] Category identification data is data obtained by extracting image features from an image. In some embodiments, category identification data includes at least one of image extracted features obtained by feature extraction of the image or categories obtained by image classification.

[0038] In one embodiment, before acquiring the target positive sample image and the target negative sample image, a process of determining a set of confusion categories may be included. The set of confusion categories contains at least two categories that are easily confused with each other, which can be considered as secondary categories above the categories; therefore, the set of confusion categories can also be called a superclass or major category. The categories that are easily confused with each other can be similar scenes. For example, coffee shops and Western restaurants are similar scenes, so coffee shops and Western restaurants can be considered easily confused categories. Therefore, coffee shops and Western restaurants can be integrated into a single set of confusion categories, which can be named "restaurant". Furthermore, the same set of confusion categories contains different scenes that are easily confused with each other. Each set of confusion categories contains 2 to n scene categories, and the scenes in different sets of confusion categories are not easily confused with each other. For example, confusion category set 1 contains office buildings and residences, and the two categories in confusion category set 1 are easily confused; confusion category set 2 contains parks, amusement parks, and gardens, and the three categories in confusion category set 2 are easily confused. However, the categories in confusion category set 1 are not easily confused with the categories in confusion category set 2. The size of n can be determined based on the actual scene category of the image; this embodiment does not limit this.

[0039] In one embodiment, the process of determining the confusion category set may include: performing category identification on the image set to obtain category identification data, performing confusion classification on the category set corresponding to the image set based on the category identification data, and forming a confusion category set by combining the mutually confused categories obtained from the confusion classification.

[0040] In one embodiment, a pre-trained neural network model can be used to perform image recognition on images in an image set to obtain category recognition data. The categories corresponding to the category recognition data are then combined into a confused category set. In this approach, the confused category set is inferred from the results of the neural network model, eliminating the need for manually labeled correspondences between samples and the confused category set. Therefore, manual labeling is not introduced, effectively reducing the cost of manual labeling.

[0041] In one embodiment, all categories can be pre-labeled with a set of confusing categories. For example, the category text labels for office buildings and residences can be aggregated into a single confusing category set called "houses". This pre-labeling method can obtain a more accurate set of confusing categories. In one embodiment, the pre-labeling process can be as follows: determine all possible categories, aggregate the category text labels corresponding to similar categories to obtain a set of confusing categories. For example, assuming there are 1000 images, each corresponding to 100 categories, the category words of the 100 categories can be aggregated to obtain 10 sets of confusing categories. In one embodiment, for the four scene categories of office buildings, residences, parks, and amusement parks, office buildings and residences are easily confused, parks and amusement parks are easily confused, and residences and parks are not easily confused. Therefore, the category text labels for "office buildings" and "residences" are recorded under the confusing category set "buildings", and the category text labels for "parks" and "amusement parks" are recorded under the confusing category set "gardens". This approach eliminates the need to label each image to its specific confusion category set, resulting in minimal labeling and thus avoiding significant labeling costs. It also ensures the accuracy of the confusion category set, guarantees the reliability of image recognition model training, and ultimately improves image recognition accuracy.

[0042] In one embodiment, the implementation process of S204 can be as follows: obtaining a predetermined set of confusion categories, and when the target positive sample image is obtained, determining the set of confusion categories to which the positive sample category of the target positive sample image belongs.

[0043] S206, take the confusion category in the confusion category set as the negative sample category, and take the image corresponding to the negative sample category as the target negative sample image corresponding to the target positive sample image.

[0044] In one embodiment, any category in the confusion category set can be used as a negative sample category, or any category other than the positive sample category in the confusion category set can be used as a negative sample category.

[0045] The number of negative sample categories can be at least one. In one embodiment, at least one category can be identified as a negative sample category from the categories in the confusion category set after removing the positive sample categories.

[0046] In one embodiment, all or some of the images corresponding to the negative sample category can be used as the target negative sample image.

[0047] In another embodiment, the implementation process of S204 and S206 can be as follows: perform category recognition on the image set to obtain category recognition data; perform confusion classification on the category set corresponding to the image set based on the category recognition data; form a confusion category set by combining the mutually confused categories obtained from the confusion classification; when the target positive sample image is obtained, determine the confusion category set to which the positive sample category of the target positive sample image belongs; determine the negative sample category from the confusion category set; and then obtain the target negative sample image.

[0048] S208, the image recognition model to be trained is trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model, and image recognition is performed based on the trained image recognition model.

[0049] The image recognition model can be a neural network model, such as a recurrent neural network (RNN) or a convolutional neural network (CNN), and different neural network models can have different network structures.

[0050] In one embodiment, after determining the structure of the image recognition model, a model to be trained can be constructed based on different network parameters. Target positive and negative sample images are input into the model. During training, the image recognition model extracts image features from the sample images, obtaining image extracted features. Based on these features, the model classifies the sample images, and the model parameters are adjusted according to the accuracy of the classification results. This process continues until the accuracy of the classification results meets the required criteria, at which point the image recognition model is considered a trained model.

[0051] In one embodiment, the target positive sample image and the target negative sample image can be used as a training pair to train the image recognition model. During training, the image recognition model extracts image features from the target positive sample image and the target negative sample image to obtain image extracted features. Classification is performed based on these image extracted features, and the model parameters are adjusted according to the classification results to obtain the trained image recognition model. The trained image recognition model is obtained after fully analyzing the differences in image features between the target positive sample image and the target negative sample image. It can accurately distinguish the category of the image to be recognized from the negative sample category; that is, it can effectively mitigate the interference of the negative sample category and thus accurately output the category of the image to be recognized.

[0052] In the above image recognition method, target positive sample images and target negative sample images are obtained from the same set of confusion categories. The obtained target negative sample images are difficult sample images that are easily confused with target positive sample images. The image recognition model to be trained is trained based on the target positive sample images and target negative sample images. The model training process can fully analyze the differences between positive sample images and negative sample images in the same set of confusion categories, thereby improving the ability of the trained model to recognize different images under similar categories. Therefore, an image recognition model that can accurately identify images that are easily confused by categories can be obtained, thus improving the accuracy of image recognition.

[0053] In one embodiment, when a neural network model is used to classify an image set to obtain a class confusion set, the neural network model used can be a deep neural network model. Alternatively, the image recognition model can also be a deep neural network model. The aforementioned neural network model and image recognition model can be implemented using the same deep neural network model, or they can be implemented using different neural network models.

[0054] In one embodiment, the deep neural network model can have a multi-layered neural network structure. For example, the deep neural network model can be a network model based on ResNet101 (a deep residual network with 101 layers). A block in ResNet101 has a structure like this... Figure 3 As shown, this diagram represents a three-layer residual module. 256-d indicates a 256-dimensional input. 1×1 and 3×3 represent the kernel size, and 64 and 256 represent the number of channels. A plus sign inside a circle represents addition, i.e., the identity mapping. ReLU (Rectified Linear Unit) indicates activation using an activation function to reduce the number of parameters. Figure 3 As shown, the module applies convolutional kernels and ReLU to perform multi-layer processing on the input, superimposes the results of the multi-layer processing with the input, performs ReLU processing on the superimposed results, and then obtains the output of the module.

[0055] Table 1 below shows the structure of ResNet101 in some deep neural network models.

[0056] Table 1 Structure of ResNet101

[0057]

[0058] Among them, ×3, ×4 and ×23 represent 3 modules, 4 modules and 23 modules respectively.

[0059] Table 2 below shows the structure of the output layer in a deep neural network model.

[0060] Table 2. Structure of the Output Layer

[0061] Layer name Output size layer Pool_cr 1×2048 Max pooling Fc_cr 1×N_cr Fully connected

[0062] The Pool_cr layer outputs a deep feature embedding, which is a 1*2048 feature vector obtained through deep feature extraction. The Fc_cr layer outputs the classification result, where Fc stands for fully connected. The network parameters of the Fc_cr layer are obtained by sampling from a Gaussian distribution; that is, for a certain weight in Fc_cr, a sample is taken from the Gaussian distribution as the weight value, and this process is repeated multiple times until all weights in Fc_cr have been assigned values. N_cr represents the number of target categories predicted by the fully connected layers.

[0063] In one embodiment, the deep neural network model is a network model formed by connecting the structures corresponding to Tables 1 and 2 above. By performing forward computation on each module in the table sequentially, one forward training iteration of the deep neural network model can be completed. A schematic diagram of the connections between the modules of the ResNet 101 neural network can be shown below. Figure 4 As shown in Table 2 above, the ResNet 101 neural network module consists of a feature extraction module and a classification module. The feature extraction module includes sub-modules such as Conv1, Conv2, Conv3, Conv4, and Conv5. Figure 4 The classification model in the example is a combination of max pooling layers and fully connected layers. Furthermore, as... Figure 4 As shown, each sub-module in the feature extraction module can be composed of max-pooling layers, Conv2_x, Conv3_x, Conv4_x, and Conv5_x, where Conv2_x, Conv3_x, Conv4_x, and Conv5_x represent the 2nd to 5th convolutional layers, respectively. For example, the Conv2 module consists of a max-pooling layer and three Conv2_x layers, the Conv3 module consists of a max-pooling layer and four Conv3_x layers, and the composition of the Conv4 and Conv5 modules is similar, so it will not be described in detail here.

[0064] The Conv5 module outputs a deep feature map of the training sample images, which is the image extraction feature of the target image. The Pool_cr layer outputs a 1*2048 one-dimensional feature vector, labeled as the embedding, based on the image extraction features from the Conv5 module. This embedding represents the deep, high-dimensional features output after the image undergoes forward computation by the deep learning neural network. The Fc_cr layer obtains the predicted probabilities for each category based on the output of the Pool_cr layer. When there are N target categories, the output of the Fc_cr layer can be a 1*N feature vector, where each feature represents the predicted probability for its corresponding category. Based on this, the deep neural network model outputs the target category corresponding to each target image. The feature map represents the feature map obtained after convolving the image with the filter. New feature maps can be generated by convolving the feature map with the filter.

[0065] In one embodiment, the step of obtaining the confusion category set includes: acquiring a target image set; the target image set includes at least two target images, and the target categories corresponding to each target image form a category set; performing image recognition on each target image in the target image set to obtain category recognition data corresponding to each target image; determining the category confusion degree between target categories based on the category recognition data; and performing confusion classification on the target categories in the category set based on the category confusion degree to obtain the confusion category set.

[0066] In this context, the target category corresponding to a target image is the class to which the target image belongs. This category can be determined through pre-labeling or by classifying the target image using a trained image recognition model. The target categories corresponding to the target images are combined into a set, resulting in a category set. The target image set includes at least two target images, and the target categories corresponding to each target image form the category set. Therefore, when one target image corresponds to one target category, the target image set can correspond to at least two target categories.

[0067] In one embodiment, image feature extraction can be performed by extracting features from the image through methods such as convolution, and can be represented as a feature vector. In some embodiments, feature extraction of the target image can be performed using neural network models, etc. The feature map obtained by convolution in the convolutional layer of the image recognition model can be pooled by a pooling layer, and the depth feature embedding output by the pooling layer can be used as the image feature vector, which can then be used as category recognition data. In one embodiment, an embedding layer is configured in the neural network model, which transforms the discrete features extracted from the image into feature vectors and outputs them as category recognition data.

[0068] In one embodiment, features extracted from an image can be used, such as through convolution, to classify the target image based on these extracted features, and the classification result can be used as category recognition data. In some embodiments, feature extraction and classification of the target image can be performed using neural network models, such as configuring an embedding layer and a classification layer in the neural network model. The embedding layer transforms the discrete features extracted from the image into feature vectors, and the classification layer classifies the feature vectors, outputting the classification result as category recognition data.

[0069] Category confusion is the degree to which target categories are easily confused with each other. The higher the category confusion, the easier it is for the corresponding target categories to be confused, and the easier it is for the image recognition model to identify the corresponding images as belonging to the same category. Category confusion can be determined based on the feature similarity between target categories. For example, feature similarity between target categories can be determined by extracting features from the image, and then category confusion can be determined based on feature similarity. Alternatively, it can be determined based on the results of classifying and recognizing the target image. For example, if an image that is originally of category A is identified as category B, then categories A and B can be considered to be easily confused. If category A is identified as category B multiple times, then the number of misidentifications can be used as the category confusion between categories A and B.

[0070] After obtaining the class confusion degree between target categories, the target categories can be confused and classified according to the class confusion degree. Specifically, corresponding target categories whose class confusion degree meets the confusion degree condition can be regarded as mutually confused categories. For example, at least two related target categories that meet the confusion degree condition can be regarded as mutually confused categories, forming a set of confused categories.

[0071] The above embodiments obtain category recognition data by performing image recognition on target images in a target image set. Based on the category recognition data, the category confusion degree between target categories is obtained. Then, based on the category confusion degree, the target categories are classified into a confusion category set, grouping easily confused target categories into similar categories. This ensures that categories within the same confusion category set are similar, while categories across different confusion category sets are dissimilar. Therefore, positive and negative sample images within the same confusion category set have high similarity, and negative sample images can serve as hard samples for positive sample images, thus enabling the training of a reliable image recognition model.

[0072] In one embodiment, the category identification data includes image extracted features obtained by feature extraction of the target image; determining the category confusion between target categories based on the category identification data includes: obtaining feature similarity between target categories based on image extracted features; and determining the category confusion between target categories based on feature similarity between target categories.

[0073] The feature similarity between target categories can include the similarity between features corresponding to the target categories, the similarity between features of target images of the target categories, and the similarity between features of a target image of a certain target category and features of other target categories.

[0074] In one embodiment, feature similarity and class confusion can be positively correlated; a high feature similarity corresponds to a high class confusion. In some cases, class confusion can also be directly represented by feature similarity. In another embodiment, class confusion can be determined based on the distribution of feature similarity. For example, when feature similarity is less than a first similarity threshold, class confusion is set to a fixed value, which can be preset and determined based on actual conditions. When feature similarity is greater than or equal to a second similarity threshold, class confusion is positively correlated with feature similarity. For example, if feature similarity is 0.1, class confusion is set to 0.2; if feature similarity is 0.2, class confusion is set to 0.4.

[0075] In one embodiment, the class confusion between at least two target categories can be determined based on the feature similarity between them. Specifically, the feature similarity between two target categories is calculated according to a predetermined positive correlation expression, for example, by multiplying it by a coefficient, and the result is determined as the class confusion between the two target categories.

[0076] The above embodiments calculate the feature similarity between target categories by extracting features from images, and then determine the category confusion between target categories. The determined category confusion fully integrates the extracted features, has high accuracy, and requires little computation, which can effectively ensure the efficiency of determining the category confusion.

[0077] In one embodiment, feature similarity can be calculated from the perspective of target categories. Specifically, obtaining feature similarity between target categories based on image-extracted features includes: statistically analyzing the image-extracted features corresponding to different target categories to obtain category representation features corresponding to each target category; and obtaining feature similarity between target categories based on the category representation features corresponding to each target category.

[0078] Here, the category representation feature is used to characterize the features of the target category and can be represented in the form of a feature vector. In some embodiments, the category representation feature can be represented as a feature vector with a vector dimension of 1*2048.

[0079] The process of statistically analyzing the image extraction features corresponding to different target categories can be as follows: For any target category, statistically analyze the image extraction features of the target images corresponding to that target category. For example, calculate at least one of the following: the average or weighted sum of the image extraction features of all target images. Use the statistical result as the category representation feature of that target category. Determine the category representation features of all target categories in a similar manner. Calculate the similarity of the category representation features between target categories to obtain the feature similarity between target categories.

[0080] The above embodiments calculate the feature similarity between target categories based on image-extracted features. Image-extracted features can often be represented in the form of vectors, and the computational cost of vector operations is relatively small. Therefore, the computational efficiency of feature similarity can be effectively improved, thereby improving the efficiency of model training.

[0081] Feature similarity can also be calculated from the perspective of target image and target category. For example, the feature similarity between target category A and target category B can be obtained based on the similarity between the features of a target image of a target category A and the features of target category B.

[0082] In one embodiment, obtaining the feature similarity between target categories based on the category representation features corresponding to each target category includes: obtaining the current image extraction features corresponding to the current target image, wherein the current target image belongs to the first target category; calculating the feature similarity between the current image extraction features and the category representation features corresponding to each second target category, and using the calculated feature similarity as the feature similarity between the first target category and the second target category.

[0083] Here, the current target image is the target image targeted by the server at a certain stage of operation, and can be any target image in the target image set. The current image extracted features are the image features extracted by image recognition of the current target image, which can be represented in the form of a feature vector. In one embodiment, the current image extracted features can be represented as a feature vector with a vector dimension of 1*2048, just like the category representation features.

[0084] In one embodiment, both the first target category and each of the second target categories can be categories from a category set. When determining the first target category, the remaining target categories in the category set can be determined as the second target categories.

[0085] In some embodiments, for any target category, the image extraction features of all target images corresponding to the target category can be compared with the category representation features of other target categories in the category set to obtain similarity calculation results. The similarity calculation results of each target image are statistically analyzed to obtain the feature similarity between the target category and other target categories.

[0086] In one embodiment, the category recognition data includes an image feature vector obtained by feature extraction of the target image, and the category representation feature of the target category is a category feature vector determined based on the image feature vector; the feature similarity between target categories is obtained based on the category representation features corresponding to the target categories, including: obtaining the target image feature vector corresponding to the current target image; determining the similarity between the target image feature vector and the category feature vector of each second target category, respectively, to obtain the feature similarity between the first target category and each second target category.

[0087] In one embodiment, the process of determining the category feature vector includes: for any target category, determining the image feature vector of the corresponding target image; calculating the average value of the image feature vector of the target image to obtain the category feature vector of the arbitrary target category.

[0088] The above embodiments calculate feature similarity based on the image extraction features of the target image and the category representation features of a specific target category. The feature similarity between target categories can be obtained through simple vector calculation, which can effectively improve the calculation efficiency of feature similarity and thus improve the efficiency of model training.

[0089] In one embodiment, determining the class confusion between target categories based on the feature similarity between target categories includes: increasing the class confusion between the first target category and the second target category when the feature similarity between the first target category and the second target category meets a similarity condition; wherein the similarity condition includes at least one of the following: the feature similarity is ranked before a first ranking threshold or the feature similarity is greater than a similarity threshold.

[0090] Increasing the category confusion between the first target category and the second target category can be achieved by increasing the numerical value of the category confusion. Furthermore, the numerical value can be increased based on the original category confusion value. For example, a preset value can be added to the original category confusion value; this preset value can be set as needed, such as 1. For instance, the initial value of the category confusion can be 0, and the category confusion can be progressively increased by 1 based on the feature similarity between each target category. The final category confusion value is then determined as the category confusion between the corresponding target categories.

[0091] The first ranking threshold can be a pre-set fixed value or determined based on the number of target categories. Specifically, the first ranking threshold can be a fixed value such as 4 or 6. The first ranking threshold can also be calculated based on the number of target categories. Further, a preset first ranking coefficient is obtained, and the first ranking coefficient is multiplied by the number of target categories to obtain the first ranking threshold. The similarity threshold can be 0.8 or 0.9, and the numerical range of the similarity threshold can be [0, 1].

[0092] In one embodiment, the feature similarities are sorted in descending order, i.e., from largest to smallest, with higher similarities ranking higher. When a feature similarity is ranked before a first ranking threshold, the feature similarity between the corresponding first target category and the second target category meets the similarity condition, and the class confusion between the corresponding first target category and the second target category is incremented. For example, when the first ranking threshold is 4, if the feature similarity between target category 1 and target category 2 is determined to be among the top three, the class confusion between target category 1 and target category 2 is incremented by 1; if the feature similarity between target category 1 and target category 3 is determined to be among the top three, the class confusion between target category 1 and target category 3 is incremented by 1.

[0093] In one embodiment, the similarity of each feature is compared with a similarity threshold. When the feature similarity is greater than the similarity threshold, the feature similarity between the corresponding first target category and the second target category meets the similarity condition, and the category confusion between the corresponding first target category and the second target category is numerically increased. For example, when the similarity threshold is 0.8, if the feature similarity between target category 1 and target category 2 is determined to be 0.75, which is less than 0.8, target category 1 and target category 2 are determined not to be confused categories, and the category confusion is not changed; if the feature similarity between target category 1 and target category 3 is determined to be 0.9, which is greater than 0.8, target category 1 and target category 3 are determined to be confused categories, and the category confusion between target category 1 and target category 3 is incremented by 1.

[0094] In the above embodiments, the feature similarity of the target category is obtained based on the image extraction features of the target image, and the category confusion degree is determined according to the feature similarity between the target categories. The accurate and reliable category confusion degree can be obtained through simple calculation.

[0095] In one embodiment, the category recognition data includes a third target category obtained by classifying the target image; determining the category confusion between target categories based on the category recognition data includes: comparing the third target category with the standard category corresponding to the target image; the standard category is a fourth target category; when the comparison between the third target category and the standard category corresponding to the target image is inconsistent, increasing the category confusion between the third target category and the fourth target category.

[0096] The standard category corresponding to the target image can be the exact category of the target image, which can be determined by pre-labeling.

[0097] In one embodiment, both the third and fourth target categories can be categories from the category set.

[0098] Increasing the class confusion between the third and fourth target categories can be achieved by increasing the numerical value of the class confusion. Furthermore, a preset value can be added to the original class confusion value; this preset value can be set as needed, for example, it could be 1. For instance, a +1 adjustment can be applied. The initial value of the class confusion can be 0. When the fourth target category is mistakenly identified as the third target category, the class confusion between the third and fourth target categories is incremented by 1. When multiple misidentifications occur, the final class confusion result is obtained based on the cumulative number of misidentifications, and this final class confusion is determined as the class confusion between the third and fourth target categories.

[0099] In one embodiment, a neural network model can be used to classify the target image, and the classification result of the neural network model can be determined as the third target category. Alternatively, the actual category of the target image can be determined as the fourth target category.

[0100] In the above embodiments, when the classification result obtained from classifying the target image is inconsistent with the standard category, it is determined that the category corresponding to the classification result and the standard category are easily confused, and then the category confusion degree between the two is increased. The process of determining the category confusion degree is simple and can effectively ensure the efficiency of determining the category confusion degree.

[0101] In one embodiment, classifying target categories in a category set based on category confusion to obtain a confused category set includes: classifying target categories in the category set whose category confusion meets the confusion condition as mutually confused categories, and forming a confused category set by combining the mutually confused categories; wherein, the confusion condition includes: the category confusion is ranked before a second ranking threshold or the category confusion is greater than at least one of the following:

[0102] The second ranking threshold can be a pre-set fixed value or determined based on the number of target categories. Specifically, the second ranking threshold can be a fixed value such as 3 or 5. The second ranking threshold can also be calculated based on the number of target categories. Further, a preset second ranking coefficient is obtained, which can be 0.1 or 0.2. Multiplying the second ranking coefficient by the number of target categories yields the second ranking threshold.

[0103] The confusion threshold can be a pre-set fixed value or dynamically determined based on the overall category confusion of the target category. Specifically, the confusion threshold can be a fixed value such as 20 or 40. The confusion threshold can also be calculated based on the overall category confusion statistics of the target category. For example, a preset confusion coefficient, which can be 0.8 or 0.9, can be obtained. Multiplying this confusion coefficient by the average category confusion of all target categories yields the confusion threshold.

[0104] In one embodiment, target categories with high confusion levels are aggregated into a confusion category set, and positive and negative sample images are determined using the confusion category set as the unit.

[0105] In one embodiment, the confusion levels of each category are sorted in descending order, with higher confusion levels ranking higher. When a category confusion level ranks before a second ranking threshold, the corresponding target categories are considered sufficiently confusing, and these target categories are treated as mutually confusing categories. For example, when the second ranking threshold is 4, if the confusion level between target category 1 and target category 2 is determined to be among the top three, target category 1 and target category 2 are treated as mutually confusing categories and aggregated into a confused category set; similarly, if the confusion level between target category 1 and target category 3 is determined to be among the top three, target category 1 and target category 3 are treated as mutually confusing categories and aggregated into a confused category set.

[0106] In one embodiment, the confusion level of each category is compared with a confusion level threshold. When the confusion level exceeds the threshold, the corresponding target categories are considered sufficiently confusing, and the corresponding target categories are identified as mutually confusing categories. For example, when the confusion level threshold is 20, if the confusion level between target category 1 and target category 2 is determined to be 10, which is less than 20, then target category 1 and target category 2 are not mutually confusing categories. If the confusion level between target category 1 and target category 3 is determined to be 30, which is greater than 20, then target category 1 and target category 3 are identified as mutually confusing categories and aggregated into a confused category set.

[0107] When there are shared categories among the confused categories, these confused categories can be treated as independent confused categories or merged into a single confused category. For example, suppose target category 1 and target category 2 are confused categories, and target category 3 is also confused categories, with target category 1 being a shared category. In this case, target category 1 and target category 2 can be aggregated into one confused category set, and target category 1 and target category 3 can be aggregated into another confused category set. Alternatively, target category 1, target category 2, and target category 3 can all be aggregated into a single confused category set, thus achieving the merging of confused categories.

[0108] Further, the set of obfuscation categories is determined based on the obfuscation categories obtained from the aggregation, including: merging obfuscation categories that have the same target category among the obfuscation categories obtained from the aggregation to obtain the set of obfuscation categories.

[0109] In the above embodiments, feature similarity of target categories is obtained based on image extraction features of the target image. Category confusion is then determined based on the feature similarity between target categories. Accurate and reliable category confusion can be obtained through simple calculations. Furthermore, since only categories with similar features are aggregated into the same confused category set, the categories within the confused category set must have a certain degree of similarity at the feature level. When features come from the embedding of a neural network model, since this embedding is the result of recognition training, it possesses semantic representation capabilities. Based on this, the above embodiments quantify the degree of confusion through semantic confusion measurement, thereby automatically generating a confused category set and producing more meaningful hard samples.

[0110] In some embodiments, clustering algorithms can be used to aggregate mutually confusing categories into a set of confusing categories. Clustering refers to the process of dividing a set into multiple classes composed of similar objects. Clustering algorithms can employ K-means or k-medoids algorithms. Clustering algorithms aggregate at least two target categories into a set of confusing categories.

[0111] In one embodiment, training an image recognition model to be trained based on a target positive sample image and a target negative sample image to obtain a trained image recognition model includes: inputting the target positive sample image into the image recognition model to be trained for feature extraction to obtain target positive sample features corresponding to the target positive sample image; inputting the target negative sample image into the image recognition model to be trained for feature extraction to obtain target negative sample features corresponding to the target negative sample image; determining a first sample feature distance between the target positive sample features and the target negative sample features; determining a target model loss value based on the first sample feature distance, wherein the target model loss value is negatively correlated with the first sample feature distance; and adjusting the model parameters of the image recognition model to be trained based on the target model loss value to obtain a trained image recognition model.

[0112] The target model loss value can be a numerical value corresponding to a pre-built loss function. A loss function is a function used to represent the "risk" or "loss" of an event.

[0113] Here, model parameters refer to the variable parameters inside the image recognition model. For neural network models, they can also be called neural network weights.

[0114] The trained image recognition model can be obtained through one or more training iterations. Specifically, the server can adjust the model parameters in the image recognition model to be trained in the direction of reducing the loss value, and obtain the trained image recognition model through one or more training iterations.

[0115] The sample feature distance can be Euclidean distance. In one embodiment, the Euclidean distance between the target positive sample features and the target negative sample features is determined to obtain the first sample feature distance. In another embodiment, a loss function can be constructed based on the calculation expression of the first sample feature distance, and then the model parameters of the image recognition model can be adjusted based on the loss value of the loss function.

[0116] In one embodiment, the target positive sample image and the target negative sample image can be constructed into a pair, and the pair is input into the image recognition model to be trained, thereby obtaining the trained image recognition model. In this case, the image recognition model performs simultaneous analysis on the target positive sample image and the target negative sample image within the pair, which can fully integrate the sample features of the target positive sample image and the target negative sample image, thereby obtaining a trained model that can accurately distinguish between the target positive sample image and the target negative sample image.

[0117] In one embodiment, adjusting the model parameters of the image recognition model to be trained based on the target model loss value to obtain the trained image recognition model includes: performing backpropagation based on the target model loss value, and updating the model parameters of the image recognition model along the gradient descent direction during the backpropagation process to obtain the trained image recognition model.

[0118] In this context, "backward propagation" refers to the fact that the parameter updates are in the opposite direction to the image recognition process. Since parameter updates propagate backward, the descent gradient can be obtained based on the target model's loss value. Starting from the last layer of the image recognition model, the model parameters are updated using the descent gradient until the first layer of the image recognition model is reached. Gradient descent methods can include stochastic gradient descent and batch gradient descent, among others. It can be understood that model training can be iterative; that is, the trained image recognition template can be obtained through iterative training. Training stops when the model convergence condition is met. The convergence condition can be that the model loss value is less than a preset loss value, or that the change in model parameters is less than a preset parameter change value.

[0119] In one embodiment, the model parameters of the image recognition model are solved using gradient descent based on SGD (Stochastic Gradient Descent): initial values ​​are set for parameters such as the convolution template parameter w and the bias parameter b in the model. During each iteration, the prediction error is calculated and backpropagated to the image recognition model, the gradient is calculated, and the parameters of the image recognition model are updated. Specifically, all parameters of the model are set to the state to be learned. During training, the image recognition model performs forward computation on an input image to obtain the prediction result. The prediction result is compared with the true result to calculate the model's loss value. The loss value is calculated using the cross-entropy loss function for classification. The loss value is backpropagated to the image recognition model to update the model parameters using stochastic gradient descent, thereby achieving one weight optimization. After multiple optimizations, a well-performing image recognition model is finally obtained. The initial learning rate of the image recognition model can be set to 0.01. After a certain number of iterations, the learning rate can be multiplied by a descent factor to reduce the learning rate, such as multiplying by 0.1 every 10 iterations.

[0120] In the above embodiments, positive and negative sample images are input into the image recognition model. The image recognition model extracts features to obtain corresponding sample features. The model loss value is determined based on the sample feature distance between the positive and negative sample images. This model loss value is negatively correlated with the sample feature distance. During model training, different model loss values ​​can be obtained by adjusting the model parameters. When the model loss value is minimized, the image recognition model training is considered complete, and the trained image recognition model is obtained. This trained image recognition model can maximize the distance between positive and negative sample features through the adjusted model parameters, thereby preventing the image recognition model from misidentifying the correct category as the category corresponding to the negative sample features during image classification, resulting in accurate image recognition results.

[0121] The number of positive sample images can be at least one, for example, it can be at least one of a target positive sample image and a reference positive sample image.

[0122] In one embodiment, the method further includes: selecting an image corresponding to a positive sample category as a reference positive sample image; determining the target model loss value based on a first sample feature distance includes: inputting the reference positive sample image into an image recognition model to be trained for feature extraction to obtain reference positive sample features corresponding to the reference positive sample image; determining a second sample feature distance between the target positive sample features and the reference positive sample features; determining a first model loss value based on the first sample feature distance, wherein the first model loss value is negatively correlated with the first sample feature distance; determining a second model loss value based on the second sample feature distance, wherein the second model loss value is positively correlated with the second sample feature distance; and obtaining the target model loss value based on the first model loss value and the second model loss value.

[0123] The target category corresponding to the reference positive sample image and the target positive sample image can be the same. In one embodiment, when the target positive sample image is obtained, a reference positive sample image is selected from the positive sample categories corresponding to the target positive sample image.

[0124] In one embodiment, the target positive sample image, the reference positive sample image, and the target negative sample image can be constructed into a triplet sample image (or simply a triplet). This triplet sample image is then input into the image recognition model to be trained, thereby obtaining the trained image recognition model. That is, in the confusion category set, a target positive sample image is determined from the target images corresponding to a certain category, and a reference positive sample image is determined from the target images corresponding to the same category. This can be achieved by randomly selecting a target image from the target images corresponding to the same category as the reference positive sample image. Similarly, a target negative sample image is determined from the target images corresponding to another category, and vice versa.

[0125] In one embodiment, a first loss function can be constructed based on the calculation expression of the first sample feature distance, and a second loss function can be constructed based on the calculation expression of the second sample feature distance. Then, the model parameters of the image recognition model can be adjusted based on the sum of the loss values ​​of the first and second loss functions.

[0126] In one embodiment, the number of triples can be at least one. If there are multiple triples, after determining the target positive sample image, reference positive sample images can be determined multiple times from target images corresponding to the same category, and target negative sample images can be determined multiple times from target images corresponding to another category. Each time, the obtained reference positive sample image, the target negative sample image, and the target positive sample image are combined to form a triple, thus obtaining multiple triples.

[0127] In one embodiment, the process of determining triples can be as follows:

[0128] For each category in the set of obfuscated categories:

[0129] For each target positive sample image (anchor, abbreviated as a):

[0130] a. Randomly select target images of the same category as reference positive sample images (p);

[0131] b. Randomly select target images from different categories within the same category set as target negative sample images (n);

[0132] c. Form a triple (a, p, n);

[0133] d. Execute the above ac process a total of 5 times, generating 5 triples for sample a. In other embodiments, the number of times ac is executed can be less or more.

[0134] These five triples can then be input into the image recognition model to be trained to obtain the trained image recognition model.

[0135] In one embodiment, the loss function can be constructed based on Triplet-loss. Triplet-loss is a metric for recognizing models in machine learning. For a given input sample, it first obtains the feature distance between the sample and a sample of the same category (i.e., the positive sample feature distance), and the feature distance between the sample and a sample of a different category (i.e., the negative sample feature distance). The difference between the negative sample feature distance and the positive sample feature distance is used as the error value for the input sample.

[0136] In one embodiment, the loss function constructed from the triplet sample images of the target positive sample image, the reference positive sample image, and the target negative sample image can be expressed as:

[0137] L=max(d(a,n)-d(a,p)-margin,0)

[0138] Here, `margin` is a pre-defined distance, the size of which can be determined based on actual conditions, and can be 4. The purpose of setting the margin is to increase the sample feature distance between a and n, and decrease the sample feature distance between a and p. `d(a,n)` represents the first sample feature distance between a and n, and `d(a,p)` represents the second sample feature distance between a and p. The loss value is the maximum value among `d(a,n) - d(a,p) - margin` and 0. This setting avoids negative loss values, thus preventing network non-convergence caused by negative loss values. Furthermore, when `margin` is 4, the above formula means: (the second sample feature distance between a and p + 4) is less than (the first sample feature distance between a and n).

[0139] In the above embodiments, the model loss value is determined based on the sample feature distance between the target positive sample features and the reference positive sample features, as well as the sample feature distance between the target positive sample features and the target negative sample features, thereby obtaining the trained image recognition model. This trained image recognition model can maximize the sample feature distance between the target positive sample features and the target negative sample features, and minimize the sample feature distance between the reference positive sample features and the target positive sample features, through adjusted model parameters. This ensures that the image recognition model will not misidentify the correct category as the category corresponding to the negative sample features during image classification, resulting in accurate image recognition results.

[0140] In one embodiment, such as Figure 5 The diagram shown is a flowchart illustrating the implementation of the image recognition method. It can be understood that this method can also be applied to terminals or servers, and can also be applied to systems including both terminals and servers, implemented through interaction between the terminals and servers. The following is a specific example of applying this method to a server:

[0141] S502, acquire the target image.

[0142] S504, Obtain the target category corresponding to the target image. Perform image recognition on the target image to obtain the target category corresponding to each target image.

[0143] S506, Perform confusion classification on the target category to obtain a set of confusion categories. Determine the confusion degree between target categories, and perform confusion classification on the target categories based on the confusion degree to obtain a set of confusion categories.

[0144] S508, Obtain triplet sample images from the confusion category set. When the target positive sample image is obtained, obtain the target negative sample image and the reference positive sample image corresponding to the target positive sample image from the confusion category set to construct triplet sample images.

[0145] S510, Training the Image Recognition Model. The image recognition model is trained based on triplet sample images, thus obtaining the trained image recognition model.

[0146] The above embodiments can train an image recognition model based on the differences in image features between positive and negative sample images, resulting in an image recognition model that can accurately classify images.

[0147] In one embodiment, besides training the image recognition model based on triplet sample images, the target image can also be combined with the image recognition model for training. Here, the triplet sample images serve as one training task, and the target image serves as another training task. These two training tasks are combined to train the image recognition model, achieving multi-task learning of the image recognition model. Figure 6 As shown, in Figure 5 Based on the corresponding embodiment, the triplet sample image and the target image are input together as training samples into the image recognition model to be trained, and the model is trained to obtain the trained image recognition model.

[0148] In the above embodiments, sample images are extracted based on a set of confused categories, and then triples are constructed. The process of determining triples does not require manual annotation, thus eliminating the need for additional manual annotation costs in multi-task learning. Furthermore, compared to traditional image recognition model training processes, the above embodiments integrate triple samples, enabling the trained image recognition model to achieve higher image recognition accuracy. Therefore, the above embodiments can effectively improve image recognition accuracy without increasing additional annotation costs.

[0149] like Figure 7 As shown, an image recognition method is provided. It can be understood that this method can be applied to a terminal or a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. Taking the application of this method to a server as an example, it includes the following steps:

[0150] S702, acquire the target image.

[0151] S704, perform depth feature extraction on the target image. Perform image recognition on the target image to obtain the target category corresponding to each target image, and extract the depth features of the target image to obtain the depth feature embedding.

[0152] S706, The embedding is analyzed by the classification layer to obtain the classification result.

[0153] S708, Adjust model parameters based on the loss value. Compare the classification result with the image annotation category, and adjust the model parameters based on the comparison result to minimize the loss value corresponding to the loss function.

[0154] S710, Determine the set of confusing categories based on the classification results. Determine the class confusion degree between target categories, and perform confusion classification on the target categories based on the class confusion degree to obtain the set of confusing categories.

[0155] S712, Obtain triplet sample images from the confusion category set. When the target positive sample image is obtained, obtain the target negative sample image and the reference positive sample image from the confusion category set, and construct triplet sample images based on the target positive sample image, the reference positive sample image, and the target negative sample image.

[0156] S714, construct training samples from the target image and triplet sample images. Input the target image and triplet sample images together as training samples into the image recognition model to be trained.

[0157] S716, performs depth feature extraction on the training samples. During training, the image recognition model extracts depth features from the target image and the triplet sample images to obtain the depth feature embedding.

[0158] S718, The embedding is analyzed by the classification layer to obtain the classification result.

[0159] S720: Adjust the model parameters based on the loss value to obtain the trained image recognition model. Compare the classification results with the image annotation categories, and adjust the model parameters based on the comparison results to minimize the loss value corresponding to the loss function, thus obtaining the trained image recognition model. The loss value can be the sum of the following two components: 1. The loss value obtained from metric learning of the triples; 2. The loss value obtained from classification analysis of the target image.

[0160] In one embodiment, such as Figure 7 As shown, after deep feature extraction to obtain the embedding, the server can process the embedding through two branches: one is metric learning to calculate the loss value of the triplet sample image, and then adjust the model parameters according to the loss value (i.e., S708); the other is inputting the classification layer Fc_cr, which analyzes the embedding to obtain the classification result (i.e., S716).

[0161] In one embodiment, scene recognition is performed on an image set using a trained scene recognition model, and the trained scene recognition model is used as the training model to be trained for image category confusion training. After training the image recognition model to be trained based on the target positive sample image and the target negative sample image, the method further includes: performing image category confusion training on the trained scene recognition model based on the target positive sample image and the target negative sample image until the confusion training result converges, thereby obtaining the confusion-trained scene recognition model. The method further includes: obtaining the current recognition image of the scene to be recognized; inputting the current recognition image into the confusion-trained scene recognition model for scene recognition, thereby obtaining the current image scene corresponding to the current recognition image.

[0162] The trained scene recognition model can be a scene recognition model trained based on traditional methods, a scene recognition model based on triplet sample images, or a scene recognition model trained based on multi-task learning.

[0163] In this embodiment, a scene recognition model is used to identify scenes in images to obtain the target category corresponding to each image. The server performs confusion classification based on the target category to obtain a confusion category set. Target positive sample images and target negative sample images are then obtained from the confusion category set, and the image recognition model is trained based on these images. The trained image recognition model can accurately classify input images, and different categories can correspond to different scenes; therefore, the image recognition model can also achieve accurate scene recognition. After training the image recognition model, the scene recognition model can be further trained to integrate information from the confusion category set into its model parameters. This allows the retrained scene recognition model to accurately distinguish between confused categories, further improving the accuracy of scene recognition.

[0164] Furthermore, the trained image recognition model is used as the standard category for the target positive sample image and the target negative sample image. Then, the trained scene recognition model is retrained based on the target positive sample image and the target negative sample image. This allows the retrained scene recognition model to form sample pairs based on the target positive sample image and the target negative sample image, and to mark similar sample images as easily confused categories according to the sample pairs. This enables the retrained scene recognition model to more accurately recognize images in similar scenes, such as the target positive sample image and the target negative sample image, thereby improving the specificity and accuracy of the scene recognition model for images in specific scenes.

[0165] In the above embodiments, the scene recognition model can perform category recognition on the image set to obtain category recognition data, perform confusion classification on the category set corresponding to the image set based on the category recognition data to obtain a confusion category set, and then obtain target positive and negative sample images based on the confusion category set. The image recognition model is trained using the target positive and negative sample images to obtain the trained image recognition model. After the image recognition model is trained, the scene recognition model can be further trained so that the scene recognition model can better recognize easily confused scenes.

[0166] Under the provided training method, the embodiments of this application have at least the following beneficial effects: 1) It can improve the distinguishability between confused categories, make the features within the category more aggregated, and make the distinguishability between categories more obvious; 2) It can achieve confusion attribute analysis and obtain the target category through the network pre-training results, which does not require additional annotation compared with traditional methods; 3) It can improve the overall scene recognition effect without increasing the network depth, and can achieve better recognition accuracy on the same basic model structure; 4) Confused categories and difficult samples are obtained through algorithms, realizing automated training.

[0167] This application also provides an application scenario in which the above-described image recognition method is applied. Specifically, the image recognition method is applied in this scenario as follows:

[0168] like Figure 8 As shown, terminal 8002 captures video through a camera, splits the captured video into a set of images, and transmits the set of images as images to be classified to server 8004.

[0169] Server 8004 is configured with a trained image recognition model. The training process of this image recognition model is as follows: triple sample images are obtained from the confusion category set, and the triple sample images and the input image are used together as training samples to train the image recognition model to be trained, thus obtaining the trained image recognition model.

[0170] Server 8004 performs image recognition on each image to be classified to determine its scene category. After determining the scene recognition result, it can return it to terminal 8002 or transmit it to another terminal 8006.

[0171] In the above embodiments, when the server obtains an image to be classified, it performs scene recognition on the image to be classified based on a trained image recognition model. The trained image recognition model can accurately distinguish the image to be classified from similar scenes, thereby obtaining accurate scene recognition results.

[0172] This application also provides an application scenario in which the above-described image recognition method is applied. Specifically, the image recognition method is applied in this scenario as follows:

[0173] Image recognition methods are applied to systems consisting of the following modules: a model pre-training module, a confusion category analysis module, a module for generating triplet sample images based on the confusion category set, and a multi-task classification learning module. In other words, the image recognition method includes the following implementation steps: 1. Model pre-training; 2. Confusion category analysis; 3. Generating triplet sample images based on the confusion category set; 4. Multi-task classification learning. The following section provides a detailed explanation of these steps:

[0174] I. Model Pre-training

[0175] 1. Model Building

[0176] A ResNet 101 neural network model is constructed as a scene recognition model. The multi-layer structure of this model mainly consists of deep learning operations such as convolution, normalization, pooling, and cross-layer connections in multiple modules.

[0177] 2. Training

[0178] The model parameters of the neural network model are solved using gradient descent based on SGD. In each iteration, the prediction error is calculated and backpropagated to the neural network model, the gradient is calculated, and the parameters of the neural network model are updated. When the loss value meets the condition, the scene recognition model is considered to have converged, and the trained scene recognition model is obtained.

[0179] II. Confusion Category Analysis

[0180] After the above model training steps, a trained scene recognition model capable of scene category identification for images is obtained. Based on this model, the scene of each image is analyzed, and the category of each image is output. According to certain clustering rules, all categories are aggregated into easily confused categories, resulting in a set of confused categories. Finally, we obtain confused category set 1, confused category set 2, ..., confused category set m. The size of m is determined based on the number of categories and the aggregation pattern.

[0181] Aggregating categories into a confused category set can be achieved in at least two of the following ways:

[0182] 1. Obtain a set of confused categories by aggregating deep feature embeddings.

[0183] 1) Obtain the embeddings obtained by the scene recognition model from the deep feature extraction of each image. Calculate the center of each category, which is represented by Center. Center is an N*2048 vector, where N represents the number of categories. The center of each category is a 1*2048 vector, i.e. Center = Center[1], Center[2], ..., Center[i], ..., Center[N].

[0184] Taking category i as an example, the process of determining Center[i] is explained (the process of determining the center of other categories is similar and will not be repeated here): For each sample (i.e., image), the depth feature embedding of the sample is obtained. The embedding is a feature vector with a vector dimension of 1*2048, which is used as the feature for image extraction. The average of all embeddings of category i is calculated to obtain the average vector. The average vector is used as the category center Center[i] of category i.

[0185] 2) Calculate the category confusion matrix S. The category confusion matrix S is an N*N matrix, and all Sij values ​​in the category confusion matrix S are initialized to 0. Sij in the category confusion matrix S represents the category confusion degree between the category centers of category i and category j. The process of updating the category confusion degree in the category confusion matrix S is as follows:

[0186] a. Calculate the class confusion between categories and update the class confusion matrix S:

[0187] For each sample j in category i:

[0188] Calculate the feature similarity between the embedding of sample j and the category centers of the other N-1 categories. This similarity can be cosine similarity; a larger value indicates a greater similarity between the sample and the corresponding category. Find the center with the highest similarity; this center represents the category most similar to sample j, and we denote this category as z. Determine if the similar category z is category i of sample j. If not, increment the category confusion between category i and category z by 1, i.e., Siz = Siz + 1, Szi = Szi + 1. Here, Siz represents the category confusion between category i and category z, and Szi represents the category confusion between category z and category i. Their counts should be equal, so both values ​​need to be updated simultaneously with each update.

[0189] The class confusion is calculated for all samples in all categories using the method described above. The latest class confusion matrix S can be obtained based on the final class confusion between each category.

[0190] b. Normalization of the category confusion matrix:

[0191] Divide each value Sij in the class confusion matrix S by the smaller of Num_i and Num_j, i.e., Tij = Sij / min(Num_i, Num_j), where Num_i represents the number of items in class i and Num_j represents the number of items in class j. Obtain the normalized class confusion level based on the calculation result, and then update the class confusion matrix S.

[0192] Obtain the normalized class confusion scores Tij, which contain N*(N-1) / 2 class confusion scores. Since Tij = Tji, this step takes the part where i > j to reduce computation. Two classes with confusion scores exceeding a specified threshold (e.g., 0.8) are considered confusion classes. For example, T13 is 0.9, indicating that classes 1 and 3 are easily confused; T34 is 0.85, indicating that classes 3 and 4 are easily confused; T25 is 0.82, indicating that classes 2 and 5 are easily confused; other classes with a confusion score less than 0.8 are not easily confused. Confusion classes with the same class are merged to obtain multiple confusion class sets. Since classes 1 and 3 are easily confused, and classes 3 and 4 are easily confused, and both confusion pairs contain class 3, merging them results in 134 confusion pairs, forming one confusion class set. After merging the confusion pairs, samples from classes 1, 3, and 4 are considered difficult negative samples; for example, samples from class 1 are all difficult negative samples from class 4. After the above processing, we obtain the confusion category set 1, which consists of categories 1, 3 and 4; and the confusion category set 2, which consists of categories 2 and 5.

[0193] 2. Aggregate the classification results to obtain a set of confused categories.

[0194] 1) Obtain the classification results of each sample obtained by the scene recognition model based on the embedding, so as to obtain the recognition category corresponding to each sample.

[0195] 2) Calculate the category confusion matrix S. The category confusion matrix S is an N*N matrix. The process of updating the category confusion values ​​in the category confusion matrix S is as follows:

[0196] a) Compare the identified category of each sample with the labeled category of the corresponding sample. If the two are inconsistent, increment the category confusion between the identified category and the labeled category by 1.

[0197] b. Normalization of the category confusion matrix:

[0198] Divide each value Sij in the category confusion matrix S by the smaller of Num_i and Num_j to obtain the normalized category confusion score. Obtain N*(N-1) / 2 category confusion scores Tij, where i>j. Select two categories with a confusion score exceeding a specified threshold (e.g., 0.8) as the confused categories. Merge confused categories with the same category to obtain multiple sets of confused categories.

[0199] III. Generating Triple Sample Images Based on Confusion Category Sets

[0200] For each sample (a) corresponding to each category in each set of confused categories:

[0201] a. Randomly select samples (p) of the same scene category as positive samples;

[0202] b. Randomly select samples (n) from different scenario categories within the same confusion category set as negative samples;

[0203] c. Form a triple (apn)

[0204] d. Execute the above ac five times in total, generating five triples for sample a.

[0205] IV. Multi-task classification learning

[0206] The ResNet 101 neural network model is used for multi-task learning, that is, the ResNet 101 neural network model is used to construct the image recognition model.

[0207] The triplet-loss metric, calculated from triplet sample images, is added to the original classification loss function. Model parameters are then adjusted based on these two loss values ​​to obtain a trained image recognition model. Specifically, for each sample, the classification loss is calculated using the original classification loss function. If the sample has triplet sample images, the triplet loss is calculated. The sum of the classification loss and the triplet loss is used as the total loss. When the convergence of the ResNet 101 neural network model is determined based on the total loss, this model is considered the trained image recognition model.

[0208] A trained image recognition model is used to classify and identify the input image, thereby achieving scene recognition. The trained image recognition model can accurately distinguish the input image from easily confused category images, thus obtaining accurate scene recognition results.

[0209] The features between confused categories are concentrated in subtle details, and conventional deep learning classification and recognition methods tend to overlook these subtle differences, making them indistinguishable. Therefore, it is necessary to design additional auxiliary recognition tasks to help effectively distinguish the features between confused categories.

[0210] The embodiments of this application have at least the following technical effects:

[0211] 1) Deep learning improves the performance of confusing scene recognition. A better-performing model can be obtained through a multi-task learning framework without increasing the amount of annotations.

[0212] 2) The class recognition results generated by the basic trained model can aggregate similar classes into a superclass. By leveraging the superclass's characteristic of intra-class similarity and inter-class dissimilarity, hard samples among similar classes can be mined. After mining hard samples in the superclass, metric learning is performed. Specifically, for a given sample, samples of the same class are considered positive samples, while samples from other classes within the same superclass are considered negative samples. Metric learning is introduced to distinguish subtle superclass confusion, thus correcting the selection of negative samples from the conventional full class selection to selection from different classes within the same superclass, improving the effectiveness of negative samples. Using the positive and negative samples obtained through this processing to train the image recognition model yields an image recognition model that fully exploits the distinction between hard and positive samples, enabling accurate image recognition.

[0213] 3) By analyzing easily confused categories and conducting targeted metric learning, ineffective metric learning was avoided and learning efficiency was improved.

[0214] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0215] Based on the same idea as the image recognition method in the above embodiments, this application also provides an image recognition apparatus that can be used to perform the above image recognition method. For ease of explanation, the structural schematic diagram of the image recognition apparatus embodiment only shows the parts related to the embodiments of this application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the apparatus, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0216] In one embodiment, such as Figure 9 As shown, an image recognition device 900 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a positive sample image acquisition module 902, a confusion category set acquisition module 904, a negative sample image acquisition module 906, and a model training module 908, wherein:

[0217] Positive sample image acquisition module 902 is used to acquire target positive sample images.

[0218] The confusion category set acquisition module 904 is used to acquire the confusion category set corresponding to the positive sample category according to the positive sample category corresponding to the target positive sample image; wherein the confusion category set is obtained by performing category recognition on the image set and performing confusion classification on the category set corresponding to the image set based on the category recognition data.

[0219] The negative sample image acquisition module 906 is used to take the confusion category in the confusion category set as the negative sample category, and take the image corresponding to the negative sample category as the target negative sample image corresponding to the target positive sample image.

[0220] The model training module 908 is used to train the image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model, and to perform image recognition based on the trained image recognition model.

[0221] In the aforementioned image recognition device, the model training process can fully analyze the differences between positive and negative sample images in the same confused category set, thereby improving the ability of the trained model to recognize different images under similar categories. Therefore, it can obtain an image recognition model that can accurately identify images that are easily confused by category, thus improving the accuracy of image recognition.

[0222] In one embodiment, the confusion category set acquisition module includes: an image set acquisition submodule, used to acquire a target image set; the target image set includes at least two target images, and the target categories corresponding to each target image form a category set; an identification data determination submodule, used to perform image recognition on each target image in the target image set to obtain category recognition data corresponding to each target image; a confusion degree determination submodule, used to determine the category confusion degree between the target categories based on the category recognition data; and a confusion category set determination submodule, used to perform confusion classification on the target categories in the category set based on the category confusion degree to obtain a confusion category set.

[0223] The above embodiments obtain category recognition data by performing image recognition on target images in a target image set. Based on the category recognition data, the category confusion degree between target categories is obtained. Then, based on the category confusion degree, the target categories are classified into a confusion category set, grouping easily confused target categories into a single confusion category set. This ensures that similar categories exist within the same confusion category set, while dissimilar categories exist between different confusion category sets. Therefore, positive and negative sample images within the same confusion category set have high similarity, and negative sample images can serve as hard samples for positive sample images, thus enabling the training of a reliable image recognition model.

[0224] In one embodiment, the category identification data includes image extraction features obtained by feature extraction of the target image; the confusion determination submodule includes: a similarity determination unit, used to obtain the feature similarity between the target categories based on the image extraction features; and a confusion determination unit, used to determine the category confusion between the target categories based on the feature similarity between the target categories.

[0225] The above embodiments calculate the feature similarity between target categories by extracting features from images, and then determine the category confusion between target categories. The determined category confusion fully integrates the extracted image features and has high accuracy.

[0226] In one embodiment, the similarity determination unit includes: a category feature determination subunit, used to extract features from images corresponding to different target categories and perform statistics to obtain category representation features corresponding to each target category; and a similarity determination subunit, used to obtain feature similarity between target categories based on the category representation features corresponding to each target category.

[0227] The above embodiments calculate the feature similarity between target categories based on image-extracted features. Image-extracted features can often be represented in the form of vectors, and the computational cost of vector operations is relatively small. Therefore, the computational efficiency of feature similarity can be effectively improved, thereby improving the efficiency of model training.

[0228] In one embodiment, the similarity determination subunit includes: an image feature acquisition block, used to acquire current image extracted features corresponding to the current target image, wherein the current target image belongs to a first target category; and a similarity determination block, used to calculate the feature similarity between the current image extracted features and the category representation features corresponding to each second target category, and use the calculated feature similarity as the feature similarity between the first target category and the second target category.

[0229] The above embodiments calculate feature similarity based on the image extraction features of the target image and the category representation features of a specific target category. The feature similarity between target categories can be obtained through simple vector calculation, which can effectively improve the calculation efficiency of feature similarity and thus improve the efficiency of model training.

[0230] In one embodiment, the confusion determination unit is further configured to increase the category confusion between the first target category and the second target category when the feature similarity between the first target category and the second target category meets a similarity condition; wherein the similarity condition includes at least one of the following: the feature similarity is ranked before a first ranking threshold or the feature similarity is greater than a similarity threshold.

[0231] In the above embodiments, feature similarity of target categories is obtained based on image extraction features of target images, and category confusion is determined based on feature similarity between target categories. Accurate and reliable category confusion can be obtained through simple calculations.

[0232] In one embodiment, the category identification data includes a third target category obtained by classifying the target image; the confusion determination submodule includes: a category comparison unit, used to compare the third target category with a standard category corresponding to the target image; the standard category is a fourth target category; and a confusion increase unit, used to increase the category confusion between the third target category and the fourth target category when the comparison between the third target category and the standard category corresponding to the target image is inconsistent.

[0233] In the above embodiments, when the classification result obtained from classifying the target image is inconsistent with the standard category, it is determined that the category corresponding to the classification result and the standard category are easily confused, and then the category confusion degree between the two is increased. The process of determining the category confusion degree is simple and can effectively ensure the efficiency of determining the category confusion degree.

[0234] In one embodiment, the confusion category set determination submodule is further configured to identify target categories in the category set whose category confusion levels meet the confusion level conditions as mutually confused categories, and to form a confusion category set of mutually confused categories; wherein, the confusion level conditions include at least one of the following: the category confusion level is ranked before a second ranking threshold or the category confusion level is greater than the confusion level threshold.

[0235] In the above embodiments, feature similarity of target categories is obtained based on image extraction features of target images, and category confusion is determined based on feature similarity between target categories. Accurate and reliable category confusion can be obtained through simple calculations.

[0236] In one embodiment, the model training module includes: a positive sample feature extraction submodule, used to input the target positive sample image into the image recognition model to be trained for feature extraction, to obtain target positive sample features corresponding to the target positive sample image; a negative sample feature extraction submodule, used to input the target negative sample image into the image recognition model to be trained for feature extraction, to obtain target negative sample features corresponding to the target negative sample image; a feature distance determination submodule, used to determine a first sample feature distance between the target positive sample features and the target negative sample features; a loss value determination submodule, used to determine a target model loss value based on the first sample feature distance, wherein the target model loss value is negatively correlated with the first sample feature distance; and a model parameter adjustment submodule, used to adjust the model parameters of the image recognition model to be trained based on the target model loss value, to obtain a trained image recognition model.

[0237] In the above embodiment, positive and negative sample images are input into the image recognition model. The image recognition model extracts features to obtain corresponding sample features. The model loss value is determined based on the sample feature distance between the positive and negative sample images. This model loss value is negatively correlated with the sample feature distance. During model training, different model loss values ​​can be obtained by adjusting the model parameters. When the model loss value is minimized, the image recognition model training is considered complete, and the trained image recognition model is obtained. This trained image recognition model can maximize the distance between positive and negative sample features through the adjusted model parameters, thereby preventing the image recognition model from misidentifying the correct category as the category corresponding to the negative sample features during image classification, resulting in accurate image recognition results.

[0238] In one embodiment, the apparatus further includes: a reference positive sample selection module, configured to select an image corresponding to the positive sample category as a reference positive sample image; a loss value determination submodule including: a feature extraction unit, configured to input the reference positive sample image into an image recognition model to be trained for feature extraction to obtain reference positive sample features corresponding to the reference positive sample image; a feature distance determination unit, configured to determine a second sample feature distance between the target positive sample feature and the reference positive sample feature; a first loss value determination unit, configured to determine a first model loss value based on the first sample feature distance, wherein the first model loss value is negatively correlated with the first sample feature distance; a second loss value determination unit, configured to determine a second model loss value based on the second sample feature distance, wherein the second model loss value is positively correlated with the second sample feature distance; and a target loss value determination unit, configured to obtain a target model loss value based on the first model loss value and the second model loss value.

[0239] The above embodiments determine the model loss value based on the sample feature distance between the target positive sample features and the reference positive sample features, as well as the sample feature distance between the target positive sample features and the target negative sample features, thereby obtaining the trained image recognition model. This trained image recognition model can maximize the sample feature distance between the target positive sample features and the target negative sample features, and minimize the sample feature distance between the reference positive sample features and the target positive sample features, through adjusted model parameters. This ensures that the image recognition model will not misidentify the correct category as the category corresponding to the negative sample features during image classification, resulting in accurate image recognition results.

[0240] In one embodiment, scene recognition is performed on the image set using a trained scene recognition model, and the trained scene recognition model is used as a training model to be trained for image category confusion training. The model training module is further configured to perform image category confusion training on the trained scene recognition model based on the target positive sample image and the target negative sample image until the confusion training result converges, thereby obtaining the confusion-trained scene recognition model. The device further includes: a recognition image acquisition module, configured to acquire the current recognition image of the scene to be recognized; and a scene recognition module, configured to input the current recognition image into the confusion-trained scene recognition model for scene recognition, thereby obtaining the current image scene corresponding to the current recognition image.

[0241] In the above embodiments, after the image recognition model is trained, the scene recognition model can be further trained so that the scene recognition model can better recognize easily confused scenes.

[0242] For specific limitations regarding the image recognition device, please refer to the limitations of the image recognition method above, which will not be repeated here. Each module in the aforementioned image recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0243] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as category recognition data, confusion category sets, and image recognition models. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image recognition method.

[0244] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0245] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0246] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0247] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0248] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0249] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0250] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image recognition method, characterized in that, The method includes: Acquire category recognition data corresponding to at least two target images; the category recognition data includes image extraction features obtained by feature extraction of the target images, and the target images correspond to target categories. Determining the category confusion between target categories based on the category recognition data includes: statistically analyzing the image extraction features corresponding to different target categories to obtain the category representation features corresponding to each target category; for any target category, calculating the similarity between the image extraction features of all target images corresponding to the target category and the category representation features of other target categories to obtain the similarity calculation results; statistically analyzing the similarity calculation results of each target image to obtain the feature similarity between the target category and other target categories; when the feature similarity between target categories does not meet the similarity condition, obtaining the category confusion between target categories based on a preset value; when the feature similarity between target categories meets the similarity condition, obtaining the category confusion between target categories based on the feature similarity between target categories; the category confusion between target categories is positively correlated with the feature similarity between target categories. Target categories whose category confusion meets the confusion condition are considered as mutually confused categories, and mutually confused categories are grouped into a confused category set; the confusion condition includes category confusion being greater than a confusion threshold, which is calculated based on the overall category confusion statistics of the target categories; Obtain the target positive sample image; Based on the positive sample category corresponding to the target positive sample image, obtain the set of confusion categories corresponding to the positive sample category; The confusion category in the confusion category set corresponding to the positive sample category is taken as the negative sample category, and the image corresponding to the negative sample category is taken as the target negative sample image corresponding to the target positive sample image; The image recognition model to be trained is trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model, and then image recognition is performed based on the trained image recognition model.

2. The method according to claim 1, characterized in that, For any given target category, the image extraction features of all target images corresponding to that target category are used to calculate the similarity between the extracted features and the category representation features of other target categories, resulting in the following similarity calculation results: Obtain the current image extraction features corresponding to the current target image, wherein the current target image belongs to the first target category; Calculate the feature similarity between the extracted features of the current image and the category representation features corresponding to each second target category, and use the calculated feature similarity as the feature similarity between the first target category and the second target category.

3. The method according to claim 2, characterized in that, When the feature similarity between target categories meets the similarity condition, the class confusion degree between target categories is obtained based on the feature similarity between target categories, including: When the feature similarity between the first target category and the second target category meets the similarity condition, the category confusion between the first target category and the second target category is increased; The similarity conditions include at least one of the following: the feature similarity is ranked before a first ranking threshold or the feature similarity is greater than a similarity threshold.

4. The method according to claim 1, characterized in that, The category recognition data includes a third target category obtained by classifying the target image; determining the category confusion between the target categories based on the category recognition data includes: The third target category is compared with the standard category corresponding to the target image; the standard category is the fourth target category. When the third target category is inconsistent with the standard category corresponding to the target image, the category confusion between the third target category and the fourth target category is increased.

5. The method according to claim 1, characterized in that, The confusion condition also includes the ranking of category confusion before the second ranking threshold.

6. The method according to any one of claims 1 to 5, characterized in that, The process of training the image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model includes: The target positive sample image is input into the image recognition model to be trained for feature extraction to obtain the target positive sample features corresponding to the target positive sample image; The target negative sample image is input into the image recognition model to be trained for feature extraction to obtain the target negative sample features corresponding to the target negative sample image; Determine the first sample feature distance between the target positive sample features and the target negative sample features; The target model loss value is determined based on the first sample feature distance, wherein the target model loss value is negatively correlated with the first sample feature distance; The model parameters of the image recognition model to be trained are adjusted based on the target model loss value to obtain the trained image recognition model.

7. The method according to claim 6, characterized in that, The method further includes: Select the image corresponding to the positive sample category as the reference positive sample image; The step of determining the target model loss value based on the feature distance of the first sample includes: The reference positive sample image is input into the image recognition model to be trained for feature extraction to obtain the reference positive sample features corresponding to the reference positive sample image. Determine the second sample feature distance between the target positive sample features and the reference positive sample features; A first model loss value is determined based on the first sample feature distance, wherein the first model loss value is negatively correlated with the first sample feature distance; The second model loss value is determined based on the second sample feature distance, wherein the second model loss value is positively correlated with the second sample feature distance; The target model loss value is obtained based on the first model loss value and the second model loss value.

8. The method according to any one of claims 1 to 5, characterized in that, Scene recognition is performed on the image set using a trained scene recognition model, and the trained scene recognition model is used as the training model to be trained for image category confusion training. After training the image recognition model to be trained based on the target positive sample image and the target negative sample image, the process further includes: Based on the target positive sample image and the target negative sample image, the trained scene recognition model is trained by image category confusion until the confusion training result converges, and the scene recognition model after confusion training is obtained. The method further includes: Obtain the current recognition image of the scene to be recognized; The current image is input into the scene recognition model after confusion training to perform scene recognition, thereby obtaining the current image scene corresponding to the current image.

9. The method according to any one of claims 1 to 5, characterized in that, The image recognition model to be trained is a trained scene recognition model, and the category recognition data is obtained by performing image recognition on the target image through the trained scene recognition model.

10. The method according to claim 9, characterized in that, The step of training the image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model includes: Select the image corresponding to the positive sample category as the reference positive sample image; The target positive sample image, the target negative sample image, and the reference positive sample image are input into the image recognition model to be trained for feature extraction to obtain the sample features corresponding to the target positive sample image, the target negative sample image, and the reference positive sample image, respectively. Based on the sample features corresponding to the target positive sample image, the target negative sample image, and the reference positive sample image, the target model loss value is obtained, and based on the sample features and target category corresponding to the target positive sample image, the target negative sample image, and the reference positive sample image, the classification loss value is obtained. The model parameters of the image recognition model to be trained are adjusted based on the target model loss value and the classification loss value to obtain the trained image recognition model.

11. The method according to claim 9, characterized in that, The trained scene recognition model is obtained by training the scene recognition model based on the target image and the target category corresponding to the target image.

12. An image recognition device, characterized in that, The device includes: A confusion category set acquisition module is used to acquire category recognition data corresponding to at least two target images respectively; the category recognition data includes image extraction features obtained by feature extraction of the target images, and the target images correspond to target categories; determining the category confusion degree between the target categories based on the category recognition data includes: statistically analyzing the image extraction features corresponding to different target categories to obtain category representation features corresponding to each target category; for any target category, calculating the similarity between the image extraction features of all target images corresponding to the target category and the category representation features of other target categories respectively, and obtaining the similarity calculation results; statistically analyzing the similarity calculation results of each target image to obtain the feature similarity between the target category and other target categories; when the feature similarity between target categories does not meet the similarity condition, obtaining the category confusion degree between target categories based on a preset value; when the feature similarity between target categories meets the similarity condition, obtaining the category confusion degree between target categories based on the feature similarity between target categories; the category confusion degree between target categories is positively correlated with the feature similarity between target categories. The module for obtaining the confusion category set is further configured to identify target categories whose confusion levels meet the confusion level conditions as mutually confused categories, and to form a confusion category set by combining the mutually confused categories; the confusion level conditions include that the category confusion level is greater than the confusion level threshold, and the confusion level threshold is calculated based on the overall category confusion level statistics of the target categories; Positive sample image acquisition module, used to acquire target positive sample images; The obfuscation category set acquisition module is further configured to acquire the obfuscation category set corresponding to the positive sample category based on the positive sample category corresponding to the target positive sample image; The negative sample image acquisition module is used to take the confusion category in the confusion category set corresponding to the positive sample category as the negative sample category, and take the image corresponding to the negative sample category as the target negative sample image corresponding to the target positive sample image; The model training module is used to train the image recognition model to be trained based on the target positive sample image and the target negative sample image to obtain the trained image recognition model, and to perform image recognition based on the trained image recognition model.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images