Image classification method, apparatus, computer device, and storage medium

Abstract

The application relates to an image classification method and device, computer equipment and a storage medium. The method comprises the following steps: determining a predicted score of each sample image belonging to a target scene graph class in a sample image set; selecting a candidate sample image with a predicted score reaching a score threshold in the sample image set; determining the similarity between each candidate sample image and a difficult sample image; screening the candidate sample images according to the similarity to obtain positive sample images and negative sample images; training an image classification model based on the positive sample images, the difficult sample images and the negative sample images until the training is stopped when a target loss value of the image classification model reaches a preset loss value, obtaining a trained image classification model, and the target loss value comprises a classification loss value, a first loss value and a second loss value; and when a to-be-classified image is acquired, classifying the to-be-classified image through the trained image classification model to obtain a classification result. The method can improve the classification ability of the image classification model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image classification method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, machine learning technology has emerged. Machine learning can be used to train machine learning models for various tasks. For example, an image classification model can be trained for image classification tasks. This image classification model can extract the features corresponding to the image and classify the image based on the features to obtain the image classification result.

[0003] Most image classification models are currently trained in advance through supervised learning. That is, the image classification model classifies the samples, obtains the classification results, and adjusts the model parameters based on the classification results to make the classification results output by the image classification model closer to the true category of the sample.

[0004] However, the image classification model obtained by the above training method is prone to overfitting, which makes it impossible for the image classification model to accurately classify certain images. For example, in the process of classifying restricted scene images, it is easy to misclassify normal images that are highly suspected to be restricted scene images as restricted scene images. Summary of the Invention

[0005] Therefore, it is necessary to provide an image classification method, apparatus, computer equipment, and storage medium that can improve the accuracy of image classification in response to the above-mentioned technical problems.

[0006] An image classification method, the method comprising:

[0007] Determine the predicted score of each sample image in the sample image set to belong to the target scene image class;

[0008] Within the sample image set, candidate sample images whose predicted scores reach the score threshold are selected;

[0009] Determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process;

[0010] The candidate sample images are filtered according to the similarity to obtain positive sample images and negative sample images;

[0011] The image classification model is trained based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image;

[0012] When an image to be classified is obtained, the trained image classification model is used to classify the image to obtain a classification result indicating whether the image to be classified belongs to the target scene image class.

[0013] An image classification device, the device comprising:

[0014] The score determination module is used to determine the predicted score of each sample image in the sample image set as belonging to the target scene image class;

[0015] The candidate sample image selection module is used to select candidate sample images whose predicted scores reach a score threshold within the sample image set.

[0016] A similarity determination module is used to determine the similarity between each of the candidate sample images and the difficult sample images; the difficult sample images are images that were misclassified as belonging to the target scene image class during the historical classification process;

[0017] The sample filtering module is used to filter the candidate sample images according to the similarity to obtain positive sample images and negative sample images;

[0018] The model training module is used to train the image classification model based on the positive sample images, the hard sample images, and the negative sample images until the target loss value of the image classification model reaches a preset loss value, at which point training stops, resulting in the trained image classification model. The target loss value includes a classification loss value, a first loss value between the feature maps of the hard sample images and the positive sample images, and a second loss value between the feature maps of the hard sample images and the negative sample images.

[0019] The classification module is used to classify the image to be classified using the trained image classification model when the image to be classified is acquired, and to obtain the classification result of whether the image to be classified belongs to the target scene image class.

[0020] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0021] Determine the predicted score of each sample image in the sample image set to belong to the target scene image class;

[0022] Within the sample image set, candidate sample images whose predicted scores reach the score threshold are selected;

[0023] Determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process;

[0024] The candidate sample images are filtered according to the similarity to obtain positive sample images and negative sample images;

[0025] The image classification model is trained based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image;

[0026] When an image to be classified is obtained, the trained image classification model is used to classify the image to obtain a classification result indicating whether the image to be classified belongs to the target scene image class.

[0027] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0028] Determine the predicted score of each sample image in the sample image set to belong to the target scene image class;

[0029] Within the sample image set, candidate sample images whose predicted scores reach the score threshold are selected;

[0030] Determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process;

[0031] The candidate sample images are filtered according to the similarity to obtain positive sample images and negative sample images;

[0032] The image classification model is trained based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image;

[0033] When an image to be classified is obtained, the trained image classification model is used to classify the image to obtain a classification result indicating whether the image to be classified belongs to the target scene image class.

[0034] A computer program includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the following steps:

[0035] Determine the predicted score of each sample image in the sample image set to belong to the target scene image class;

[0036] Within the sample image set, candidate sample images whose predicted scores reach the score threshold are selected;

[0037] Determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process;

[0038] The candidate sample images are filtered according to the similarity to obtain positive sample images and negative sample images;

[0039] The image classification model is trained based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image;

[0040] When an image to be classified is obtained, the trained image classification model is used to classify the image to obtain a classification result indicating whether the image to be classified belongs to the target scene image class.

[0041] The aforementioned image classification method, apparatus, computer equipment, and storage medium determine the predicted score of each sample image in the sample image set as belonging to the target scene image class. Within the sample image set, candidate sample images whose predicted scores reach a threshold are selected, and the similarity between each candidate sample image and a difficult sample image is determined. Candidate sample images are then filtered based on similarity to obtain positive and negative sample images. The image classification model is then trained based on these positive, difficult, and negative sample images until the target loss value of the image classification model reaches a preset loss value, at which point training stops, resulting in a trained image classification model. The difficult sample images are those misclassified as belonging to the target scene image class during historical classification. The target loss value includes a classification loss value, a first loss value between the feature maps of the difficult and positive sample images, and a second loss value between the feature maps of the difficult and negative sample images. The image classification model trained using this method has good classification ability for images highly suspected of belonging to the target scene. Therefore, when classifying images to be classified using the trained image classification model, misclassification of images highly suspected of belonging to the target scene is reduced, thereby improving the classification accuracy of the image classification model. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the application environment of an image classification method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating an image classification method in one embodiment;

[0044] Figure 3 This is a schematic diagram of a firecracker scene in one embodiment;

[0045] Figure 4 This is a schematic diagram of the network structure of a feature extraction model in one embodiment;

[0046] Figure 5 This is a schematic diagram of the residual module in one embodiment;

[0047] Figure 6 This is a flowchart illustrating the processing steps of an image classification model in another embodiment;

[0048] Figure 7 This is a flowchart illustrating an image classification method in one embodiment;

[0049] Figure 8 This is a flowchart illustrating an image classification method in another embodiment;

[0050] Figure 9 This is a flowchart illustrating the data coarse screening step in one embodiment;

[0051] Figure 10 This is a flowchart illustrating the data screening steps in one embodiment;

[0052] Figure 11 This is a flowchart illustrating the secondary training steps of a model in one embodiment;

[0053] Figure 12 This is a flowchart illustrating an image classification method in another embodiment;

[0054] Figure 13 This is a structural block diagram of an image classification device in one embodiment;

[0055] Figure 14 This is an internal structural diagram of a computer device in one embodiment;

[0056] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] The image classification method provided in this application involves artificial intelligence technologies such as computer perspective and machine learning, wherein:

[0059] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0060] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0061] Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in identifying, tracking, and measuring targets from a machine perspective, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, CV studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. CV technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0062] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0063] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0064] The image classification method provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. This image classification method can be executed independently on terminal 102 or server 104, or it can be implemented through interaction between terminal 102 and the server. Taking image classification performed on server 104 as an example, server 104 determines the predicted score of each sample image in the sample image set belonging to the target scene graph class; within the sample image set, candidate sample images whose predicted scores reach a score threshold are selected; the similarity between each candidate sample image and the difficult sample image is determined; the difficult sample image is an image that was misclassified as belonging to the target scene graph class in the historical classification process; candidate sample images are filtered according to similarity to obtain positive sample images and negative sample images; the image classification model is trained based on the positive sample images, difficult sample images, and negative sample images until the loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes the classification loss value, the first loss value between the features of the difficult sample image and the features of the positive sample image, and the second loss value between the features of the difficult sample image and the features of the negative sample image. When the image to be classified is obtained, the trained image classification model is used to classify the image to be classified to obtain the classification result of whether the image to be classified belongs to the target scene graph class.

[0065] In one embodiment, after obtaining a trained image classification model, server 104 can deploy the trained image classification model. Subsequently, server 104 can receive image classification requests sent by terminal 102. Server 104 extracts features from the image carried in the image classification request using the deployed trained image classification model, obtains image features, classifies the image based on the extracted features, obtains a classification result, and returns the classification result to terminal 102. The image carried in the image retrieval request of terminal 102 can be obtained through an image acquisition device, uploaded locally by the terminal, or acquired from a third-party webpage by the terminal.

[0066] Among them, terminals 102 and 106 can be, but are not limited to, various personal computers, laptops, smartphones, tablets and portable wearable devices. Server 104 can be an independent physical server or a server cluster composed of multiple service nodes in the blockchain system. The service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol that runs on top of the Transmission Control Protocol (TCP).

[0067] In addition, server 104 can also be a server cluster consisting of multiple physical servers, which can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0068] In one embodiment, such as Figure 2 As shown, an image classification method is provided, which can be applied to... Figure 1 Taking a computer device (terminal or server) as an example, the following steps are included:

[0069] S202, determine the predicted score of each sample image in the sample image set as belonging to the target scene image class.

[0070] Among them, the sample images are training samples used for model training, including images containing the target scene and images not containing the target scene. Images containing the target scene belong to the target scene graph class, and images not containing the target scene belong to the non-target scene graph class.

[0071] It is understandable that each sample image has a corresponding category label, which includes positive sample labels and negative sample labels. For example, positive sample labels can be added to sample images belonging to the target scene image category, and negative sample labels can be added to sample images belonging to the non-target scene image category; or negative sample labels can be added to sample images belonging to the target scene image category, and positive sample labels can be added to sample images belonging to the non-target scene image category.

[0072] The prediction score is the score that predicts the probability of a sample image belonging to a certain category when classifying a sample image. It can be understood that the higher the prediction score, the greater the probability that the sample image belongs to that category.

[0073] The target scenario can be a first-level restricted scenario, a fire scenario, a second-level restricted scenario, etc. Correspondingly, when the target scenario is a first-level restricted scenario, the non-target scenario is a non-first-level restricted scenario; when the target scenario is a fire scenario, the non-target scenario is a non-fire scenario; when the target scenario is a second-level restricted scenario, the non-target scenario is a non-second-level restricted scenario. Among them, a first-level restricted scenario refers to a scenario containing sensitive content 1, sensitive content 2, or horror content, a fire scenario refers to a scenario containing content such as burning or explosion, and a second-level restricted scenario refers to a scenario containing content such as sex, revealing clothing, or vulgar behavior.

[0074] In one embodiment, S202 includes the following steps: obtaining a sample image set; inputting each sample image in the sample image set into a pre-trained image classification model; and performing classification prediction on the input sample images using the pre-trained image classification model to obtain the prediction score of whether the sample image belongs to the target scene image class.

[0075] The pre-trained image classification model is used to classify and predict the input image. The pre-trained image classification model includes a feature extraction network and a classification prediction network. The feature extraction network is used to extract the image features of the input image, and the classification prediction network is used to predict the prediction score of each category of the image based on the image features. The obtained prediction score includes the prediction score of the image belonging to the target scene image class.

[0076] Feature extraction networks can be convolutional neural networks (CNNs), or networks such as ResNet101 (deep residual network 101) or ResNet18 (deep residual network 18). Among them, convolutional neural networks are a type of feedforward neural networks that include convolutional computations and have a deep structure.

[0077] Specifically, after obtaining each sample image, the computer device inputs the sample image into a pre-trained image classification model. The feature extraction network of the pre-trained image classification model extracts the sample feature map of the input sample image, and inputs the sample feature map into the classification prediction network of the pre-trained image classification model. Based on the input sample feature map, the classification prediction network determines the predicted score of whether the sample image belongs to the target scene image class.

[0078] For example, if the target scene is a first-level restricted scene, the computer device inputs the sample image of the first-level restricted scene into a pre-trained image classification model. The feature extraction network of the pre-trained image classification model extracts the sample feature map of the input first-level restricted scene image, and inputs the sample feature map into the classification prediction network of the pre-trained image classification model. Based on the input sample feature map, the classification prediction network determines the predicted score of the first-level restricted scene image as the first-level restricted scene image.

[0079] In this embodiment, the computer device inputs each sample image in the sample image set into a pre-trained image classification model. The pre-trained image classification model can then determine the predicted score of each sample image belonging to the target scene image class. Based on the predicted score, candidate sample images that can be used for model optimization training can be selected from the sample images. This avoids the long training time and failure to achieve the desired training effect when directly using the full set of sample images for model optimization training.

[0080] In one embodiment, the computer device can also pre-train the image classification model, wherein the pre-training process of the image classification model specifically includes the following steps: obtaining a pre-training sample image set, pre-training the image classification model using each pre-training sample image in the pre-training sample image set, stopping training when the pre-training loss value of the image classification model reaches a preset pre-training loss value, and obtaining the pre-trained image classification model.

[0081] In this context, the target scene corresponding to the pre-training sample image set is the same as the target scene corresponding to the sample image set. If the target scene corresponding to the sample image set is a first-level restricted scene, then the target scene corresponding to the pre-training sample image set is also a first-level restricted scene. The images in the pre-training sample image set can be completely different from the images in the sample image set, or they can be partially or completely the same. For example, the pre-training sample image set contains images A, B, and C, while the sample image set contains images B, C, and D.

[0082] It is understandable that training sample images have corresponding category labels. The labeling method of the category labels of training sample images is the same as that of the labeling method of sample images. For example, the category label of training sample images containing the target scene is a positive sample label, and the category label of training sample images not containing the target scene is a negative sample label; or, the category label of training sample images containing the target scene is a negative sample label, and the category label of training sample images not containing the target scene is a negative sample label.

[0083] Specifically, after obtaining the training sample image set, the computer device inputs each training sample image in the training sample image set into the image classification model, and extracts the training sample feature map of the training sample image through the feature extraction network of the image classification model. The classification prediction network of the image classification model classifies the input training sample image based on the training sample feature map to obtain the classification result. Based on the classification result and the category label of the training sample image, the pre-training loss value is determined, and the image classification model is adjusted based on the pre-training loss value until the pre-training loss value of the image classification model reaches the preset pre-training loss value, at which point training stops, and the pre-trained image classification model is obtained.

[0084] S204: Within the sample image set, select candidate sample images whose predicted scores reach the score threshold.

[0085] The prediction score is the score of the sample image being predicted as the target scene graph class, and the score threshold can be set based on the misclassification situation of historical classification tasks.

[0086] Understandably, in image classification tasks, image classification models can predict the probability that an input image belongs to each category. The magnitude of the probability can be reflected by the magnitude of the predicted score. The higher the predicted score of an image belonging to a certain category, the higher the probability that the image belongs to that category. Therefore, based on the predicted score, the classification result of the image is determined to belong to that category.

[0087] The classification result of an image is determined based on the predicted score. Specifically, the category with the predicted score reaching the classification score threshold can be used as the classification result. For example, if the classification score threshold is 0.8, and the image classification model predicts that a sample image has a predicted score of 0.9 for the target scene image class and a predicted score of 0.1 for the non-target scene image class, then the classification result of the sample image is output as belonging to the target scene image class.

[0088] However, when the predicted category of an image is inconsistent with its true category, it indicates that the image classification model has misclassified the image. Therefore, sample images with a score threshold greater than the classification score threshold include both images whose classification result matches the true category and images whose classification result does not match the true category. For example, if the target scene is a first-level restricted scene, sample images with a score threshold greater than the classification score threshold for the first-level restricted scene graph class include images whose true category is the first-level restricted scene graph class and images whose true category is not the first-level restricted scene graph class.

[0089] like Figure 3 The firecracker scene image shown (large area of ​​red) will be classified as a first-level restricted scene image when classified by the image classification model that identifies first-level restricted scenes, thus resulting in a misclassification of the firecracker scene image.

[0090] Specifically, after obtaining the predicted scores of each sample image in the sample image set, the computer device acquires a score threshold and selects the sample images whose predicted scores reach the score threshold as candidate sample images. It can be understood that the selected candidate sample images include sample images whose true category is the target scene image class, as well as sample images whose true category is not the target scene image class.

[0091] The score threshold is a value greater than or equal to the classification score threshold. The score threshold may include only the lower limit threshold, or it may include both the lower limit threshold and the upper limit threshold. It can be understood that if the score threshold only includes the lower limit threshold, then the predicted score is considered to have reached the score threshold if it is greater than or equal to the lower limit threshold. If the score threshold includes both the lower limit threshold and the upper limit threshold, then the predicted score is considered to have reached the score threshold if it is greater than or equal to the lower limit threshold and less than or equal to the upper limit threshold.

[0092] For example, if the lower threshold of the score threshold is 0.8 and the upper threshold is 1, then a predicted score greater than or equal to 0.8 and less than or equal to 1 is considered to have reached the score threshold. In this case, the computer device will select sample images belonging to the target scene image class with predicted scores in the range of [0.8, 1] as candidate images.

[0093] In one embodiment, the computer device can also determine a score threshold based on the misclassification rate of historical classification tasks. Specifically, the computer device acquires classification data from historical classification tasks, determines the misclassification rate within each score range based on the classification data, and determines the score threshold based on the misclassification rate.

[0094] In one embodiment, a computer device acquires classification data for classifying target scene images within a historical time period. Based on the classification data, it divides the classification threshold interval (where the score exceeds a classification threshold) into multiple sub-intervals, calculates the misclassification rate for each sub-interval, selects a target sub-interval based on the misclassification rate, and determines a score threshold based on the target sub-interval. The classification data includes the predicted classification score and the predicted score for each image belonging to the target scene image class.

[0095] For example, if the target scene is a first-level restricted scene, the computer device acquires classification data obtained from classifying images of the first-level restricted scene within a historical time period. The classification data includes a classification prediction score of 0.6 and a predicted score for each image belonging to the first-level restricted scene image class. That is, if the predicted score for an image belonging to the first-level restricted scene image class is greater than the classification prediction score of 0.6, then the classification result of the image is the first-level restricted scene image class. Based on the classification prediction score, the classification threshold interval is determined to be [0.6, 1]. The classification threshold interval is divided into two sub-intervals: [0.6, 0.8] and [0.8, 1]. The misclassification rates corresponding to the sub-intervals [0.6, 0.8] and [0.8, 1] are calculated respectively. The endpoint of the sub-interval with the highest misclassification rate is determined as the score threshold. If the misclassification rate of the sub-interval [0.8, 1] is relatively large, then 0.8 and 1 are determined as the score thresholds, that is, 0.8 is the lower threshold and 1 is the upper threshold.

[0096] S206, determine the similarity between each candidate sample image and the difficult sample image.

[0097] Similarity is used to characterize the degree of similarity between candidate sample images and difficult sample images. Difficult sample images are those that were misclassified as belonging to the target scene image class during historical classification. Similarity includes at least one of image similarity and clustering similarity between the candidate sample images and the difficult sample images. Image similarity refers to the similarity between a single candidate sample image and a single difficult sample image, while clustering similarity refers to the similarity between the cluster centers of a single candidate sample image and a difficult sample image.

[0098] It is understandable that computer equipment acquires classification data for target scene images within a historical period, and determines the misclassified images that were misclassified as target scene images based on the classification data. The misclassified images are then used to form a set of difficult sample images, and each difficult sample image in the set of difficult sample images is a misclassified image.

[0099] In one embodiment, S206 specifically includes the following steps: performing feature extraction on each candidate sample image to obtain a candidate sample feature map; performing feature extraction on the difficult sample image to obtain a difficult sample feature map; and determining the similarity between the candidate sample image and the difficult sample image based on the candidate sample feature map and the difficult sample feature map.

[0100] Specifically, the computer device inputs each candidate sample image and each hard sample image into the feature extraction model. The model extracts features from the candidate sample images to obtain candidate sample feature maps. Similarly, it extracts features from the hard sample images to obtain hard sample feature maps. Based on these two feature maps, the similarity between the candidate and hard sample images is determined. The feature extraction model uses the same network structure as the feature extraction network in the image classification model.

[0101] Figure 4 The diagram illustrates the network structure of a feature extraction model in one embodiment. This model employs a neural network, specifically comprising three convolutional layers and six ResBlocks. An image with width W and height H is input into this model and processed by the three convolutional layers, reducing its width and height to one-quarter of their original values ​​and increasing the number of channels from 3 to 128, resulting in a w / 4*h / 4*128 feature map. This feature map then passes through a sub-network consisting of six ResBlocks to generate a new feature map. The structure of the ResBlocks is as follows... Figure 5 As shown, ResBlock consists of two convolutional layers and one identity layer. After passing through 6 ResBlocks, the feature map becomes w / 4*h / 4*128. The final feature map output by the feature extraction model is the feature map after performing global max pooling (GMP) on the w / 4*h / 4*128 feature map (which can be denoted as f). t The dimension is 128. The definition of identity is: when performing polygon overlay, the output layer retains all polygons within the control boundary of one of the input layers.

[0102] In the above embodiments, the computer device extracts features from candidate sample images and hard sample images, and calculates the similarity between candidate sample images and hard sample images based on the extracted features. Based on the similarity, it further selects target sample images that can be used for model optimization training from the candidate images. This reduces the amount of training sample data while ensuring the quality of training samples, and avoids the long training time and failure to achieve the best training effect when directly using all candidate sample images for model optimization training.

[0103] S208: Filter candidate sample images based on similarity to obtain positive sample images and negative sample images.

[0104] The similarity includes at least one of image similarity and clustering similarity between the candidate sample image and the difficult sample image. Image similarity refers to the similarity between a single candidate sample image and a single difficult sample image, and clustering similarity refers to the similarity between the cluster centers of a single candidate sample image and a difficult sample image.

[0105] Specifically, after obtaining the similarity between each candidate sample image and the difficult sample image, the computer device filters the candidate sample images based on the similarity and determines the positive sample image and the negative sample image based on the filtered candidate sample images.

[0106] In one embodiment, S208 specifically includes the following steps: filtering candidate sample images based on similarity to obtain target sample images; obtaining category labels for target sample images; determining the target sample image corresponding to the positive sample label as a positive sample image; and determining the target sample image corresponding to the negative sample label as a negative sample image.

[0107] The category label is a label that is assigned to the target sample image before determining the predicted score of the target sample image belonging to the target scene image class. The category label includes positive sample label and negative sample label. It can be understood that if the target sample image labeled with a positive sample label contains the target scene, then the sample image labeled with a negative sample label does not contain the target scene; if the target sample image labeled with a positive sample label does not contain the target scene, then the sample image labeled with a negative sample label contains the target scene.

[0108] In the above embodiments, the computer device further selects target sample images that can be used for model optimization training from the candidate images based on similarity. This reduces the amount of training sample data while ensuring the quality of the training samples, and avoids the long training time and failure to achieve the best training effect when directly using all candidate sample images for model optimization training.

[0109] S210: The image classification model is trained based on positive sample images, hard sample images, and negative sample images until the loss value of the image classification model reaches the preset loss value, at which point the training stops and the trained image classification model is obtained.

[0110] The target loss values ​​include the classification loss value, the loss value between the features of the hard sample image and the features of the positive sample image, and the loss value between the features of the hard sample image and the features of the negative sample image.

[0111] Specifically, after obtaining positive sample images, difficult sample images, and negative sample images, the computer equipment inputs the positive sample images, difficult sample images, and negative sample images into the image classification model. The image classification model extracts and classifies the input images, and determines the target loss value based on the feature extraction results and classification results. The model parameters of the image classification model are adjusted based on the target loss value until the loss value of the image classification model reaches the preset loss value, at which point training stops, and the trained image classification model is obtained.

[0112] In some embodiments, stochastic gradient descent, Adagrad (Adaptive Gradient), Adadelta (an improvement of AdaGrad), RMSprop (an improvement of AdaGrad), Adam (Adaptive Moment Estimation), etc., can be used to adjust the model parameters of image-type models.

[0113] In one embodiment, S210 specifically includes the following steps: training a pre-trained image classification model based on positive sample images, hard sample images, and negative sample images until the loss value of the image classification model reaches a preset loss value, and then stopping the training to obtain the trained image classification model.

[0114] The pre-trained image classification model, as described in S202, is used to determine the predicted score of each sample image in the sample image set to belong to the target scene image class. It can be understood that training the pre-trained image classification model with positive sample images, difficult sample images, and negative sample images constitutes secondary training. This further improves the feature extraction capability of the trained image classification model for difficult sample images, thereby enabling the trained image classification model to classify images more accurately and reduce misclassification of images highly suspected of belonging to the target scene image class.

[0115] S212, when the image to be classified is obtained, the trained image classification model is used to classify the image to be classified, and the classification result of whether the image to be classified belongs to the target scene image class is obtained.

[0116] In this context, the image to be classified refers to the image that needs to be categorized, or in other words, the image that needs to be scene-identified. For example, if the image to be classified is a target scene, then it's necessary to identify whether the image contains a target scene. The target scene can be a first-level restricted scene, a fire scene, a second-level restricted scene, etc. Correspondingly, when the target scene is a first-level restricted scene, the non-target scene is a non-first-level restricted scene; when the target scene is a fire scene, the non-target scene is a non-fire scene; and when the target scene is a second-level restricted scene, the non-target scene is a non-second-level restricted scene. Specifically, a first-level restricted scene refers to a scene containing sensitive content 1, sensitive content 2, or horrifying content; a fire scene refers to a scene containing burning or explosion content; and a second-level restricted scene refers to a scene containing sexual content, revealing clothing, or vulgar behavior.

[0117] Specifically, after the computer device inputs the image to be classified into the trained image classification model, it extracts the unclassified feature map of the image through the feature extraction network of the trained image classification model, and inputs the extracted unclassified feature map into the classification prediction network of the trained image classification model. Based on the unclassified feature map, the classification prediction network predicts the category to which the image belongs, obtains the classification result, and determines whether the image to be classified belongs to the target scene image class based on the classification result, that is, determines whether the image to be classified contains the target scene based on the classification result.

[0118] The aforementioned image classification method determines the predicted score of each sample image in the sample image set as belonging to the target scene image class. Within the sample image set, candidate sample images whose predicted scores reach a threshold are selected, and the similarity between each candidate sample image and the difficult sample image is determined. Based on this similarity, candidate sample images are filtered to obtain positive and negative sample images. The image classification model is then trained based on these positive, difficult, and negative sample images until the target loss value of the image classification model reaches a preset loss value, at which point training stops, resulting in the trained image classification model. The difficult sample images are those that were misclassified as belonging to the target scene image class during historical classification. The target loss value includes the classification loss value, a first loss value between the feature maps of the difficult and positive sample images, and a second loss value between the feature maps of the difficult and negative sample images. The image classification model trained using this method has good classification ability for images highly suspected of belonging to the target scene. Therefore, when classifying images, the trained image classification model can reduce misclassification of images highly suspected of belonging to the target scene, thereby improving the classification accuracy of the image classification model.

[0119] In one embodiment, similarity includes at least one of image similarity and clustering similarity between candidate sample images and difficult sample images. The computer device determines the similarity between candidate sample images and difficult sample images based on candidate sample feature maps and difficult sample feature maps by the following steps: determining the image similarity between candidate sample images and difficult sample images based on candidate sample feature maps and difficult sample feature maps; clustering the difficult sample feature maps to obtain cluster centers; and determining the clustering similarity between candidate sample images and difficult sample images based on candidate sample feature maps and cluster centers.

[0120] Specifically, for each candidate sample image, the computer device calculates the image similarity between the candidate sample image and each difficult sample image. For example, if there are 100 candidate sample images and 10 difficult sample images, then for any one of the 100 candidate sample images, the image similarity between the candidate sample image and each of the 10 difficult sample images is calculated, thereby obtaining the 10 image similarities corresponding to the candidate sample image.

[0121] For difficult sample images, the computer device first determines their cluster centers, which can be one or more. For each candidate sample image, the computer device calculates the cluster similarity between the candidate sample image and each cluster center. For example, if there are 100 candidate sample images and 10 difficult sample images, the difficult sample images are clustered into 2 classes, resulting in 2 cluster centers. For any candidate sample image among the 100, the image similarity between the candidate sample image and each of the 2 cluster centers is calculated, thus obtaining the 2 cluster similarity values ​​corresponding to the candidate sample image.

[0122] In one embodiment, the process by which a computer device determines the image similarity between a candidate sample image and a hard sample image based on a candidate sample feature map and a hard sample feature map includes: calculating the cosine similarity between the candidate sample feature map and the hard sample feature map, and determining the calculated cosine similarity as the image similarity between the candidate sample image and the hard sample image; the process by which a computer device determines the cluster similarity between a candidate sample image and a hard sample image based on a candidate sample feature map and cluster centers includes: calculating the cosine similarity between the candidate sample feature map and the cluster centers, and determining the calculated cosine similarity as the cluster similarity between the candidate sample image and the hard sample image.

[0123] Cosine similarity, also known as cosine distance, is a measure of the difference between two vectors. Cosine distance uses the cosine of the angle between two vectors in a vector space as a measure of the difference between the two individuals. It can be understood that the closer the cosine value is to 1, the closer the angle is to 0 degrees, which means that the two vectors are more similar. This is called "cosine similarity".

[0124] The cosine similarity between the feature maps of candidate samples and hard samples can be calculated using the following formula (1):

[0125]

[0126] Where A is the hard sample feature map, B is the candidate sample feature map, similarity(A,B) is the cosine similarity between the candidate and hard sample feature maps, and n is the dimension of the feature maps. When n = 128, it means that both the hard and candidate sample feature maps have 128 dimensions. i B represents the feature value corresponding to the i-th dimension of the feature map of hard samples. i This represents the feature value corresponding to the i-th dimension of the candidate sample feature map.

[0127] The cosine similarity between the candidate sample feature map and the cluster center can be calculated using the following formula (2):

[0128]

[0129] Where Ac represents the cluster centers of the hard sample feature map, B represents the candidate sample feature map, similarity(Ac,B) is the cosine similarity between the cluster centers of the candidate and hard sample feature maps, and n is the dimension of the feature map. When n = 128, it means that both the cluster centers of the hard sample feature map and the candidate sample feature map have a dimension of 128. i B represents the feature value corresponding to the i-th dimension of the cluster center of the feature map of hard samples. i This represents the feature value corresponding to the i-th dimension of the candidate sample feature map.

[0130] In one embodiment, the process of a computer device clustering hard sample feature maps to obtain cluster centers includes the following steps: selecting a preset number of target hard feature maps from each hard sample feature map; using the target hard feature maps as initial cluster centers, and determining a first distance between other hard sample feature maps and the initial cluster centers, determining the classification of other hard samples based on the first distance, and obtaining an initial category; determining the centroid in the hard sample feature maps of the category, where the category is either the initial category or the updated category, and determining a second distance between other hard sample feature maps outside the centroid and the centroid, determining the classification of the hard sample feature maps outside the centroid based on the second distance, obtaining the updated category of each hard sample feature map, and repeating the step of determining the centroid in the hard sample feature maps of the category until the maximum number of iterations is reached or the second distance is less than a preset distance threshold, and determining the centroid corresponding to the second distance at this time as the cluster center obtained by clustering the hard sample feature maps.

[0131] The first distance represents the similarity between the feature maps of other difficult samples and the feature maps of difficult samples corresponding to the initial cluster center. It can be understood that the larger the first distance, the smaller the corresponding similarity, and the smaller the first distance, the larger the corresponding similarity. The centroid is the updated cluster center.

[0132] In one embodiment, similarity includes image similarity. The process of a computer device filtering candidate sample images based on similarity to obtain a target sample image includes: obtaining an image similarity threshold, and determining candidate sample images whose image similarity reaches the image similarity threshold as target sample images.

[0133] Specifically, for any candidate sample image, after obtaining the image similarity between the candidate sample image and each difficult sample image, the computer device determines whether there is a target image similarity among the image similarities corresponding to the candidate sample image that reaches the image similarity threshold. If there is, the candidate sample image is selected as the target sample image; if not, the candidate sample image is not selected.

[0134] For example, if a candidate sample image has image similarities of 80%, 50%, and 40% with three difficult sample images, and the image similarity threshold is 70%, then the candidate sample image is determined to be the target sample image if the image similarity of 80% in the candidate sample image reaches the image similarity threshold. If the similarity threshold is 85%, then the candidate sample image is not selected if there are no images in the candidate sample image that reach the image similarity threshold.

[0135] In one embodiment, similarity includes cluster similarity. The process of a computer device filtering candidate sample images based on similarity to obtain a target sample image includes: obtaining a cluster similarity threshold, and determining candidate sample images whose cluster similarity reaches the cluster similarity threshold as target sample images.

[0136] Specifically, for any candidate sample image, after obtaining the cluster similarity between the candidate sample image and the cluster centers of each difficult sample image, the computer device determines whether there is a target cluster similarity that reaches the cluster similarity threshold among the cluster similarities corresponding to the candidate sample image. If there is, the candidate sample image is selected as the target sample image; if not, the candidate sample image is not selected.

[0137] For example, if the cluster similarity of a candidate sample image with the three cluster centers of a difficult sample image is 70%, 55%, and 40%, respectively, and the cluster similarity threshold is 70%, then the candidate sample image is determined to be the target sample image if the cluster similarity of 70% in the candidate sample image reaches the cluster similarity threshold; if the similarity threshold is 80%, then the candidate sample image is not selected if there is no cluster similarity in the candidate sample image that reaches the cluster similarity threshold.

[0138] In the above embodiments, the computer device determines the image similarity between candidate sample images and difficult sample images, and determines the cluster similarity between the cluster centers of candidate sample images and difficult sample images. Based on the image similarity and / or cluster similarity, it can further select target sample images that can be used for model optimization training from the candidate images. This reduces the amount of training sample data while ensuring the quality of training samples, and avoids the long training time and failure to achieve the best training effect when directly using all candidate sample images for model optimization training.

[0139] In one embodiment, similarity includes image similarity and cluster similarity. The process of a computer device filtering candidate sample images based on similarity to obtain a target sample image includes: comparing image similarity and cluster similarity; if the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

[0140] Specifically, for any candidate sample image, after obtaining the image similarity between the candidate sample image and each difficult sample image, and the cluster similarity between the candidate sample image and the cluster centers of each difficult sample image, the computer device determines whether there is a target image similarity among the image similarities corresponding to the candidate sample image that is greater than the cluster similarity corresponding to the candidate sample image. If so, the candidate sample image corresponding to the target image similarity is determined as the target sample image. The selection of the target sample image can refer to the following formula (3):

[0141] similarity(A,B)>similarity(Ac,B) (3)

[0142] Where similarity(A,B) is the cosine similarity between the candidate sample feature map and the hard sample feature map, and similarity(Ac,B) is the cosine similarity between the cluster centers of the candidate sample feature map and the hard sample feature map.

[0143] For example, if a candidate image has image similarities of 65%, 58%, 55%, 49%, 45%, and 40% with six difficult sample images, and the six difficult sample images are clustered into two cluster centers, and the candidate image has cluster similarities of 60% and 70% with the two cluster centers, then the candidate image is determined to have a target image similarity of 65% greater than the cluster similarity of 60%, and is therefore selected as the target image. Conversely, if a candidate image has image similarities of 50%, 58%, 55%, 49%, 45%, and 40% with six difficult sample images, and the six difficult sample images are clustered into two cluster centers, and the candidate image has cluster similarities of 60% and 65% with the two cluster centers, then the candidate image is not selected because its image similarity does not exceed the target image similarity of the cluster similarity.

[0144] In one embodiment, similarity includes image similarity and cluster similarity. The process by which a computer device filters candidate sample images based on similarity to obtain a target sample image includes: calculating an adjusted image similarity based on the predicted score of the candidate sample image and the corresponding image similarity; comparing the adjusted image similarity with the cluster similarity; and if the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

[0145] Specifically, for any candidate sample image, after obtaining the image similarity between the candidate sample image and each difficult sample image, and the cluster similarity between the candidate sample image and the cluster centers of each difficult sample image, the computer device uses the product of the predicted score of the candidate sample image and the corresponding image similarity as the corresponding adjusted image similarity. It then determines whether there exists a target adjusted image similarity among the adjusted image similarities corresponding to the candidate sample image that is greater than the cluster similarity corresponding to the candidate sample image. If so, the candidate sample image corresponding to the target adjusted image similarity is determined as the target sample image. The selection of the target sample image can refer to the following formula (4):

[0146] pred(B)*similarity(A,B)>similarity(Ac,B) (4)

[0147] Where pred(B) is the predicted score corresponding to the candidate sample feature map, similarity(A,B) is the cosine similarity between the candidate sample feature map and the hard sample feature map, and similarity(Ac,B) is the cosine similarity between the cluster centers of the candidate sample feature map and the hard sample feature map.

[0148] For example, if a candidate image has image similarities of 65%, 58%, 55%, 49%, 45%, and 40% with six difficult sample images, and the six difficult sample images are clustered into two cluster centers, and the cluster similarities between the candidate image and the two cluster centers are 60% and 70%, respectively, and the candidate image's de prediction score is 0.8, then the adjusted image similarities between the candidate image and the six difficult sample images are determined to be 52%, 46.4%, 44%, 39.2%, 36%, and 32%, respectively. Therefore, it is determined that the adjusted image similarity of the candidate image does not have a target image similarity greater than the cluster similarity, and thus the candidate image is not selected.

[0149] In the above embodiments, the computer device further selects high-quality target sample images that can be used for model optimization training from the candidate images based on image similarity and cluster similarity. This reduces the amount of training sample data while ensuring the quality of the training samples, and avoids the long training time and failure to achieve the best training effect when directly using all candidate sample images for model optimization training.

[0150] In one embodiment, S210 specifically includes the following steps: grouping positive sample images, hard sample images, and negative sample images into sample image groups, each sample group containing at least one positive sample image, at least one hard sample image, and at least one negative sample image; sequentially inputting each sample image group into the image classification model to train the image classification model based on the sample image groups until the target loss value of the image classification model reaches the preset loss value and then stopping the training.

[0151] Specifically, the computer device acquires preset batch processing parameters and groups positive sample images, difficult sample images, and negative sample images based on the batch processing parameters to obtain sample image groups. That is, the positive sample images, difficult sample images, and negative sample images are divided into batches, and each batch is a sample image group. After obtaining the sample image groups, the same batch of sample images, difficult sample images, and negative sample images are taken out and input into the image classification model each time. The image classification model is trained based on the input sample images, difficult sample images, and negative sample images until the target loss value of the image classification model reaches the preset loss value and then the training stops.

[0152] The batch processing parameter (batch_size) refers to the size of each batch of data. In this embodiment, the batch processing parameter can be the number of positive sample images, hard sample images, and negative sample images contained in each batch. For example, a batch processing parameter of "1:1:1" means that each batch contains 1 positive sample image, 1 hard sample image, and 1 negative sample image. A batch processing parameter of "2:2:2" means that each batch contains 2 positive sample images, 2 hard sample images, and 2 negative sample images. Taking one positive sample image, one hard sample image, and one negative sample image as a sample pair, the number of sample pairs corresponding to the batch processing parameter "1:1:1" is 1, and the number of sample pairs corresponding to the batch processing parameter "2:2:2" is 3.

[0153] In the above embodiments, the computer device groups positive sample images, difficult sample images, and negative sample images, and uses each sample image group to train the image classification model. Each sample image group contains positive sample images, difficult sample images, and negative sample images, thereby enabling triple training of the image classification model. This allows the trained image classification model to have better classification ability for images with highly suspected target scenes, reducing misclassification of images with highly suspected target scenes and thus improving the classification accuracy of the image classification model.

[0154] In one embodiment, the process of a computer device training an image classification model based on a set of sample images includes the following steps: using the image classification model, extracting features from each sample image in the input set of sample images to obtain a hard sample feature map, a positive sample feature map, and a negative sample feature map; determining a first loss value based on the hard sample feature map and the positive sample feature map; and determining a second loss value based on the hard sample feature map and the negative sample feature map; classifying the sample images in the set of sample images according to the hard sample feature map, the positive sample feature map, and the negative sample feature map, and determining a classification loss value based on the classification result and the corresponding category label; determining a target loss value based on the classification loss value, the first loss value, and the second loss value; and adjusting the parameters of the image classification model based on the target loss value.

[0155] Specifically, after obtaining the feature maps of difficult samples, positive samples, and negative samples, the computer device obtains a first loss value based on the difference between the feature maps of difficult samples and positive samples, and a second loss value based on the difference between the feature maps of difficult samples and negative samples. It then classifies the difficult samples based on the feature maps of difficult samples to obtain a first classification result, classifies the positive sample images based on the feature maps of positive samples to obtain a second classification result, and classifies the negative sample images based on the feature maps of negative samples to obtain a third classification result. Finally, it obtains a first classification loss value based on the difference between the first classification result and the category label of the difficult sample image, a second classification loss value based on the difference between the second classification result and the category label of the positive sample image, and a third classification loss value based on the difference between the third classification result and the category label of the negative sample image. Based on the first, second, and third classification loss values, it determines the classification loss value, determines the triplet loss value based on the first and second loss values, and calculates the target loss value based on the classification loss value and the triplet, referring to the following formula (5):

[0156] L total =w1L calss +w2L em (5)

[0157] Among them, L total L is the target loss value. calss L is the classification loss value. em is the triplet loss value, and w1 and w2 are the weights of the loss value, respectively.

[0158] It is understandable that difficult sample images are images that were misclassified as belonging to the target scene image class during the historical classification process. That is, difficult sample images do not contain the target scene themselves. Therefore, the category label of difficult sample images is consistent with the category label of target sample images that do not contain the target scene (non-target scene image class). That is, if the category label of the target sample image containing the target scene is a positive sample label, then the category label of the difficult sample image is also a positive sample label; if the category label of the training sample image containing the target scene is a negative sample label, then the category label of the difficult sample image is also a negative sample label.

[0159] In one embodiment, the process by which a computer device determines a classification loss value based on a first classification loss value, a second classification loss value, and a third classification loss value includes: obtaining the first classification loss weight corresponding to a difficult sample image, the second classification loss weight corresponding to a positive sample image, and the third classification loss weight corresponding to a negative sample image; performing a weighted summation of the first classification loss value, the second classification loss value, and the third classification loss value based on the first classification loss weight, the second classification loss weight, and the third classification loss weight to obtain a summation result; and using the summation result as the classification loss value of the image classification model.

[0160] In the above embodiments, during the triplet training of the image classification model, the computer device calculates the target loss value by determining the first loss value between the feature map of the difficult sample image and the feature map of the positive sample image, the second loss value between the feature map of the difficult sample image and the feature map of the negative sample image, and the classification loss value. Based on the target loss value, the model parameters of the image classification model are adjusted to achieve triplet training of the image classification model. This enables the trained image classification model to have a better classification ability for images with highly suspected target scenes, reduces misjudgments of images with highly suspected target scenes, and thus improves the classification accuracy of the image classification model.

[0161] In one embodiment, the process by which a computer device determines a first loss value based on a hard sample feature map and a positive sample feature map includes: determining the Euclidean distance between the hard sample feature map and the positive sample feature map, and using the Euclidean distance between the hard sample feature map and the positive sample feature map as the first loss value. Here, the Euclidean distance is the feature distance between the hard sample feature map and the positive sample feature map, and is used to characterize the difference between the hard sample feature map and the positive sample feature map.

[0162] In one embodiment, the process by which a computer device determines a second loss value based on a hard sample feature map and a negative sample feature map includes: determining the Euclidean distance between the hard sample feature map and the negative sample feature map, and using the Euclidean distance between the hard sample feature map and the negative sample feature map as the second loss value. Here, the Euclidean distance is the feature distance between the hard sample feature map and the negative sample feature map, and is used to characterize the difference between the hard sample feature map and the negative sample feature map.

[0163] In the above embodiments, the computer device determines the Euclidean distance between the hard sample feature map and the positive sample feature map, and the Euclidean distance between the hard sample feature map and the negative sample feature map, thereby obtaining a first loss value and a second loss value. Then, it calculates the target loss value and adjusts the model parameters of the image classification model based on the target loss value to achieve triple training of the image classification model. This enables the trained image classification model to have a better classification ability for images with highly suspected target scenes, reduces misjudgments of images with highly suspected target scenes, and thus improves the classification accuracy of the image classification model.

[0164] In one embodiment, the process by which a computer device determines a target loss value based on a classification loss value, a first loss value, and a second loss value includes: obtaining a distance constant; determining the difference between the first loss value and the second loss value; when the sum of the difference and the distance constant is greater than zero, using the sum of the difference, the distance constant, and the classification loss value as the target loss value; and when the sum of the difference and the distance constant is less than or equal to zero, determining the target loss value based on the classification loss value.

[0165] The distance constant can also be called the regulator, specifically the regulation factor α in formula (6). The sum of the difference, the distance constant, and the classification loss value can be obtained by weighted summation.

[0166] Specifically, after obtaining the distance constant, the computer device can determine the triplet loss value based on the distance constant, the first loss value, and the second loss value. When the sum of the difference between the first and second loss values ​​and the distance constant is greater than zero, the sum of the difference and the distance constant is determined as the triplet loss value, and the sum of the triplet loss value and the classification loss value is used as the target loss value. When the sum of the difference between the first and second loss values ​​and the distance constant is not greater than zero, the triplet loss value is determined to be 0, and the target loss value is determined based on the classification loss value. The computer device can refer to formula (5) to determine the target loss value.

[0167] In one embodiment, the computer device may determine the triplet loss value by referring to the following formula (6):

[0168]

[0169] Among them, L em Let x be the triplet loss value. a For hard sample images, x p For positive sample images, x n Let N be the negative sample image, and N represent the number of sample pairs in the input sample image group. This can be understood as one negative sample image, one positive sample image, and one negative sample image constituting a sample pair, with j representing the j-th sample pair. This refers to the difficult sample image in the j-th sample pair of the currently input sample image group. This refers to the positive sample image in the j-th sample pair of the currently input sample image group. This refers to the negative sample image in the j-th sample pair of the currently input sample image group. This is the hard sample feature map of the j-th sample pair in the current input sample image group. This is the positive sample feature map of the positive sample image in the j-th sample pair of the current input sample image group. This is the negative sample feature map of the negative sample image in the j-th sample pair of the current input sample image group. This indicates the calculation of Euclidean distance, where α is a condition factor, and its value can be set as needed. For example, α can be 4. + The value in brackets [] indicates that when the value is greater than 0, the value in brackets [] is taken as the triplet loss value, and when the value in brackets [] is less than 0, the triplet loss value is taken as 0.

[0170] The purpose of formula (6) is to make the Euclidean distance between the hard sample feature map and the positive sample feature map smaller than the Euclidean distance between the hard sample feature map and the negative sample feature map by α. It can also be seen from formula (6) that the triplet loss value is 0 only when the Euclidean distance between the hard sample feature map and the negative sample feature map is at least α greater than the Euclidean distance between the hard sample feature map and the positive sample feature map. Otherwise, the triplet loss value is greater than 0. Therefore, in the process of reducing the target loss value, the Euclidean distance between the hard sample feature map and the negative sample feature map develops in the direction of being α greater than the Euclidean distance between the hard sample feature map and the positive sample feature map, so that the image classification model can better focus on the preservation of classification features.

[0171] In the above embodiments, the computer device obtains a distance constant and determines the difference between the first loss value and the second loss value. Based on this difference and the classification loss value, a target loss value can be determined. This causes the Euclidean distance between the hard sample feature map and the negative sample feature map to increase compared to the Euclidean distance between the hard sample feature map and the positive sample feature map during the reduction of the target loss value. This allows the image classification model to better preserve classification features, resulting in a better classification ability for images with highly suspected target scenes, reducing misclassifications of such images, and ultimately improving the classification accuracy of the image classification model.

[0172] In one embodiment, such as Figure 6 As shown, the process of a computer device (terminal or server) processing an image classification model includes the following steps:

[0173] S602, Obtain the sample image set, and input each sample image in the sample image set into the pre-trained image classification model.

[0174] S604 uses a pre-trained image classification model to classify and predict the input sample images, obtaining the predicted score of whether the sample image belongs to the target scene image class.

[0175] S606: Within the sample image set, select candidate sample images whose predicted scores reach the score threshold.

[0176] S608: Perform feature extraction on each candidate sample image to obtain the candidate sample feature map; perform feature extraction on the difficult sample image to obtain the difficult sample feature map.

[0177] Among them, the difficult sample images are those that were misclassified as belonging to the target scene image class during the historical classification process.

[0178] S610, based on the candidate sample feature map and the hard sample feature map, determine the image similarity between the candidate sample image and the hard sample image.

[0179] S612, cluster the feature maps of difficult samples to obtain cluster centers, and determine the cluster similarity between candidate sample images and difficult sample images based on the feature maps of candidate samples and cluster centers.

[0180] S614 compares image similarity and cluster similarity, and determines the target sample image based on the comparison results.

[0181] If the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

[0182] S616, Obtain the category label of the target sample image, and determine the positive and negative sample images based on the category label.

[0183] The category labels include positive sample labels and negative sample labels.

[0184] Specifically, the computer device determines the target sample image corresponding to the positive sample label as a positive sample image and the target sample image corresponding to the negative sample label as a negative sample image.

[0185] S618, group the positive sample images, difficult sample images and negative sample images into sample image groups, each sample group containing at least one positive sample image, at least one difficult sample image and at least one negative sample image.

[0186] S620: Input each sample image group into the image classification model in sequence to train the image classification model based on the sample image groups until the target loss value of the image classification model reaches the preset loss value and then stop training.

[0187] The target loss value includes the classification loss value, the first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and the second loss value between the feature map of the hard sample image and the feature map of the negative sample image.

[0188] In one embodiment, such as Figure 7As shown, an image classification method is also provided. This image classification method obtains an image classification model through the processing steps of the above-described image classification model, and is applied to... Figure 1 Taking a computer device (terminal or server) as an example, the following steps are included:

[0189] S702, Obtain the image to be classified.

[0190] Among them, the image to be classified is the image that needs to be classified, or the image that needs to be scene recognized. For example, the image to be classified is the image that needs to be target scene recognized, that is, to identify whether the image to be classified contains the target scene.

[0191] S704: Input the image to be classified into the trained image classification model.

[0192] The trained image classification model is obtained by training on positive sample images, hard sample images, and negative sample images. Hard sample images are images that were misclassified as belonging to the target scene image class during the historical classification process. Positive and negative sample images are selected from the candidate sample images based on the similarity between the candidate sample images and the hard sample images. The candidate sample images are selected from the sample image set based on the predicted scores of each sample image belonging to the target scene image class. The training process of the image classification model can be referred to S202 to S210.

[0193] S706 uses a trained image classification model to predict the classification of the input image to be classified, and obtains the classification result.

[0194] The trained image classification model includes a feature extraction network and a classification prediction network. The feature extraction network is used to extract feature maps from the input image, and the classification prediction network is used to predict the category to which the image belongs based on the feature maps.

[0195] Specifically, after the computer device inputs the image to be classified into the trained image classification model, it extracts the unclassified feature map of the image through the feature extraction network of the trained image classification model, and inputs the extracted unclassified feature map into the classification prediction network of the trained image classification model. Based on the unclassified feature map, the classification prediction network predicts the category to which the image belongs, and obtains the classification result.

[0196] This application also provides a training application scenario for an image separation model for image classification of a first-level restricted scene, which applies the processing steps of the image classification model in the above-described image classification method.

[0197] Specifically, refer to Figure 8 The image classification method is applied in this scenario as follows:

[0198] Step 1: Coarse data screening, filtering candidate sample images from a massive number of sample images. For example... Figure 9 As shown, the specific process is as follows:

[0199] (1) The computer device acquires sample images containing the first restricted level scene and sample images not containing the first restricted level scene, and adds category labels to each sample image, wherein the category label of the sample image not containing the first restricted level scene is a positive sample label, and the category label of the sample image containing the first restricted level scene is a negative sample label.

[0200] (2) Input the sample images into the pre-trained image classification model, and determine the predicted score of each sample image belonging to the first restricted level scene graph class through the pre-trained image classification model.

[0201] (3) In each sample image, select the candidate sample image whose predicted score reaches the score threshold.

[0202] Specifically, the computer device divides each sample image into multiple clusters at 0.1-point intervals based on the predicted score, and selects sample images with scores greater than 0.9 as candidate sample images. It is understood that the selected candidate sample images include sample images containing the first-level restriction scene and sample images not containing the first-level restriction scene.

[0203] Step 2: Data refinement. Target sample images are selected from the candidate sample images to obtain a training sample set for secondary training of the image classification model. For example... Figure 10 As shown, the specific process is as follows:

[0204] (1) Obtain difficult sample images. Difficult sample images are images that were misclassified as belonging to the first restricted level scene image class during the historical classification process. Input the candidate sample images and difficult sample images into the feature extraction model respectively, so as to extract the candidate sample features corresponding to the candidate sample images and the difficult sample features corresponding to the difficult sample images through the feature extraction model.

[0205] (2) Cluster the difficult sample images based on the difficult sample features to obtain the cluster centers of the difficult sample images; calculate the image similarity and cluster similarity between the candidate sample images and the difficult sample images based on the candidate sample features, the difficult sample features and the cluster centers of the difficult sample images.

[0206] (3) Adjust the image similarity based on the predicted score of the candidate sample image to obtain the adjusted image similarity, and determine the candidate sample image with the adjusted image similarity greater than the cluster similarity as the target sample image.

[0207] (4) Based on the category label of the target sample image, determine the positive sample image and the negative sample image.

[0208] Specifically, the target sample image with positive sample label is determined as the positive sample image, and the target sample image with negative sample label is determined as the negative sample image, thus obtaining positive sample image, negative sample image and hard sample image. The image set composed of positive sample image, negative sample image and hard sample image is the training sample set used for secondary training of image classification model.

[0209] Step 3: Secondary model training. This involves using a training sample set containing positive, negative, and difficult sample images to perform secondary training on the pre-trained image classification model, thereby obtaining an image recognition model with high classification ability for difficult samples. For example... Figure 11 As shown, the specific process is as follows:

[0210] (1) The images in the training sample set, which includes positive sample images, negative sample images and hard sample images, are batched.

[0211] Each batch contains at least one sample pair, and each sample pair contains one positive sample image, one negative sample image, and one hard sample image.

[0212] (2) Take out the same batch of images each time and input them into the pre-trained image classification model to train the pre-trained image classification model with the input images until the target loss value of the image classification model reaches the preset loss value and then stop training.

[0213] Specifically, image features of the input image are extracted through a pre-trained image classification model. On the one hand, the triplet loss value is calculated based on the image features, and on the other hand, the input image is classified based on the image features. The classification loss value is determined based on the classification result. The target loss value of the image classification model is obtained based on the classification loss value and the triplet loss value. The model parameters of the pre-trained image classification model are adjusted based on the target loss value. The next batch of images is then taken and input into the pre-trained image classification model to train the pre-trained image classification model with the input images until the target loss value of the image classification model reaches the preset loss value, at which point training stops.

[0214] The triplet loss value includes the first loss value between the feature maps of the hard sample image and the positive sample image, and the second loss value between the feature maps of the hard sample image and the negative sample image. The triplet loss value can be determined with reference to formula (6). The purpose of formula (6) is to make the Euclidean distance between the feature maps of the hard sample image and the positive sample image smaller than the Euclidean distance between the feature maps of the hard sample image and the negative sample image by α. It can also be seen from formula (6) that the triplet loss value is 0 only when the Euclidean distance between the feature maps of the hard sample image and the negative sample image is at least α greater than the Euclidean distance between the feature maps of the hard sample image and the positive sample image. Otherwise, the triplet loss value is greater than 0. Therefore, in the process of reducing the target loss value, the Euclidean distance between the feature maps of the hard sample image and the negative sample image tends to be larger than the Euclidean distance between the feature maps of the hard sample image and the positive sample image by α, so that the image classification model can better focus on the preservation of classification features.

[0215] It is understandable that the application of this image classification method in this scenario can be summarized as follows: Figure 12 The process shown involves mining sample images from a set of sample images based on hard sample images to obtain a training sample set for secondary training. The image classification model is then trained using triples based on the training sample set, resulting in an optimized image classification model. This optimized model can better extract image features to classify images, thereby improving the accuracy of image classification.

[0216] It should be understood that, although Figure 2 , Figures 6-12 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 2 , Figures 6-12 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0217] In one embodiment, such as Figure 13As shown, an image classification device is provided. This device can be a software module, a hardware module, or a combination of both integrated into a computer device. Specifically, the device includes: a score determination module 1302, a candidate sample image selection module 1304, a similarity determination module 1306, a sample screening module 1308, a model training module 1310, and a classification module 1312, wherein:

[0218] The score determination module 1302 is used to determine the predicted score of each sample image in the sample image set belonging to the target scene image class;

[0219] The candidate sample image selection module 1304 is used to select candidate sample images whose predicted scores reach a score threshold within the sample image set.

[0220] The similarity determination module 1306 is used to determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process;

[0221] The sample filtering module 1308 is used to filter candidate sample images based on similarity to obtain positive sample images and negative sample images;

[0222] The model training module 1310 is used to train the image classification model based on positive sample images, hard sample images, and negative sample images until the target loss value of the image classification model reaches the preset loss value, at which point the training stops and the trained image classification model is obtained. The target loss value includes the classification loss value, the first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and the second loss value between the feature map of the hard sample image and the feature map of the negative sample image.

[0223] The classification module 1312, when it acquires an image to be classified, classifies the image to be classified using the trained image classification model to obtain the classification result of whether the image to be classified belongs to the target scene image class.

[0224] In the above embodiments, by determining the predicted score of each sample image in the sample image set belonging to the target scene image class, candidate sample images with predicted scores reaching a score threshold are selected in the sample image set, and the similarity between each candidate sample image and the difficult sample image is determined. Candidate sample images are then filtered based on similarity to obtain positive sample images and negative sample images. The image classification model is then trained based on the positive sample images, difficult sample images, and negative sample images until the target loss value of the image classification model reaches a preset loss value, at which point training stops, resulting in a trained image classification model. The difficult sample images are those that were misclassified as belonging to the target scene image class during historical classification. The target loss value includes the classification loss value, a first loss value between the feature maps of the difficult sample images and the positive sample images, and a second loss value between the feature maps of the difficult sample images and the negative sample images. The image classification model trained using this method has a good classification ability for images highly suspected of belonging to the target scene. Therefore, when classifying images to be classified using the trained image classification model, it can reduce misclassification of images highly suspected of belonging to the target scene, thereby improving the classification accuracy of the image classification model.

[0225] In one embodiment, the score determination module 1302 is used to: acquire a sample image set; input each sample image in the sample image set into a pre-trained image classification model; and perform classification prediction on the input sample images using the pre-trained image classification model to obtain the predicted score of the sample image belonging to the target scene image class.

[0226] In one embodiment, the model training module 1310 is used to: train a pre-trained image classification model based on positive sample images, hard sample images, and negative sample images until the loss value of the image classification model reaches a preset loss value, and then stop training to obtain the trained image classification model.

[0227] In one embodiment, the similarity determination module 1306 is configured to: extract features from each candidate sample image to obtain a candidate sample feature map; extract features from the difficult sample image to obtain a difficult sample feature map; and determine the similarity between the candidate sample image and the difficult sample image based on the candidate sample feature map and the difficult sample feature map.

[0228] In one embodiment, similarity includes image similarity and cluster similarity between candidate sample images and difficult sample images; the similarity determination module 1306 is used to: determine the image similarity between candidate sample images and difficult sample images based on candidate sample feature maps and difficult sample feature maps; cluster the difficult sample feature maps to obtain cluster centers; and determine the cluster similarity between candidate sample images and difficult sample images based on candidate sample feature maps and cluster centers.

[0229] In one embodiment, the sample filtering module 1308 is configured to: filter candidate sample images based on similarity to obtain target sample images; obtain category labels for the target sample images; the category labels include positive sample labels and negative sample labels; determine the target sample image corresponding to the positive sample label as a positive sample image; and determine the target sample image corresponding to the negative sample label as a negative sample image.

[0230] In one embodiment, similarity includes image similarity and cluster similarity between candidate sample images and difficult sample images; the root sample screening module 1308 is used to: compare image similarity and cluster similarity; if the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

[0231] In one embodiment, the model training module 1310 is configured to: group positive sample images, hard sample images, and negative sample images into sample image groups, each sample group containing at least one positive sample image, at least one hard sample image, and at least one negative sample image; and sequentially input each sample image group into the image classification model to train the image classification model based on the sample image groups until the loss value of the image classification model reaches a preset loss value and training stops.

[0232] In one embodiment, the model training module 1310 is configured to: extract features from each sample image in the input sample image group using an image classification model to obtain a hard sample feature map, a positive sample feature map, and a negative sample feature map; determine a first loss value based on the hard sample feature map and the positive sample feature map; and determine a second loss value based on the hard sample feature map and the negative sample feature map; classify the sample images in the sample image group according to the hard sample feature map, the positive sample feature map, and the negative sample feature map, and determine a classification loss value based on the classification result and the corresponding category label; determine a target loss value based on the classification loss value, the first loss value, and the second loss value; and adjust the parameters of the image classification model based on the target loss value.

[0233] In one embodiment, the model training module 1310 is configured to: determine the Euclidean distance between the hard sample feature map and the positive sample feature map, and use the Euclidean distance between the hard sample feature map and the positive sample feature map as a first loss value; determine the Euclidean distance between the hard sample feature map and the negative sample feature map, and use the Euclidean distance between the hard sample feature map and the negative sample feature map as a second loss value.

[0234] In one embodiment, the model training module 1310 is configured to: obtain a distance constant; determine the difference between a first loss value and a second loss value; when the sum of the difference and the distance constant is greater than zero, use the sum of the difference, the distance constant, and the classification loss value as the target loss value; when the sum of the difference and the distance constant is less than or equal to zero, determine the target loss value based on the classification loss value.

[0235] Specific limitations regarding the image classification device can be found in the limitations of the image classification method above, and will not be repeated here. Each module in the aforementioned image classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0236] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 14 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image classification models and image data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image classification method.

[0237] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 15As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image classification method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0238] Those skilled in the art will understand that Figure 14 and Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0239] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0240] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0241] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0242] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0243] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0244] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image classification method, characterized in that, The method includes: Determine the predicted score of each sample image in the sample image set to belong to the target scene image class; Within the sample image set, candidate sample images whose predicted scores reach the score threshold are selected; Determine the similarity between each candidate sample image and the difficult sample image; the difficult sample image is an image that was misclassified as belonging to the target scene image class during the historical classification process; The candidate sample images are filtered based on the similarity to obtain target sample images; the category labels of the target sample images are obtained; the category labels include positive sample labels and negative sample labels; the target sample images corresponding to the positive sample labels are determined as positive sample images; the target sample images corresponding to the negative sample labels are determined as negative sample images. The image classification model is trained based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image; When an image to be classified is obtained, the trained image classification model is used to classify the image to obtain a classification result of whether the image to be classified belongs to the target scene image class.

2. The method according to claim 1, characterized in that, The prediction score for determining the target scene image class for each sample image in the sample image set includes: Obtain the sample image set; Each sample image in the sample image set is input into the pre-trained image classification model; The pre-trained image classification model is used to classify and predict the input sample images to obtain the predicted score of the sample image belonging to the target scene image class.

3. The method according to claim 2, characterized in that, The step of training the image classification model based on the positive sample images, the difficult sample images, and the negative sample images until the target loss value of the image classification model reaches a preset loss value includes: The pre-trained image classification model is trained based on the positive sample images, the difficult sample images, and the negative sample images until the loss value of the image classification model reaches the preset loss value, at which point training stops, and the trained image classification model is obtained.

4. The method according to claim 1, characterized in that, Determining the similarity between each candidate sample image and the difficult sample image includes: Feature extraction is performed on each of the candidate sample images to obtain candidate sample feature maps; Feature extraction is performed on the difficult sample images to obtain difficult sample feature maps; Based on the candidate sample feature map and the hard sample feature map, the similarity between the candidate sample image and the hard sample image is determined.

5. The method according to claim 4, characterized in that, The similarity includes image similarity and clustering similarity between the candidate sample image and the difficult sample image; Determining the similarity between the candidate sample image and the hard sample image based on the candidate sample feature map and the hard sample feature map includes: Based on the candidate sample feature map and the hard sample feature map, the image similarity between the candidate sample image and the hard sample image is determined; Clustering is performed on the feature maps of the difficult samples to obtain cluster centers; Based on the candidate sample feature map and the cluster center, the cluster similarity between the candidate sample image and the difficult sample image is determined.

6. The method according to claim 5, characterized in that, The similarity includes image similarity and clustering similarity between the candidate sample image and the difficult sample image; The step of filtering the candidate sample images based on the similarity to obtain the target sample image includes: The image similarity and the cluster similarity are compared. If the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

7. The method according to any one of claims 1 to 6, characterized in that, The training of the image classification model based on the positive sample images, the difficult sample images, and the negative sample images includes: The positive sample image, the difficult sample image, and the negative sample image are grouped to obtain sample image groups, and each sample group contains at least one positive sample image, at least one difficult sample image, and at least one negative sample image; Each of the sample image groups is sequentially input into the image classification model to train the image classification model based on the sample image groups until the target loss value of the image classification model reaches the preset loss value and training stops.

8. The method according to claim 7, characterized in that, The training of the image classification model based on the sample image group includes: The image classification model extracts features from each sample image in the input sample image group to obtain feature maps of difficult samples, positive samples, and negative samples. A first loss value is determined based on the hard sample feature map and the positive sample feature map; and a second loss value is determined based on the hard sample feature map and the negative sample feature map. Based on the hard sample feature map, the positive sample feature map, and the negative sample feature map, the sample images in the sample image group are classified respectively, and the classification loss value is determined based on the classification result and the corresponding category label; The target loss value is determined based on the classification loss value, the first loss value, and the second loss value; The parameters of the image classification model are adjusted based on the target loss value.

9. The method according to claim 8, characterized in that, The first loss value is determined based on the feature map of the difficult samples and the feature map of the positive samples; And, determining a second loss value based on the hard sample feature map and the negative sample feature map, including: Determine the Euclidean distance between the hard sample feature map and the positive sample feature map, and use the Euclidean distance between the hard sample feature map and the positive sample feature map as the first loss value; The Euclidean distance between the hard sample feature map and the negative sample feature map is determined, and the Euclidean distance between the hard sample feature map and the negative sample feature map is used as the second loss value.

10. The method according to claim 9, characterized in that, The step of determining the target loss value based on the classification loss value, the first loss value, and the second loss value includes: Obtain the distance constant; Determine the difference between the first loss value and the second loss value; When the sum of the difference and the distance constant is greater than zero, the sum of the difference, the distance constant, and the classification loss value is taken as the target loss value; When the sum of the difference and the distance constant is less than or equal to zero, the target loss value is determined based on the classification loss value.

11. An image classification device, characterized in that, The device includes: The score determination module is used to determine the predicted score of each sample image in the sample image set as belonging to the target scene image class; The candidate sample image selection module is used to select candidate sample images whose predicted scores reach a score threshold within the sample image set. A similarity determination module is used to determine the similarity between each of the candidate sample images and the difficult sample images; the difficult sample images are images that were misclassified as belonging to the target scene image class during the historical classification process; A sample filtering module is used to filter the candidate sample images according to the similarity to obtain target sample images; obtain the category labels of the target sample images; the category labels include positive sample labels and negative sample labels; determine the target sample images corresponding to the positive sample labels as positive sample images; and determine the target sample images corresponding to the negative sample labels as negative sample images. The model training module is used to train the image classification model based on the positive sample image, the hard sample image, and the negative sample image until the target loss value of the image classification model reaches a preset loss value, at which point training stops, and the trained image classification model is obtained; wherein, the target loss value includes a classification loss value, a first loss value between the feature map of the hard sample image and the feature map of the positive sample image, and a second loss value between the feature map of the hard sample image and the feature map of the negative sample image; The classification module is used to classify the image to be classified using the trained image classification model when the image to be classified is acquired, and to obtain the classification result of whether the image to be classified belongs to the target scene image class.

12. The apparatus according to claim 11, characterized in that, The score determination module also includes: Obtain the sample image set; Each sample image in the sample image set is input into the pre-trained image classification model; The pre-trained image classification model is used to classify and predict the input sample images to obtain the predicted score of the sample image belonging to the target scene image class.

13. The apparatus according to claim 12, characterized in that, The model training module is also used for: The pre-trained image classification model is trained based on the positive sample images, the difficult sample images, and the negative sample images until the loss value of the image classification model reaches the preset loss value, at which point training stops, and the trained image classification model is obtained.

14. The apparatus according to claim 11, characterized in that, The similarity determination module is further used for: Feature extraction is performed on each of the candidate sample images to obtain candidate sample feature maps; Feature extraction is performed on the difficult sample images to obtain difficult sample feature maps; Based on the candidate sample feature map and the hard sample feature map, the similarity between the candidate sample image and the hard sample image is determined.

15. The apparatus according to claim 14, characterized in that, The similarity includes image similarity and clustering similarity between the candidate sample image and the difficult sample image; The similarity determination module is further used for: Based on the candidate sample feature map and the hard sample feature map, the image similarity between the candidate sample image and the hard sample image is determined; Clustering is performed on the feature maps of the difficult samples to obtain cluster centers; Based on the candidate sample feature map and the cluster center, the cluster similarity between the candidate sample image and the difficult sample image is determined.

16. The apparatus according to claim 15, characterized in that, The similarity includes image similarity and clustering similarity between the candidate sample image and the difficult sample image; The sample screening module is also used for: The image similarity and the cluster similarity are compared. If the image similarity is greater than the cluster similarity, then the candidate sample image corresponding to the image similarity greater than the cluster similarity is determined as the target sample image.

17. The apparatus according to any one of claims 11 to 16, characterized in that, The model training module is also used for: The positive sample image, the difficult sample image, and the negative sample image are grouped to obtain sample image groups, and each sample group contains at least one positive sample image, at least one difficult sample image, and at least one negative sample image; Each of the sample image groups is sequentially input into the image classification model to train the image classification model based on the sample image groups until the target loss value of the image classification model reaches the preset loss value and training stops.

18. The apparatus according to claim 17, characterized in that, The model training module is also used for: The image classification model extracts features from each sample image in the input sample image group to obtain feature maps of difficult samples, positive samples, and negative samples. A first loss value is determined based on the hard sample feature map and the positive sample feature map; and a second loss value is determined based on the hard sample feature map and the negative sample feature map. Based on the hard sample feature map, the positive sample feature map, and the negative sample feature map, the sample images in the sample image group are classified respectively, and the classification loss value is determined based on the classification result and the corresponding category label; The target loss value is determined based on the classification loss value, the first loss value, and the second loss value; The parameters of the image classification model are adjusted based on the target loss value.

19. The apparatus according to claim 18, characterized in that, The model training module is also used for: Determine the Euclidean distance between the hard sample feature map and the positive sample feature map, and use the Euclidean distance between the hard sample feature map and the positive sample feature map as the first loss value; The Euclidean distance between the hard sample feature map and the negative sample feature map is determined, and the Euclidean distance between the hard sample feature map and the negative sample feature map is used as the second loss value.

20. The apparatus according to claim 19, characterized in that, The model training module is also used for: Obtain the distance constant; Determine the difference between the first loss value and the second loss value; When the sum of the difference and the distance constant is greater than zero, the sum of the difference, the distance constant, and the classification loss value is taken as the target loss value; When the sum of the difference and the distance constant is less than or equal to zero, the target loss value is determined based on the classification loss value.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

22. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images