A method, device, equipment and storage medium for training an image classification model

Abstract

The application provides a method and device for training an image classification model, an equipment and a storage medium, which can be applied to the field of artificial intelligence or the field of smart agriculture, and is used to solve the problem of low classification accuracy and reliability of the target image classification model. The method comprises the following steps: adopting multiple image processing strategies to perform pixel-level feature extraction on a sample image set respectively to obtain multiple feature map sets; each iteration comprises the following steps: extracting the semantic features of the selected sample image and the corresponding multiple feature maps respectively, and performing feature fusion on the obtained semantic features to obtain the comprehensive features of the sample image; determining the predicted classification of the sample image based on the comprehensive features; and adjusting the model parameters of the image classification model based on the multiple feature maps, the comprehensive features and the predicted classification. By expanding the sample image data and fully learning each sample image, the classification accuracy and reliability of the target image classification model are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for training an image classification model. Background Technology

[0002] With the continuous development of technology, more and more devices can provide image classification services through trained target image classification models. Image classification services can be used to determine the category to which a target belongs in a target image.

[0003] In related technologies, the method to obtain a trained target image classification model is usually to perform multiple rounds of iterative training on the image classification model based on a sample image set. In each iteration, the image features of the sample image are extracted first, then the predicted classification of the sample image is determined based on the image features, and finally the model parameters of the image classification model are adjusted based on the error between the predicted classification and the classification label of the sample image.

[0004] However, using the above method for model training has the following drawbacks: First, when the target size of the object contained in the sample image is smaller than the image size of the sample image, the sample image contains more noise information, which causes the extracted image features to also contain more noise features, thus producing a large training error.

[0005] Secondly, among the pixel-level features represented by the sample image, usually only one type of pixel-level feature is dominant, while other pixel-level features do not have obvious representation. For example, among the pixel-level features represented by a photographic image, color-related features are usually dominant, while grayscale features do not have obvious representation; as another example, among the pixel-level features represented by a textured image, texture-related features are usually dominant, while local binary features do not have obvious representation.

[0006] Therefore, when training an image classification model based on a sample image set, the image classification model usually learns mainly based on the dominant pixel-level features represented by the sample image, and cannot comprehensively learn all pixel-level features represented by the sample image, so the classification accuracy of the trained target image classification model always has a bottleneck.

[0007] It is evident that the training methods employed under the relevant technologies cannot guarantee the classification accuracy and reliability of the target image classification model obtained through training. Summary of the Invention

[0008] This application provides a method, apparatus, computer device, and storage medium for training an image classification model, which addresses the problem of low classification accuracy and reliability of the trained target image classification model.

[0009] Firstly, a method for training an image classification model is provided, including: Multiple image processing strategies are employed to extract pixel-level features from the sample image set, resulting in multiple feature map sets. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set. Each sample image corresponds to one feature map in each feature map set. Based on the sample image set and the obtained multiple feature maps, the image classification model to be trained is subjected to multiple rounds of iterative training, outputting the trained target image classification model. Each round of iteration includes: Semantic features of the selected sample image and its corresponding multiple feature maps are extracted respectively, and the obtained semantic features are fused to obtain the comprehensive features of the sample image. Based on the comprehensive features, the predicted classification of the sample image is determined; Based on the multiple feature maps, the integrated features, and the predicted classification, the model parameters of the image classification model are adjusted.

[0010] Secondly, an apparatus for training an image classification model is provided, comprising: Acquisition module: Used to extract pixel-level features from the sample image set using various image processing strategies to obtain multiple feature map sets. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set. Each sample image corresponds to one feature map in each feature map set. Processing module: Used to perform multiple rounds of iterative training on the image classification model to be trained based on the sample image set and the obtained multiple feature maps, and output the trained target image classification model, wherein each round of iteration includes: The processing module is also used to: extract the semantic features of the selected sample image and the corresponding multiple feature maps respectively, and perform feature fusion on the obtained semantic features to obtain the comprehensive features of the sample image; The processing module is further configured to: determine the predicted classification of the sample image based on the comprehensive features; The processing module is also used to: adjust the model parameters of the image classification model based on the multiple feature maps, the comprehensive features, and the predicted classification.

[0011] Optionally, the processing module is specifically used for: Based on the multiple feature maps, the integrated features, and the predicted classification, the adversarial loss, fusion loss, and classification loss of the image classification model are determined, wherein the adversarial loss represents the accuracy of extracting semantic features, the fusion loss represents the accuracy of feature fusion, and the classification loss represents the accuracy of determining the predicted classification. The model parameters of the image classification model are adjusted based on the adversarial loss, the fusion loss, and the classification loss.

[0012] Optionally, the processing module is specifically used for: For each of the multiple feature maps, perform the following operations: Based on the semantic features of the feature map and the semantic features of the sample image, the feature map is converted into a pseudo image, wherein the pseudo image and the sample image represent the same type of pixel-level features; Based on the sample images and the pseudo images, the adversarial loss of the image classification model is determined.

[0013] Optionally, the processing module is specifically used for: Extract the semantic features of the pseudo-image, and determine the first reconstruction loss of the image classification model based on the error between the semantic features of the sample image and the semantic features of the pseudo-image; Based on the error between the sample image and the pseudo image, the second reconstruction loss of the image classification model is determined; Based on the first reconstruction loss and the second reconstruction loss, the adversarial loss of the image classification model is determined.

[0014] Optionally, the image classification model includes a discriminant network, which is used to predict the discrimination probability of the input data being a sample image or the semantic features of a sample image, wherein the input data has associated reference probabilities; then the processing module is specifically used for: The sample image and the pseudo image are used as input data to the discriminant network to predict the corresponding first discrimination probability. The semantic features of the sample image and the semantic features of the pseudo image are used as input data to the discriminant network to predict the corresponding second discrimination probability. Based on the error between each obtained first discrimination probability and each second discrimination probability and their corresponding reference probability, the discrimination loss of the image classification model is determined; The weighted sum of the first reconstruction loss, the second reconstruction loss, and the discriminant loss is used as the adversarial loss of the image classification model.

[0015] Optionally, the processing module is specifically used for: The semantic features of the sample image and the semantic features of the multiple feature maps are weighted and summed to obtain reference features; The fusion loss of the image classification model is determined based on the error between the integrated features and the reference features.

[0016] Optionally, the processing module is specifically used for: The training loss of the image classification model is obtained by weighted summing of the adversarial loss, the fusion loss, and the classification loss. If the obtained training loss does not meet the training objective, the model parameters of the image classification model are adjusted, and the next round of iterative training begins.

[0017] Optionally, each feature set can be one of the following: grayscale feature set, texture feature set, local binary feature set, or oriented gradient histogram set.

[0018] Thirdly, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] Fourthly, a computer device is provided, comprising: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method described in the first aspect according to the obtained program instructions.

[0020] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method as described in the first aspect.

[0021] In this embodiment, multiple image processing strategies are employed to extract pixel-level features from the sample image set, thereby obtaining multiple feature map sets. Each sample image corresponds to a feature map in each feature map set. Each feature map corresponding to a sample image can represent multiple types of pixel-level features. Therefore, when the image classification model to be trained is iterated multiple times based on multiple feature map sets of the sample image set, the image classification model can fully learn the multiple types of pixel-level features contained in each sample image. This avoids the problem that when the image classification model is trained directly based on the sample image, the image classification model only learns the dominant type of pixel-level feature in the sample image, resulting in low classification accuracy and reliability of the trained target image classification model.

[0022] Furthermore, each sample image corresponds to multiple feature maps representing different types of pixel-level features. These feature maps interpret the target object, background, and noise contained in the sample image differently. Noise may not be clearly represented in some feature maps, and the background may not be clearly represented in others. However, since each feature map is obtained based on the target object contained in the sample image, the target object is clearly represented in all feature maps. Therefore, after extracting the semantic features of the sample image and multiple feature maps and performing feature fusion, the resulting comprehensive features primarily reflect the features of the target object contained in the sample image, while weakening unnecessary content such as background and noise. Thus, the target image classification model trained based on the comprehensive features can more accurately extract the semantic features of the target object contained in the image to be classified, enabling accurate and reliable classification.

[0023] Furthermore, after obtaining multiple feature maps corresponding to the sample image set, the image classification model to be trained can be trained based on the sample image set and multiple feature maps. This is equivalent to expanding the number of sample images contained in the original sample image set to many times the original number, thereby increasing the training data. This allows the image classification model to be fully trained, thereby improving the classification accuracy and reliability of the trained target image classification model. Attached Figure Description

[0024] Figure 1A This is a schematic diagram illustrating the application field of the image classification model provided in the embodiments of this application; Figure 1B This is one application scenario of the method for training an image classification model provided in the embodiments of this application; Figure 2 A flowchart illustrating a method for training an image classification model provided in this application embodiment; Figure 3A A schematic diagram illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 3B A schematic diagram of the principle of the method for training an image classification model provided in the embodiments of this application. Figure 2 ; Figure 4 A schematic diagram three illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 5A A schematic diagram five illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 5B A schematic diagram six illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 5CA schematic diagram of the principle of the method for training an image classification model provided in the embodiments of this application. Figure 7 ; Figure 5D A schematic diagram of the principle of the method for training an image classification model provided in the embodiments of this application. Figure 8 ; Figure 5E A schematic diagram of the principle of the method for training an image classification model provided in the embodiments of this application. Figure 9 ; Figure 6A A schematic diagram ten illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 6B A flowchart illustrating a method for training an image classification model provided in an embodiment of this application. Figure 2 ; Figure 7 11. A schematic diagram illustrating the principle of a method for training an image classification model provided in this application embodiment; Figure 8 A schematic diagram of a device for training an image classification model provided in an embodiment of this application; Figure 9 A schematic diagram of the structure of an apparatus for training an image classification model provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0026] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0027] (1) Local Binary Pattern (LBP) features: LBP features are operators used to describe local features of an image, possessing significant advantages such as gray-level invariance and rotation invariance. At pixel 3... Within the neighborhood of 3, using the center pixel as a threshold, the grayscale values ​​of the eight adjacent pixels are compared with the value of the center pixel. If the surrounding pixels have a higher grayscale value than the center pixel, the pixel is marked as 1; otherwise, it is marked as 0. Thus, 3 The comparison of 8 points within a 3-neighborhood produces an 8-bit binary number. These 8 bits are arranged sequentially to form a binary number, which is the LBP value of the center pixel. There are 28 possible LBP values, so there are 256 possible LBP values. The LBP value of the center pixel can reflect the texture information of the area surrounding the pixel.

[0028] (2) Histogram of Oriented Gradient (HOG) features: Histogram of Oriented Gradients (HOG) features are feature descriptors used for object detection in computer vision and image processing. HOG features are constructed by calculating and statistically analyzing the gradient orientation histograms of local regions in an image. In an image, the appearance and shape of a local object can be well described by the distribution of gradient or edge orientation density.

[0029] This application relates to the field of Artificial Intelligence (AI), and is designed based on Computer Vision (CV) and Machine Learning (ML) technologies. It can be applied to fields such as cloud computing, smart transportation, smart agriculture, and mapping.

[0030] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that studies the design principles and implementation methods of various machines, attempting to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence, enabling machines to have perception, reasoning, and decision-making functions.

[0031] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, interactive operating systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, machine learning / deep learning, autonomous driving, and intelligent transportation. With the development and progress of AI, it has been researched and applied in numerous fields, such as smart homes, intelligent customer service, virtual assistants, smart speakers, intelligent marketing, wearable devices, autonomous driving, drones, robots, smart healthcare, vehicle networking, and intelligent transportation. It is believed that with further technological advancements, AI will be applied in even more fields, playing an increasingly important role. The solutions provided in this application's embodiments relate to deep learning and augmented reality technologies in AI, which are further illustrated by the following examples.

[0032] Computer vision is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in identifying, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0033] Machine learning is a multidisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers acquire new knowledge or skills by simulating human learning behavior, reorganize existing knowledge structures, and continuously improve their performance.

[0034] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Deep learning, on the other hand, is the core of machine learning and a technology for realizing machine learning. Machine learning typically includes techniques such as deep learning, reinforcement learning, transfer learning, inductive learning, artificial neural networks, and instructional learning. Deep learning includes techniques such as convolutional neural networks (CNNs), deep belief networks, recurrent neural networks, autoencoders, and generative adversarial networks.

[0035] It should be noted that in the embodiments of this application, data such as sample image sets or images to be classified are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0036] The following is a brief introduction to the application areas of the method for training image classification models provided in the embodiments of this application.

[0037] With the continuous development of technology, more and more devices can provide image classification services through trained target image classification models. Image classification services can be used to determine the category to which a target belongs in a target image.

[0038] For example, in the agricultural sector, smart agriculture has become a mainstream emerging technology, relying on artificial intelligence to achieve tasks such as detection, analysis, early warning, and prevention in agricultural production. Pest and disease image classification, as an important component of smart agriculture, can assist agricultural workers in dealing with crop pests and diseases, enabling early detection and prevention.

[0039] The task of classifying pest and disease images can identify different types of diseases present in crops based on crop images. Please refer to [reference needed]. Figure 1A Based on the leaf or fruit condition in crop images, diseases such as downy mildew or anthracnose can be identified. By identifying the diseases present in crops, targeted treatments can be administered, reducing the overuse of chemical pesticides, avoiding environmental pollution, and minimizing the emergence of drug-resistant pathogens. This also reduces planting costs and economic losses caused by misdiagnosis.

[0040] Compared to relying on agricultural experts to identify crop pests and diseases, image classification services provided by pre-trained target image classification models do not require highly skilled and experienced agricultural experts. This avoids subjective factors, improves the efficiency of identifying crop pests and diseases, and reduces labor costs.

[0041] In related technologies, the method to obtain a trained target image classification model is usually to perform multiple rounds of iterative training on the image classification model based on a sample image set. In each iteration, the image features of the sample image are extracted first, then the predicted classification of the sample image is determined based on the image features, and finally the model parameters of the image classification model are adjusted based on the error between the predicted classification and the classification label of the sample image.

[0042] However, using the above method for model training has the following drawbacks: First, when the target size of the object contained in the sample image is smaller than the image size of the sample image, the sample image contains more noise information, which causes the extracted image features to also contain more noise features, thus producing a large training error.

[0043] Secondly, among the pixel-level features represented by the sample image, usually only one type of pixel-level feature is dominant, while other pixel-level features do not have obvious representation. For example, among the pixel-level features represented by a photographic image, color-related features are usually dominant, while grayscale features do not have obvious representation; as another example, among the pixel-level features represented by a textured image, texture-related features are usually dominant, while local binary features do not have obvious representation.

[0044] Therefore, when training an image classification model based on a sample image set, the image classification model usually learns mainly based on the dominant pixel-level features represented by the sample image, and cannot comprehensively learn all pixel-level features represented by the sample image, so the classification accuracy of the trained target image classification model always has a bottleneck.

[0045] It is evident that the training methods employed under the relevant technologies cannot guarantee the classification accuracy and reliability of the target image classification model obtained through training.

[0046] To address the issue of low classification accuracy and reliability in trained target image classification models, this application proposes a method for training image classification models. This method employs multiple image processing strategies to extract pixel-level features from a sample image set, obtaining multiple feature maps. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set, with each sample image corresponding to one feature map in each feature map set. Based on the sample image set and the obtained multiple feature maps, the image classification model to be trained undergoes multiple rounds of iterative training, outputting a trained target image classification model. Each iteration includes: Semantic features are extracted from the selected sample images and their corresponding multiple feature maps. These semantic features are then fused to obtain the comprehensive features of the sample images. Based on these comprehensive features, the predicted classification of the sample images is determined. Finally, the model parameters of the image classification model are adjusted based on the multiple feature maps, the comprehensive features, and the predicted classification.

[0047] In this embodiment, multiple image processing strategies are employed to extract pixel-level features from the sample image set, thereby obtaining multiple feature map sets. Each sample image corresponds to a feature map in each feature map set. Each feature map corresponding to a sample image can represent multiple types of pixel-level features. Therefore, when the image classification model to be trained is iterated multiple times based on multiple feature map sets of the sample image set, the image classification model can fully learn the multiple types of pixel-level features contained in each sample image. This avoids the problem that when the image classification model is trained directly based on the sample image, the image classification model only learns the dominant type of pixel-level feature in the sample image, resulting in low classification accuracy and reliability of the trained target image classification model.

[0048] Furthermore, each sample image corresponds to multiple feature maps representing different types of pixel-level features. These feature maps interpret the target object, background, and noise contained in the sample image differently. Noise may not be clearly represented in some feature maps, and the background may not be clearly represented in others. However, since each feature map is obtained based on the target object contained in the sample image, the target object is clearly represented in all feature maps. Therefore, after extracting the semantic features of the sample image and multiple feature maps and performing feature fusion, the resulting comprehensive features primarily reflect the features of the target object contained in the sample image, while weakening unnecessary content such as background and noise. Thus, the target image classification model trained based on the comprehensive features can more accurately extract the semantic features of the target object contained in the image to be classified, enabling accurate and reliable classification.

[0049] Furthermore, after obtaining multiple feature maps corresponding to the sample image set, the image classification model to be trained can be trained based on the sample image set and multiple feature maps. This is equivalent to expanding the number of sample images contained in the original sample image set to many times the original number, thereby increasing the training data. This allows the image classification model to be fully trained, thereby improving the classification accuracy and reliability of the trained target image classification model.

[0050] The following describes the application scenarios of the method for training the image classification model provided in this application.

[0051] Please refer to Figure 1B This is a schematic diagram illustrating an application scenario of the method for training an image classification model provided in this application. The application scenario includes a client 101 and a server 102. The client 101 and the server 102 can communicate with each other. The communication method can be wired, such as through a network cable or serial cable; or wireless, such as through Bluetooth or Wi-Fi. No specific limitation is imposed.

[0052] Client 101 generally refers to a device that can provide sample image sets to server 102 or use a trained target image classification model, such as a terminal device, a third-party application accessible to the terminal device, or a webpage accessible to the terminal device. Terminal devices include, but are not limited to, mobile phones, computers, smart medical devices, smart home appliances, vehicle terminals, or aircraft. Server 102 generally refers to a device that can train an image classification model, such as a terminal device or a server. Servers include, but are not limited to, cloud servers, local servers, or associated third-party servers. Both client 101 and server 102 can use cloud computing to reduce the consumption of local computing resources; similarly, they can also use cloud storage to reduce the consumption of local storage resources.

[0053] As one embodiment, the client 101 and the server 102 can be the same device, and there is no specific limitation. In this embodiment, the client 101 and the server 102 are described as different devices.

[0054] The following is based on Figure 1B Using server 102 as the main component, this paper provides a detailed description of the method for training an image classification model provided in this embodiment. Please refer to [link / reference]. Figure 2 This is a flowchart illustrating a method for training an image classification model provided in an embodiment of this application.

[0055] S201 employs multiple image processing strategies to extract pixel-level features from the sample image set, thereby obtaining multiple feature maps.

[0056] Before performing multiple rounds of training on an image classification model, a set of sample images for training can be obtained first. The model can then be iteratively trained using this sample image set. The server can obtain the sample image set from network resources or collect individual sample images to form the set based on actual needs. Each sample image in the set can have an associated classification label, which characterizes the actual classification of the target object contained in the corresponding sample image.

[0057] After obtaining the sample image set, the server can employ various image processing strategies to extract pixel-level features from the sample image set, resulting in multiple feature maps. These image processing strategies include color processing, grayscale processing, texture processing, Local Binary Pattern (LBP) processing, and Histogram of Oriented Gradients (HOG) processing. Each image processing strategy, after extracting pixel-level features from the sample images, yields a corresponding type of pixel-level feature, thus generating a feature map. In other words, each image processing strategy, after extracting pixel-level features from the sample image set, generates a corresponding feature map set, with each sample image corresponding to one feature map in each feature map set. A feature map can serve as a view of the corresponding sample image.

[0058] For example, color processing strategies can be used to extract the RGB features of each sample image, obtaining the RGB feature map corresponding to the sample image. Thus, the RGB feature maps corresponding to each sample image can be combined to form an RGB feature map set.

[0059] Gray-scale processing strategies can be used to extract the gray-scale features of each sample image, obtaining the gray-scale feature map corresponding to the sample image. Thus, the gray-scale feature maps corresponding to each sample image can be combined to form a gray-scale feature map set.

[0060] Texture processing strategies can be used to extract texture features from each sample image, resulting in a texture feature map corresponding to each sample image. These texture feature maps can then be combined to form a texture feature map set. Please refer to [reference needed]. Figure 3A (1) is a sample image. Figure 3A (2) is a feature map corresponding to the sample image, which represents the texture features of the sample image.

[0061] Local binary processing strategies can be used to extract local binary features from each sample image, obtaining the corresponding local binary feature map. These local binary feature maps from each sample image can then be combined to form a local binary feature map set. Please refer to [reference needed]. Figure 3B (1) is a sample image. Figure 3B (2) is a feature map corresponding to the sample image, which represents the local binary features of the sample image.

[0062] The Histogram of Oriented Gradients (HGP) processing strategy can be used to extract the HGP features of each sample image, thereby obtaining the HGP feature map corresponding to the sample image. The HGP feature maps corresponding to each sample image can be combined to form an HGP feature map set.

[0063] In this embodiment, by employing various image processing strategies to process the sample image set, the pixel-level features represented by each sample image can be obtained more comprehensively. The obtained feature maps, together with the sample images, serve as training data for the image classification model, thereby expanding the amount of training data. Adding feature maps for training eliminates the problem of large training errors caused by unfavorable factors such as lighting or shooting conditions in the sample images, which leads to inaccurate target image classification models when training solely with sample images.

[0064] As one embodiment, after obtaining the sample image set, each sample image contained in the sample image set can be preprocessed to obtain preprocessed sample images. The preprocessed sample images can be added to the sample image set as sample images to expand the sample image set.

[0065] Preprocessing can include various methods such as rotating, misaligning, or cropping the sample images. Preprocessing can be carried out with the premise of keeping the image size uniform, so as to ensure that the sample images in the sample image set have a uniform image size, which is convenient for training the image classification model.

[0066] S202, based on the sample image set and multiple feature maps obtained, performs multiple rounds of iterative training on the image classification model to be trained, and outputs the trained target image classification model.

[0067] After obtaining a set of sample images and multiple feature images, the server can perform multiple rounds of iterative training on the image classification model to be trained based on the sample image set and the obtained multiple feature images, outputting the trained target image classification model. Thus, by learning from the sample images and the corresponding multiple feature images, the image classification model can more accurately extract various pixel-level features contained in the sample images, rather than focusing only on the dominant pixel-level features represented in the sample images.

[0068] The following describes the process of training an image classification model using one round of iterative training as an example. The process of each round of iterative training is similar and will not be repeated here. Please refer to S203~S205.

[0069] S203: Extract the semantic features of the selected sample image and the corresponding multiple feature maps respectively, and perform feature fusion on the obtained semantic features to obtain the comprehensive features of the sample image.

[0070] For a sample image selected from the sample image set, the server can use an image classification model to extract the semantic features of the sample image and its corresponding multiple feature maps. The semantic features represent the deep abstract features represented by the corresponding sample image or feature map.

[0071] After obtaining the semantic features of the sample image and the semantic features of multiple feature maps, the server can perform feature fusion to obtain the comprehensive features of the sample image. Since the semantic features are extracted specifically for the target objects contained in the sample image and feature maps, some of the extracted semantic features may contain noise or background features that are not helpful in classifying the target objects. However, each semantic feature contains features related to the target objects. Therefore, feature fusion further strengthens the representation of features related to the target objects while weakening the representation of features unrelated to the target objects. Thus, through feature fusion, the target object-related features in the sample image can be obtained more accurately, leading to a more accurate image classification task.

[0072] As one embodiment, the image classification model can include an encoder network, which is used to extract semantic features from the input image. Sample images and feature maps can be represented by different encoders within the encoder network. Each encoder can focus on extracting semantic features from the input image representing one type of pixel-level features. This eliminates the need for a single encoder to learn and extract semantic features from input images representing multiple types of pixel-level features, simplifying the training process for each encoder and improving training efficiency.

[0073] For example, please refer to a feature map. Figure 4 The image classification model comprises an encoder network and a fusion network. The encoder network contains two encoders: a first encoder for each sample image, used to extract semantic features from the sample image; and a second encoder for each feature map, used to extract semantic features from the feature map. The fusion network performs feature fusion on the semantic features of the sample image and the feature map to obtain a comprehensive feature set of the sample image.

[0074] A first encoder is used to extract features from the sample image to obtain first semantic features, and a second encoder is used to extract features from the feature map to obtain second semantic features. After obtaining the first and second semantic features, a fusion network is used to fuse the first and second semantic features to obtain comprehensive features.

[0075] S204, based on comprehensive features, determines the predicted classification of the sample image.

[0076] After obtaining the comprehensive features, the server can use an image classification model to determine the predicted classification of the sample image based on these features. The predicted classification can characterize the category of the target object contained in the sample image. For example, in the field of agricultural pests and diseases, taking the scenario of image classification for grape leaf pests and diseases as an example, the sample image can be an image containing grape leaves. Then, the predicted classification and classification label for the sample image can be whether the grape leaves have pests and diseases, or the type of pests and diseases present in the grape leaves, etc., without any specific limitations.

[0077] Predicted classification can be represented as a probability distribution. Depending on the application scenario, multiple categories can be pre-set, which can be the various categories included in the classification labels corresponding to the sample image set. Thus, when determining the predicted classification of a sample image based on comprehensive features, the probability of the sample image belonging to each category can be predicted separately, forming a probability distribution. The classification label of the sample image can also be represented as a probability distribution, for example, the probability of the sample image belonging to a category is 1, and the probability of other categories is 0, etc., without any specific restrictions.

[0078] S205 adjusts the model parameters of the image classification model based on multiple feature maps, integrated features, and predicted classification.

[0079] The server can adjust the model parameters of the image classification model based on multiple feature maps obtained from sample images, combined features, and predicted classification. By adjusting the model parameters from multiple perspectives—feature maps, combined features, and predicted classification—the image classification model can focus more on extracting semantic features from feature maps, performing feature fusion on semantic features, and determining the predicted classification of sample images. Compared to methods that only adjust the model parameters based on predicted classification, the trained image classification model has higher classification accuracy and reliability.

[0080] As one implementation, the server can determine the adversarial loss, fusion loss, and classification loss of an image classification model based on multiple feature maps, combined features, and predicted classification. The adversarial loss represents the accuracy of semantic feature extraction; the fusion loss represents the accuracy of feature fusion; and the classification loss represents the accuracy of determining the predicted classification. Based on the adversarial loss, fusion loss, and classification loss, the model parameters of the image classification model are adjusted.

[0081] By using multiple loss functions, the training effect of the image classification model is measured, allowing the image classification model to be trained more comprehensively, thereby enabling the trained target image classification model to have more accurate and reliable image classification capabilities.

[0082] There are several methods for determining the adversarial loss, fusion loss, and classification loss of an image classification model based on multiple feature maps, integrated features, and predicted classification. The following section uses one of these methods as an example to describe the process of obtaining the adversarial loss, fusion loss, and classification loss. Furthermore, the process of determining the adversarial loss, fusion loss, and classification loss of an image classification model based on a single feature map is used as an example; the process based on other feature maps is similar and will not be elaborated upon here.

[0083] Combat losses: Based on the semantic features of the feature map and the sample image, the feature map is converted into a pseudo image. Based on the sample image and the pseudo image, the adversarial loss of the image classification model is determined.

[0084] The server can employ an image classification model. Based on feature maps and semantic features of sample images, it converts feature maps into pseudo-images, where the pseudo-images and sample images represent the same type of pixel-level features. Since the pseudo-images are generated from the sample images, the more similar the pseudo-images are to the sample images, the more accurately the image classification model has learned the transformation process between sample images and feature maps. Therefore, the trained target image classification model, when faced with sample images, no longer focuses solely on the dominant pixel-level features represented by the sample images, but can comprehensively extract semantic features that can represent multiple pixel-level features from the sample images, improving the classification accuracy of the target image classification model. Thus, the adversarial loss of the image classification model can be determined based on the pseudo-images and sample images, and the model can be trained based on this adversarial loss.

[0085] As one example, after obtaining the comprehensive features, the server can also use an image classification model to convert the feature map into a pseudo image based on the comprehensive features and the semantic features of the sample image, without any specific limitations.

[0086] As one embodiment, the image classification model may include a generator network, which generates corresponding pseudo-images based on the input data and a specified image. Each feature map can correspond to a generator in the generator network, and different generators can generate corresponding pseudo-images based on different feature maps and sample images. Alternatively, all feature maps can correspond to the same generator in the generator network, etc., without any specific restrictions.

[0087] Please refer to Figure 5A After feature extraction from the sample images using an image classification model, semantic features, or feature subspace, are obtained. A generative network is then used to convert the feature maps into pseudo-images based on the semantic features and feature maps of the sample images.

[0088] As one example, after obtaining the pseudo image generated based on the feature map, an image classification model can be further used to extract the semantic features of the pseudo image. Based on the error between the semantic features of the sample image and the semantic features of the pseudo image, the first reconstruction loss of the image classification model is determined. From the abstract perspective of the semantic features represented by the pseudo image and the sample image respectively, the similarity between the pseudo image and the sample image is measured.

[0089] Taking a feature map as an example, an image classification model includes an editor network and a generator network. The editor network contains a first editor for extracting semantic features from sample images and a second editor for extracting pseudo-images generated based on the feature map. The generator network is used to transform the feature map into pseudo-images. Please refer to [reference needed]. Figure 5B The sample image is processed by a first editor to extract the first semantic features. The first semantic features and the feature map are then used by a generative network to generate a pseudo-image. The pseudo-image is processed by a second editor to extract the second semantic features. Based on the error between the first and second semantic features, the first reconstruction loss is determined. Please refer to formula (1) to determine the first reconstruction loss of the image classification model. One method.

[0090] (1)

[0091] in, The sample images represent data and follow a data distribution. , The feature map represents the data distribution. , Characterize the semantic features of the sample image. Characterizing semantic features z and feature maps based on sample images The generated pseudo-image, Characterizing the semantic features of pseudo-images Characterizes the L1 norm.

[0092] After obtaining the pseudo-images generated based on the feature maps, the second reconstruction loss of the image classification model can be determined based on the error between the sample images and the pseudo-images. This measures the similarity between the pseudo-images and the sample images from a concrete perspective, comparing them directly.

[0093] Taking a feature map as an example, an image classification model includes an editor network and a generator network. The editor network extracts semantic features from the sample image, while the generator network transforms the feature map into a pseudo-image. Please refer to [reference needed]. Figure 5CThe semantic features of the sample images are extracted through an editor network. These semantic features and feature maps are then used to generate pseudo-images through a generator network. The second reconstruction loss is determined based on the error between the sample images and the pseudo-images. Please refer to formula (2) for determining the second reconstruction loss of the image classification model. One method.

[0094] (2)

[0095] in, The sample images represent data and follow a data distribution. , The feature map represents the data distribution. , Characterize the semantic features of the sample image. Characterizing semantic features z and feature maps based on sample images The generated pseudo-image, Characterizes the L1 norm.

[0096] After obtaining the first reconstruction loss, the adversarial loss of the image classification model can be determined based solely on the first reconstruction loss; after obtaining the second reconstruction loss, the adversarial loss of the image classification model can be determined based solely on the second reconstruction loss; after obtaining the first reconstruction loss and the second reconstruction loss, the adversarial loss of the image classification model can be determined together based on the first reconstruction loss and the second reconstruction loss.

[0097] As one embodiment, in S203, when extracting the semantic features of the feature map, the feature map can be converted into a pseudo image using an image classification model according to the aforementioned content, and the semantic features of the pseudo image can be extracted using the image classification model. The semantic features of the pseudo image can then be used as the semantic features of the feature map.

[0098] As one embodiment, the image classification model may include a discriminative network that predicts the probability that the input data is a sample image or a semantic feature of a sample image. The input data has associated reference probabilities, which may be represented as a probability distribution. For example, the probability distribution may contain two categories: one where the input data is a sample image, and the other where the input data is not a sample image. In the probability distribution of the reference probabilities for sample images, the probability that the input data is a sample image is 1, and the probability that the input data is not a sample image is 0.

[0099] The sample image and the pseudo image are used as input data to the discriminant network to predict the corresponding first discrimination probabilities, namely the first discrimination probability of the sample image and the first discrimination probability of the pseudo image. Simultaneously, the semantic features of the sample image and the pseudo image are used as input data to the discriminant network to predict the corresponding second discrimination probabilities, namely the second discrimination probability of the semantic features of the sample image and the second discrimination probability of the semantic features of the pseudo image.

[0100] The discrimination loss of the image classification model is determined based on the errors between each obtained first and second discrimination probability and their corresponding reference probabilities. The discrimination loss of the image classification model is also determined based on the errors between the first discrimination probability and the reference probability of the sample image, the first discrimination probability and the reference probability of the pseudo-image, the second discrimination probability of the semantic features of the sample image and the reference probability of the sample image, and the second discrimination probability of the semantic features of the pseudo-image and the reference probability of the pseudo-image.

[0101] As one embodiment, the discriminative loss can include a first discriminative loss and a second discriminative loss. Taking a feature map as an example, the image classification model includes an editor network, a generator network, and a discriminator network. The editor network is used to extract semantic features from the sample image. The generator network is used to convert the feature map into a pseudo-image, and the discriminator network is used to determine the discriminative probability of the input data. The generator network and the discriminator network form an adversarial network. The idea of ​​an adversarial network is to have the generator network and the discriminator network participate in a minimization or maximization game, that is, the generator network and the discriminator network should minimize and maximize the probability of the pseudo-image being judged as fake, respectively. The first discriminative loss is determined based on the error between the first discriminative probability of the sample image and the reference probability of the sample image, and the error between the first discriminative probability of the pseudo-image and the reference probability of the pseudo-image.

[0102] Please refer to Figure 5D The sample image is processed by an editor network to extract its semantic features. These semantic features and feature maps are then processed by a generator network to generate pseudo-images. The sample image and the pseudo-image are used as input data to a discriminator network to obtain the first discrimination probability of the sample image and the first discrimination probability of the pseudo-image. Based on the error between the first discrimination probability of the sample image and its reference probability, and the error between the first discrimination probability of the pseudo-image and its reference probability, a first discrimination loss is determined.

[0103] Please refer to formula (3) to determine the first discriminant loss. One method.

[0104] (3) in, The sample images represent data and follow a data distribution. , The feature map represents the data distribution. , Characterize the semantic features of the sample image. Characterizing semantic features z and feature maps based on sample images The generated pseudo-image, Characterizes the discrimination probability for a sample image. It represents the probability of discrimination against pseudo-images.

[0105] Continuing with a feature map example, the image classification model comprises an editor network, a generator network, and a discriminator network. The editor network includes a first editor for extracting semantic features from the sample image and a second editor for extracting semantic features from the pseudo-image converted from the feature map. The generator network converts the feature map into a pseudo-image, and the discriminator network determines the discrimination probability of the input data. The second discriminator loss is determined based on the error between the second discrimination probability of the sample image's semantic features and the reference probability of the sample image, and the error between the second discrimination probability of the pseudo-image's semantic features and the reference probability of the pseudo-image.

[0106] Please refer to Figure 5E The sample image is processed by a first editor to extract a first semantic feature. This first semantic feature and the feature map are then processed by a generative network to generate a pseudo-image. The pseudo-image is then processed by a second editor to extract a second semantic feature. The first and second semantic features are used as input data to a discriminative network to obtain a second discrimination probability for each semantic feature. Based on the error between the second discrimination probability of the first semantic feature and its reference probability, and the error between the second discrimination probability of the second semantic feature and its reference probability, a second discrimination loss is determined.

[0107] Please refer to formula (4) to determine the second discriminant loss. One method.

[0108] (4) in, Characterize the sample image, The feature map represents the data distribution. , The semantic features of the sample image are characterized by a data distribution z. , Characterize pseudo-images generated based on semantic features z and feature maps of sample images. Characterizing the semantic features of pseudo-images Characterizes the discriminative probability of semantic features for a sample image. The probability of discriminating against the semantic features of a pseudo-image.

[0109] After obtaining the discriminant loss, the adversarial loss of the image classification model can be determined solely based on the discriminant loss, or the weighted sum of the first reconstruction loss, the second reconstruction loss, and the discriminant loss can be used as the adversarial loss of the image classification model after obtaining the first reconstruction loss, the second reconstruction loss, and the discriminant loss, etc. There are no specific restrictions.

[0110] Taking a feature map as an example, the image classification model includes an editor network, a generator network, and a discriminator network. The editor network includes a first editor for extracting semantic features from sample images and a second editor for extracting semantic features from pseudo-images converted from the feature map. The generator network is used to convert the feature map into pseudo-images, and the discriminator network is used to determine the discrimination probability of the input data. When determining the adversarial loss, the first reconstruction loss and the second reconstruction loss can be weighted first, then the weighted first and second reconstruction losses can be summed with the first and second discriminator losses to obtain the adversarial loss. Please refer to formula (5) for determining the adversarial loss. One way.

[0111] (5) in, Characterizes the first discriminant loss. Characterizes the second discriminant loss, Characterizing the first reconstruction loss, Characterizing the second reconstruction loss, The weighting coefficients characterizing the first reconstruction loss, The weighting coefficients characterizing the second reconstruction loss.

[0112] In this embodiment, disk constraints and reconstruction constraints for semantic features are added to reduce the impact of noise contained in the sample image on the pseudo image, thereby improving the quality of the generated pseudo image and the robustness of the generation network.

[0113] Fusion loss: A reference feature is obtained by weighted summation of the semantic features of the sample image and the semantic features of multiple feature maps. The fusion loss of the image classification model is determined based on the error between the combined feature and the reference feature.

[0114] After the server uses an image classification model to perform a weighted summation of the semantic features of the sample image and the semantic features of each feature map to obtain a comprehensive feature, it can then perform a weighted summation of the semantic features of the sample image and the semantic features of multiple feature maps to obtain a reference feature. Based on the error between the comprehensive feature and the reference feature, the fusion loss of the image classification model is determined.

[0115] When the error between the integrated features and the reference features is large, the features related to the target object obtained by the image classification model from each semantic feature differ significantly from the weighted sum of the semantic features. This indicates that the image classification model has not yet learned an accurate feature fusion method, and to some extent, it also indicates that the image classification model has not yet learned an accurate way to extract semantic features.

[0116] When the error between the integrated features and the reference features is small, it indicates that the features related to the target object obtained by the image classification model from each semantic feature are quite similar to the weighted sum of each semantic feature. This indicates that the image classification model has learned an accurate feature fusion method, and to a certain extent, it can also indicate that the image classification model has learned an accurate way to extract semantic features related to the target object.

[0117] Classification loss: The classification loss of the image classification model is determined based on the error between the obtained predicted classification and the classification label corresponding to the sample image.

[0118] After obtaining the predicted classification of the sample image, the server can determine the classification loss of the image classification model based on the error between the obtained predicted classification and the corresponding classification label of the sample image, thus realizing a supervised training process.

[0119] When the error between the predicted classification and the classification label is large, the image classification model's predicted classification for the sample image does not match the actual classification of the sample image. This indicates that the image classification model has not yet learned an accurate prediction method, and it can also indicate to some extent that the image classification model has not yet learned an accurate feature fusion method.

[0120] When the error between the predicted classification and the classification label is small, the image classification model's predicted classification for the sample image matches the actual classification of the sample image, indicating that the image classification model has learned an accurate prediction method. This can also indicate to some extent that the image classification model has learned an accurate feature fusion method.

[0121] As one implementation, the server can use a weighted sum of adversarial loss, fusion loss, and classification loss as the training loss for the image classification model. Thus, if the obtained training loss does not meet the training objective, the model parameters of the image classification model are adjusted, and the next iteration of training begins. When the obtained training loss meets the training objective, the image classification model is output, resulting in the trained target image classification model. The training objective can be to achieve convergence of the training loss.

[0122] Please refer to formulas (6), (7), (8), and (9) to determine the training loss of the image classification model. One way.

[0123] (6) (7) (8) (9) in, Characterizing adversarial loss, Characterizing fusion loss, Characterize the classification loss. Representing the i-th set in dataset X, dataset X can be represented as: ,So, It can characterize a set of sample images. arrive It can represent V-1 feature maps; Characterization based on The generated pseudo-images, the pseudo-image set Y can be represented as ; Characterization The semantic features, namely The semantic feature set can be represented as By adaptively weighting and summing the semantic features, a consistent comprehensive feature, namely the subspace Z, can be obtained, with each semantic feature as a factor. Configure an appropriate weight , It is learned through the continuous training process of the image classification model. satisfy . Characterize the predicted classification for the sample image. The classification label that represents the sample image. Characterizes the F-norm.

[0124] The server can also set corresponding training objectives for adversarial loss, fusion loss, and classification loss. If any of these losses fails to meet its corresponding training objective, the model parameters of the image classification model are adjusted, and the training proceeds to the next iteration. When the obtained adversarial loss, fusion loss, and classification loss satisfy the training objective, the image classification model is output, resulting in the trained target image classification model.

[0125] The following section uses the agricultural field as an example, with sample images of agricultural pests and diseases, such as grape leaf pests and diseases. It introduces the method for training the image classification model provided in this application embodiment, as well as the method for using the trained target image classification model. The trained target image classification model can be used to develop corresponding desktop or mobile view products and software for grape leaf pests and diseases.

[0126] Please refer to Figure 6A This diagram illustrates the architecture of the server and client components used for training, testing, and using the image classification model. The server includes a data acquisition module, which is used to establish a basic grape leaf dataset, perform preprocessing such as cleaning the original dataset, and can also be used to update or expand the dataset through crowdsourcing to obtain a sample image set.

[0127] The data augmentation module is used to build a multi-view data augmentation network. This multi-view data augmentation network can obtain multiple feature maps based on the sample image set. For example, based on the basic grape leaf disease and pest image dataset, a multi-view agricultural disease and pest dataset can be built, which can effectively repair missing view data and complete the expansion and enhancement of the sample image set.

[0128] The training and testing module is used to train an image classification model on the augmented dataset, i.e., on the sample image set and multiple feature maps, and to test the effectiveness of the trained target image classification model.

[0129] The model distribution module is used to distribute the target image classification model to the corresponding client after obtaining a valid target image classification model on the server.

[0130] The model update module is used to collect feedback data from clients in real time through online learning, and optimize the target image classification model based on the collected feedback data.

[0131] The client includes a model loading module, which receives the target image classification model sent by the server and loads the target image classification model into memory.

[0132] The model uses a module to classify image data when inputting image data to the client via taking a photo (mobile device) or importing images (mobile device or web device), thereby obtaining the type and severity of the pest or disease to which the image data belongs.

[0133] The data feedback module is used to upload feedback data to the server based on actual usage, so as to help the server optimize the target image classification model.

[0134] The model update module is used to periodically communicate with the server to detect whether there is a model update for the target image classification model, and download the updated target image classification model when network environment or storage conditions permit.

[0135] Please refer to Figure 6B This is a schematic diagram illustrating the interaction between the server and client used for training and using an image classification model.

[0136] S601, the server collects basic grape leaf datasets and establishes a sample image set.

[0137] S602: The server employs multiple image processing strategies to extract pixel-level features from the sample image set, thereby obtaining multiple feature sets. In other words, based on the basic grape leaf disease and pest image dataset, a multi-view agricultural disease and pest dataset is established to expand and enhance the sample image set.

[0138] S603, the server generates corresponding pseudo-image sets based on multiple feature map sets, thereby expanding and enhancing the sample image set. Based on the semantic features corresponding to each sample image in the sample image set, and with each feature map set, the server generates a pseudo-image set, expanding the sample image set to several times its original size.

[0139] S604: The server builds the image classification model to be trained, and performs multiple rounds of iterative training on the image classification model to be trained based on the sample image set, multiple feature map sets and multiple pseudo image sets, and outputs the trained target image classification model.

[0140] S605: The server sends the target image classification model to the client, and the client loads the received target image classification model into memory.

[0141] S606 allows the client to call the target image classification model when it obtains an image to be classified, and determine the target classification of the image.

[0142] Taking a sample image as an example, let's treat the sample image as an RGB feature map, denoted as... ,in, The i-th feature element in the RGB feature map is represented; a texture feature processing strategy is used to extract pixel-level features from the sample image to obtain a texture feature map, denoted as . ,in, The i-th feature element in the texture feature map is represented; a local binary feature processing strategy is used to extract pixel-level features from the sample image to obtain a local binary feature map, denoted as . ,in, V represents the i-th feature element in the local binary feature map, and V represents the total number of feature maps. Other image processing strategies are employed to obtain other feature maps, thus obtaining multiple feature maps corresponding to the sample image.

[0143] The image classification model is trained iteratively based on multiple feature maps.

[0144] Please refer to Figure 7 This is a schematic diagram illustrating the principle of training an image classification model. The image classification model includes an encoder network E, which is used to extract semantic features. The encoder network contains multiple encoders, each consisting of M fully connected layers and N fully connected layers. The M fully connected layers in each encoder have different model parameters, while the N fully connected layers in each encoder have the same model parameters.

[0145] The image classification model also includes a decoder network D, which is used to generate pseudo-images; the network structure of the decoder network is the opposite of that of the encoder network.

[0146] Image classification models also include adaptive fusion networks for feature fusion.

[0147] Image classification models also include a classifier used to determine the predicted classification.

[0148] The encoder network extracts semantic features from multiple feature maps, where each feature map contains semantic features. The semantic features of the i-th feature map are denoted as . ,in, The r-th feature element in the semantic features of the i-th feature map.

[0149] Adaptive fusion networks fuse the semantic features of multiple feature maps to obtain comprehensive features. This can be achieved by weighted summation of the feature elements at corresponding positions of the semantic features in each feature map to obtain a comprehensive feature. , recorded as Please refer to formula (10).

[0150] (10)

[0151] in, The semantic features representing the corresponding i-th feature map The weight.

[0152] Decoder networks are based on comprehensive features and the semantic features of the sample images, i.e. It can generate a pseudo-image corresponding to each feature map, and the i-th pseudo-image is denoted as . ,in, The m-th feature element in the i-th pseudo-image is represented.

[0153] Classifiers based on comprehensive features The predicted classification of the sample image is determined. The adversarial loss is determined based on the error between each feature map and its corresponding pseudo-image; the fusion loss is determined based on the error between the combined feature and the weighted sum of the semantic features of each feature map; and the classification loss is determined based on the error between the predicted classification and the classification label of the sample image. The training loss of the image classification model is determined based on the weighted sum of the adversarial loss, fusion loss, and classification loss. If the training loss does not converge, the model parameters of the image classification model are adjusted, and the next round of training begins. When the training loss converges, the trained target image classification model is output.

[0154] Based on multiple pixel-level features of images, i.e., multiple views, deep learning networks are used to effectively fuse and reclassify these multiple views, breaking through the bottleneck limitations of existing pest and disease image classification methods and improving the accuracy and reliability of pest and disease image classification technology. By achieving data augmentation through multiple views, a multi-view agricultural pest and disease dataset can be constructed on incomplete agricultural pest and disease datasets for training image classification models, effectively solving the problem of missing data.

[0155] Based on the same inventive concept, embodiments of this application provide an apparatus for training an image classification model, capable of achieving the functions corresponding to the aforementioned method for training an image classification model. Please refer to... Figure 8 The device includes an acquisition module 801 and a processing module 802, wherein: Acquisition module 801: Used to perform pixel-level feature extraction on the sample image set using multiple image processing strategies to obtain multiple feature map sets. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set. Each sample image corresponds to a feature map in each feature map set. Processing module 802: Used to perform multiple rounds of iterative training on the image classification model to be trained based on the sample image set and multiple feature maps obtained, and output the trained target image classification model. Each round of iteration includes: The processing module 802 is also used to: extract the semantic features of the selected sample image and the corresponding multiple feature maps respectively, and perform feature fusion on the obtained semantic features to obtain the comprehensive features of the sample image; The processing module 802 is also used to: determine the predicted classification of the sample image based on the comprehensive features; The processing module 802 is also used to adjust the model parameters of the image classification model based on multiple feature maps, integrated features, and predicted classification.

[0156] In one possible embodiment, the processing module 802 is specifically used for: Based on multiple feature maps, comprehensive features, and predicted classification, the adversarial loss, fusion loss, and classification loss of the image classification model are determined. The adversarial loss represents the accuracy of extracting semantic features, the fusion loss represents the accuracy of feature fusion, and the classification loss represents the accuracy of determining the predicted classification. The model parameters of the image classification model are adjusted based on adversarial loss, fusion loss, and classification loss.

[0157] In one possible embodiment, the processing module 802 is specifically used for: For multiple feature maps, perform the following operations respectively: Based on the semantic features of the feature map and the semantic features of the sample image, the feature map is converted into a pseudo image, where the pseudo image and the sample image represent the same kind of pixel-level features. Based on sample images and pseudo images, the adversarial loss of the image classification model is determined.

[0158] In one possible embodiment, the processing module 802 is specifically used for: Extract semantic features from pseudo-images and determine the first reconstruction loss of the image classification model based on the error between the semantic features of the sample images and the semantic features of the pseudo-images. The second reconstruction loss of the image classification model is determined based on the error between sample images and pseudo images. Based on the first reconstruction loss and the second reconstruction loss, the adversarial loss of the image classification model is determined.

[0159] In one possible embodiment, the image classification model includes a discriminant network used to predict the discrimination probability of the input data being a sample image or semantic features of a sample image, and the input data having associated reference probabilities; then the processing module 802 is specifically used for: The sample image and the pseudo image are used as input data to the discriminant network to predict the corresponding first discrimination probability. The semantic features of the sample image and the semantic features of the pseudo image are used as input data to the discriminant network to predict the corresponding second discrimination probability. Based on the error between each obtained first and second discrimination probability and its corresponding reference probability, the discrimination loss of the image classification model is determined. The weighted sum of the first reconstruction loss, the second reconstruction loss, and the discriminant loss is used as the adversarial loss of the image classification model.

[0160] In one possible embodiment, the processing module 802 is specifically used for: The semantic features of the sample image and the semantic features of multiple feature maps are weighted and summed to obtain the reference features; The fusion loss of the image classification model is determined based on the error between the integrated features and the reference features.

[0161] In one possible embodiment, the processing module 802 is specifically used for: The training loss of the image classification model is obtained by weighted summation of the adversarial loss, fusion loss, and classification loss. If the obtained training loss does not meet the training objective, adjust the model parameters of the image classification model and proceed to the next round of iteration training.

[0162] In one possible embodiment, each feature set is one of a grayscale feature set, a texture feature set, a local binary feature set, or an oriented gradient histogram set.

[0163] Please refer to Figure 9 The aforementioned apparatus for training the image classification model can run on a computer device 900. The current and historical versions of the data storage program, as well as the application software corresponding to the data storage program, can be installed on the computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, which includes a display panel 941 for displaying a user-interactive interface, etc.

[0164] In one possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0165] The processor 980 is used to read a computer program and then execute the methods defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this application.

[0166] The memory 920 generally includes main memory and secondary storage. Main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. Secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 920 is used to store computer programs and other data. The computer programs include applications corresponding to each client, and other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, program instructions are stored in the memory 920, and the processor 980 executes the program instructions in the memory 920 to implement any of the methods described in the preceding figures.

[0167] The aforementioned display unit 940 is used to receive input digital information, character information, or contact touch operations / non-contact gestures, and to generate signal inputs related to user settings and function control of the computer device 900. Specifically, in this embodiment, the display unit 940 may include a display panel 941. The display panel 941, for example, is a touch screen, which can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0168] In one possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980. It can also receive and execute commands from the processor 980.

[0169] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may also include an input unit 930. The input unit 930 may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0170] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, and a pressure sensor. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, for example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0171] As one embodiment, the number of processors 980 can be one or more, and the processors 980 and the memory 920 can be coupled together or relatively independent.

[0172] As one example, Figure 9 The processor 980 in the middle can be used to implement, for example Figure 8 The functions of the acquisition module 801 and the processing module 802 in the process.

[0173] As one example, Figure 9 The processor 980 in the text can be used to implement the functions of the server or terminal devices discussed above.

[0174] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of software products, for example, through a computer program product. This computer program product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0176] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0177] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for training an image classification model, characterized in that, include: Multiple image processing strategies are employed to extract pixel-level features from the sample image set, resulting in multiple feature map sets. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set. Each sample image corresponds to one feature map in each feature map set. Based on the sample image set and the obtained multiple feature maps, the image classification model to be trained is subjected to multiple rounds of iterative training, outputting the trained target image classification model. Each round of iteration includes: Semantic features of the selected sample image and its corresponding multiple feature maps are extracted respectively, and the obtained semantic features are fused to obtain the comprehensive features of the sample image. Based on the comprehensive features, the predicted classification of the sample image is determined; For the multiple feature maps, the sample image and the pseudo image obtained by transforming the feature maps based on the semantic features of the sample image are respectively used as input data to the discriminant network in the image classification model to predict the corresponding first discrimination probability. The semantic features of the sample image and the pseudo image are respectively used as input data to the discriminant network to predict the corresponding second discrimination probability. The discriminant network is used to predict the probability that the input data is the sample image or the semantic features of the sample image, and each input data has an associated reference probability. Based on the error between each obtained first discrimination probability and each second discrimination probability and their corresponding reference probability, a discrimination loss is determined; and based on the error between the semantic features of the sample image and the semantic features of the pseudo image, a first reconstruction loss of the image classification model is determined; and based on the error between the sample image and the pseudo image, a second reconstruction loss of the image classification model is determined. An adversarial loss is determined based on the first reconstruction loss, the second reconstruction loss, and the discriminative loss. A fusion loss and a classification loss are determined based on the multiple feature maps, the integrated features, and the predicted classification, in order to adjust the model parameters of the image classification model.

2. The method according to claim 1, characterized in that, The step of determining adversarial loss based on the first reconstruction loss, the second reconstruction loss, and the discriminative loss, and determining fusion loss and classification loss based on the multiple feature maps, the integrated features, and the predicted classification, to adjust the model parameters of the image classification model, includes: The adversarial loss of the image classification model is determined by a weighted sum of the first reconstruction loss, the second reconstruction loss, and the discriminative loss; the adversarial loss characterizes the accuracy of semantic feature extraction. The model parameters of the image classification model are adjusted based on the adversarial loss, the fusion loss, and the classification loss; the fusion loss represents the accuracy of feature fusion, and the classification loss represents the accuracy of determining the predicted classification.

3. The method according to claim 2, characterized in that, The determination of the fusion loss of the image classification model based on the multiple feature maps, the integrated features, and the predicted classification includes: The semantic features of the sample image and the semantic features of the multiple feature maps are weighted and summed to obtain reference features; The fusion loss of the image classification model is determined based on the error between the integrated features and the reference features.

4. The method according to claim 2, characterized in that, Based on the adversarial loss, the fusion loss, and the classification loss, the model parameters of the image classification model are adjusted, including: The training loss of the image classification model is obtained by weighted summing of the adversarial loss, the fusion loss, and the classification loss. If the obtained training loss does not meet the training objective, the model parameters of the image classification model are adjusted, and the next round of iterative training begins.

5. The method according to any one of claims 1 to 4, characterized in that, Each feature set is one of the following: grayscale feature set, texture feature set, local binary feature set, or oriented gradient histogram set.

6. An apparatus for training an image classification model, characterized in that, include: Acquisition module: Used to extract pixel-level features from the sample image set using various image processing strategies to obtain multiple feature map sets. Each image processing strategy corresponds to a type of pixel-level feature and a corresponding feature map set. Each sample image corresponds to one feature map in each feature map set. Processing module: Used to perform multiple rounds of iterative training on the image classification model to be trained based on the sample image set and the obtained multiple feature maps, and output the trained target image classification model, wherein each round of iteration includes: The processing module is also used to: extract the semantic features of the selected sample image and the corresponding multiple feature maps respectively, and perform feature fusion on the obtained semantic features to obtain the comprehensive features of the sample image; The processing module is further configured to: determine the predicted classification of the sample image based on the comprehensive features; The processing module is also used for, For the multiple feature maps, the sample image and the pseudo image obtained by transforming the feature maps based on the semantic features of the sample image are respectively used as input data to the discriminant network in the image classification model to predict the corresponding first discrimination probability. The semantic features of the sample image and the pseudo image are respectively used as input data to the discriminant network to predict the corresponding second discrimination probability. The discriminant network is used to predict the probability that the input data is the sample image or the semantic features of the sample image, and each input data has an associated reference probability. The processing module is further configured to: determine a discrimination loss based on the error between each obtained first discrimination probability and each second discrimination probability and their corresponding reference probability; determine a first reconstruction loss of the image classification model based on the error between the semantic features of the sample image and the semantic features of the pseudo image; and determine a second reconstruction loss of the image classification model based on the error between the sample image and the pseudo image. The processing module is further configured to determine an adversarial loss based on the first reconstruction loss, the second reconstruction loss, and the discriminative loss, and to determine a fusion loss and a classification loss based on the multiple feature maps, the integrated features, and the predicted classification, so as to adjust the model parameters of the image classification model.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 5.

8. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 5 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 5.

Images