Image classification processing method, device and equipment, and storage medium

Abstract

The application relates to a classification processing method and device for images, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting a first image sample into an image processing model to obtain a first feature map, a second feature map, a first prediction result and a second prediction result; obtaining first classification information and second classification information based on the first feature map, the second feature map, the first prediction result and the second prediction result; obtaining a loss function value; and performing parameter updating on the image processing model based on the loss function value. The above scheme improves the accuracy of the image processing model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for image classification processing. Background Technology

[0002] Semi-supervised learning aims to solve the problem of relying on a large number of labeled samples in fully supervised learning by using unlabeled data, and has recently attracted much attention in the field of image classification.

[0003] In related technologies, in the scenario of image classification and processing, when performing semi-supervised learning, a single Convolutional Neural Network (CNN) can be used for feature mining, and small perturbations can be added to the input image or network parameters to constrain the consistency of the prediction results.

[0004] However, when semi-supervised learning methods in related technologies are used for image classification, their ability to mine the distribution of categories is inconsistent, which affects the accuracy of the trained model. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for image classification processing, which can improve the accuracy of models used for image classification processing. The technical solution is as follows.

[0006] On the one hand, a method for image classification processing is provided, the method comprising:

[0007] A first image sample is input into an image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result. The first feature map is obtained by a first processing branch in the image processing model performing feature extraction on the first image sample. The second feature map is obtained by a second processing branch in the image processing model performing feature extraction on the first image sample. The first prediction result indicates the classification to which each pixel in the first sample image belongs, as predicted by the first processing branch. The second prediction result indicates the classification to which each pixel in the first sample image belongs, as predicted by the second processing branch. The first image sample is an unlabeled image sample.

[0008] Based on the first feature map, the second feature map, the first prediction result, and the second prediction result, first classification information and second classification information are obtained; the first classification information is used to indicate the classification of the first image sample by the first processing branch; the second classification information is used to indicate the classification of the first image sample by the second processing branch.

[0009] Obtain the loss function value; the loss function value includes a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information;

[0010] The image processing model is updated based on the loss function value;

[0011] The image processing model, trained to convergence, is used to process the input target image and predict the classification of each pixel in the target image.

[0012] Furthermore, an image classification processing apparatus is provided, the apparatus comprising:

[0013] An image input module is used to input a first image sample into an image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result. The first feature map is a feature map obtained by a first processing branch in the image processing model performing feature extraction on the first image sample. The second feature map is a feature map obtained by a second processing branch in the image processing model performing feature extraction on the first image sample. The first prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the first processing branch. The second prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the second processing branch. The first image sample is an unlabeled image sample.

[0014] The classification information acquisition module is used to acquire first classification information and second classification information based on the first feature map, the second feature map, the first prediction result, and the second prediction result; the first classification information is used to indicate the classification of the first image sample by the first processing branch; the second classification information is used to indicate the classification of the first image sample by the second processing branch.

[0015] The loss function value acquisition module is used to acquire loss function values; the loss function value includes a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information;

[0016] The parameter update module is used to update the parameters of the image processing model based on the loss function value.

[0017] The image processing model, trained to convergence, is used to process the input target image and predict the classification of each pixel in the target image.

[0018] In one possible implementation, the classification information acquisition module is used to,

[0019] Based on the first prediction result and the second prediction result, the classification label of each pixel in the first image sample is obtained; the classification label is used to indicate the classification of the corresponding pixel;

[0020] The first feature map is transformed into a first manifold feature, wherein the first manifold feature includes a feature corresponding to each pixel in the first image sample;

[0021] Based on the first manifold feature and the classification label of each pixel, the first classification information is obtained;

[0022] The second feature map is transformed into a second manifold feature, the second manifold feature including a feature corresponding to each pixel in the first image sample;

[0023] Based on the second manifold feature and the classification label of each pixel, the second classification information is obtained.

[0024] In one possible implementation, the classification information acquisition module is used to,

[0025] Based on the first manifold feature and the classification label of each pixel, the prototype corresponding to each category in the first manifold feature is obtained; the prototype is the average result of the feature belonging to the corresponding category.

[0026] Based on the prototypes of each category in the first manifold features, obtain the first category information;

[0027] The step of obtaining the second classification information based on the second manifold feature and the classification label of each pixel includes:

[0028] Based on the second manifold feature and the classification label of each pixel, obtain the prototype of each category in the second manifold feature;

[0029] Based on the prototypes of each category in the second manifold features, the second classification information is obtained.

[0030] In one possible implementation, the classification information acquisition module is used to,

[0031] Based on the features of the first manifold, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the first classification information;

[0032] The step of obtaining the second classification information based on the prototypes corresponding to each classification in the second manifold features includes:

[0033] Based on the second manifold features, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the second classification information.

[0034] In one possible implementation, the loss function value acquisition module is used to acquire the F-norm between the first classification information and the second classification information, as a sub-value of the first loss function.

[0035] In one possible implementation, the loss function value acquisition module is further configured to:

[0036] Obtain the second loss function sub-value from the loss function values. The second loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the first manifold features.

[0037] Obtain the third loss function sub-value from the loss function values. The third loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the second manifold features.

[0038] In one possible implementation, the loss function value acquisition module is further configured to:

[0039] From each of the categories, select the K features with the highest attention scores from the candidate features corresponding to each category and add them to the storage space of the corresponding category; K is an integer greater than or equal to 1; the candidate features are the features extracted by the image processing model from the second image sample and transformed into a manifold; the second image sample is a labeled image sample;

[0040] Obtain the fourth loss function sub-value from the loss function values. The fourth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the first manifold features and the features in the storage space of each category.

[0041] Obtain the fifth loss function sub-value from the loss function values. The fifth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the second manifold features and the features in the storage space of each category.

[0042] In one possible implementation, the device further includes:

[0043] The result acquisition module is used to acquire the third prediction result and the fourth prediction result output by the image processing model after processing the second image sample; the third prediction result is used to indicate the category to which each pixel in the second sample image belongs, as predicted by the first processing branch; the fourth prediction result is used to indicate the category to which each pixel in the second sample image belongs, as predicted by the second processing branch.

[0044] The loss function value acquisition module is further configured to acquire a sixth loss function sub-value in the loss function value based on the third prediction result, the fourth prediction result, and the annotation information of the second image sample. The sixth loss function sub-value is used to indicate the differences between the third prediction result, the fourth prediction result, and the annotation information. The annotation information is used to indicate the classification of each pixel in the second image sample.

[0045] In one possible implementation, the loss function value acquisition module is further configured to:

[0046] Obtain the seventh loss function sub-value from the loss function values; the seventh loss function sub-value is used to indicate the difference between the first prediction result and the classification label of each pixel;

[0047] Obtain the eighth loss function sub-value from the loss function values; the eighth loss function sub-value is used to indicate the difference between the second prediction result and the classification label of each pixel.

[0048] In one possible implementation, the classification information acquisition module is used to,

[0049] Based on the first prediction result, obtain the first entropy value of the prediction result of the first processing branch for each pixel in the first image sample;

[0050] Based on the second prediction result, obtain the second entropy value of the prediction result of the second processing branch for each pixel in the first image sample;

[0051] The first entropy value is used as the confidence level of the prediction result corresponding to each pixel in the first prediction result, and the second entropy value is used as the confidence level of the prediction result corresponding to each pixel in the second prediction result. The classification label of each pixel in the first image sample is extracted from the first prediction result and the second prediction result.

[0052] In one possible implementation, the first processing branch and the second processing branch are heterogeneous networks.

[0053] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the above-described image classification processing method.

[0054] In another aspect, a computer-readable storage medium is provided, wherein at least one computer instruction is stored therein, the at least one computer instruction being loaded and executed by a processor to implement the above-described image classification processing method.

[0055] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned image classification processing method.

[0056] The beneficial effects of the technical solutions provided in this application include at least the following:

[0057] During training, the image processing model extracts features and predicts for the first image sample through two processing branches. Based on the feature maps and prediction results extracted by the two processing branches, the classification of the first image sample by the two processing branches is obtained. The difference between the classifications of the first image sample by the two processing branches is used as part of the loss function to update the parameters of the image processing model. Since the loss calculated by the classifications of the first image sample by the two processing branches is introduced, the optimization process of the image processing model fully considers the rich category distribution, so that the trained image processing model can extract sufficiently rich features, thereby improving the accuracy of the image processing model.

[0058] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0060] Figure 1 This is a system configuration diagram of the image classification processing system according to various embodiments of this application;

[0061] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of a method for classifying images.

[0062] Figure 3 This is a schematic flowchart of an image classification processing method provided in an exemplary embodiment;

[0063] Figure 4 This is a diagram showing the effect of image segmentation provided in this application;

[0064] Figure 5 This is a schematic diagram illustrating the principle of a model framework involved in this application;

[0065] Figure 6 This is a system framework diagram of the image segmentation service involved in the embodiments of this application;

[0066] Figures 7 to 11 This is a schematic diagram of the experimental results involved in this application;

[0067] Figure 12 This is a structural block diagram of an image classification processing apparatus according to an exemplary embodiment;

[0068] Figure 13 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0069] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0070] Before describing the various embodiments shown in this application, several concepts involved in this application will be introduced first.

[0071] 1) AI (Artificial Intelligence)

[0072] AI (Artificial Intelligence) is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Artificial intelligence technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0073] 2) Computer Vision (CV) technology

[0074] Computer vision is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, detecting, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0075] 3) ML (Machine Learning)

[0076] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0077] 4) Medical Cloud

[0078] Medical cloud refers to a cloud platform for healthcare services created using cloud computing, mobile technology, multimedia, wireless communication, big data, and the Internet of Things, combined with medical technology. This platform enables the sharing of medical resources and expands the scope of medical services. Because of the application of cloud computing technology, medical cloud improves the efficiency of medical institutions and facilitates access to healthcare for residents. Features such as hospital appointment booking and medical insurance are products of the integration of cloud computing and the medical field. Medical cloud also offers advantages such as data security, information sharing, dynamic expansion, and a global deployment.

[0079] Please refer to Figure 1 This illustrates a system configuration diagram of an image classification processing system according to various embodiments of this application. Figure 1As shown, the system includes an image acquisition device 120, a terminal 140, and a server 160; optionally, the system may also include a database 180.

[0080] Image acquisition device 120 can be a device used for acquiring images.

[0081] For example, taking the image acquisition device 120 as a device for acquiring medical images, the aforementioned image acquisition device 120 can be a medical scanning device, such as a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, and so on. Correspondingly, the images acquired by the image acquisition device 120 refer to three-dimensional scan images, such as CT images, MRI images, and so on.

[0082] Alternatively, the image acquisition device 120 can also be a device for acquiring other images, such as a camera device for acquiring two-dimensional images.

[0083] The image acquisition device 120 may include an image output interface, such as a Universal Serial Bus (USB) interface, a High Definition Multimedia Interface (HDMI) interface, or an Ethernet interface, etc.; or, the above-mentioned image output interface may also be a wireless interface, such as a Wireless Local Area Network (WLAN) interface, a Bluetooth interface, etc.

[0084] Accordingly, depending on the type of image output interface, operators can export images in various ways. For example, images can be imported to terminal 140 via wired or short-range wireless means, or images can be imported to terminal 140 or server 160 via local area network or Internet.

[0085] Terminal 140 can be a terminal device with certain processing capabilities and interface display functions. For example, terminal 140 can be a mobile phone, tablet computer, e-book reader, smart glasses, laptop computer, desktop computer, etc.

[0086] Terminal 140 may include terminals used by developers and terminals used by users (such as medical personnel).

[0087] When terminal 140 is implemented as a terminal for developers, developers can use terminal 140 to develop machine learning models for image classification and deploy the machine learning models to server 160 or to terminals used by users (such as medical personnel).

[0088] When terminal 140 is implemented as a user terminal, an application can be installed in terminal 140 to obtain and present the results of image classification processing. After terminal 140 obtains the image acquired by image acquisition device 120, it can obtain the processing results obtained by image classification processing through the above application and present the processing results for user reference, such as for doctors to perform pathological diagnosis, surgical navigation and other operations.

[0089] exist Figure 1 In the system shown, terminal 140 and image acquisition device 120 are physically separate entities. Alternatively, in another possible implementation, when terminal 140 is implemented as a terminal for use by a user (e.g., medical personnel), terminal 140 and image acquisition device 120 can also be integrated into a single entity; for example, terminal 140 can be a terminal device with medical 3D image acquisition function.

[0090] Among them, server 160 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0091] The aforementioned server 160 may be a server that provides background services for applications installed in terminal 140. This background server may be responsible for application version management, background processing of images obtained by the application and returning processing results, background training of machine learning models developed by developers, etc.

[0092] The aforementioned database 180 can be a Redis database, or it can be any other type of database. Database 180 is used to store various types of data.

[0093] Optionally, terminal 140 and server 160 are connected via a communication network. Optionally, image acquisition device 120 and server 160 are connected via a communication network. Optionally, this communication network can be a wired network or a wireless network.

[0094] Optionally, the system may also include a management device ( Figure 1(Not shown), the management device is connected to the server 160 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0095] Figure 2 This is a flowchart illustrating a method for image classification processing according to an exemplary embodiment. The method can be executed by a computer device, such as a server, a terminal, or a combination of both, wherein the server can be one of the aforementioned... Figure 1 The server 160 in the illustrated embodiment can be the terminal described above. Figure 1 The terminal 140 shown in the embodiment is used by the developer. This computer device can be implemented as a model training device for performing model training. Figure 2 As shown, the image classification processing method may include the following steps.

[0096] Step 210: Input the first image sample into the image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result; the first feature map is the feature map obtained by the first processing branch of the image processing model performing feature extraction on the first image sample, and the second feature map is the feature map obtained by the second processing branch of the image processing model performing feature extraction on the first image sample; the first prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the first processing branch; the second prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the second processing branch; the first image sample is an unlabeled image sample.

[0097] The image processing model described above includes a first processing branch and a second processing branch, and the first processing branch and the second processing branch can process the first image sample respectively.

[0098] For example, the first and second processing branches mentioned above can be two parallel network branches in an image processing model.

[0099] For example, the image processing model described above could be a machine learning model based on a dual-student architecture.

[0100] In one possible implementation, the first processing branch and the second processing branch can be heterogeneous networks. That is, the network architectures of the first processing branch and the second processing branch are different.

[0101] In machine learning, a network's ability to extract features is related to its architecture; different architectures extract different types of features. For example, CNNs are good at extracting local features, but because their receptive field is very limited, they neglect the ability to aggregate global context and local features. Therefore, CNNs typically only focus on local information and cannot combine long-range information. On the other hand, Transformers have achieved significant performance in visual tasks, benefiting from their multi-head self-attention ability to capture long-range dependencies. However, Transformers are not as good as CNNs at capturing local details.

[0102] In the scheme shown in the embodiments of this application, two heterogeneous networks are used as two processing branches in the image processing model, thereby combining the ability of multiple network architectures to extract features of different scales / types, thereby improving the accuracy of the entire model.

[0103] For example, the first and second processing branches mentioned above can be CNN network branches and Transformer network branches, respectively, so that the image processing model can extract local and global features at the same time, thereby improving the accuracy of the entire model.

[0104] In some embodiments, the first feature map mentioned above may be a feature map before the output layer (also called the classification layer) of the first processing branch (that is, the feature map output by the layer before the output layer of the first processing branch); for example, the first processing branch may be divided into a feature extraction layer and a classification layer according to its function. The feature extraction layer is used to process the input image to obtain a feature map for formal classification, which can be the first feature map mentioned above; the first feature map is input into the classification layer, and after processing by the classification layer, the first prediction result can be obtained.

[0105] Alternatively, the first feature map mentioned above can also be a feature map output by other network layers other than the layer preceding the output layer of the first processing branch. For example, the first feature map mentioned above can also be a feature map output by the second or third layer preceding the output layer of the first processing branch.

[0106] Similarly, the second feature map can be a feature map before the output layer of the second processing branch, or it can be a feature map output by other network layers other than the layer preceding the output layer of the second processing branch.

[0107] The aforementioned first prediction result can be the prediction result output by the output layer of the first processing branch. For example, the first prediction result can include the probability distribution of each pixel in the first image sample belonging to each preset category. For instance, if there are 5 preset pixel categories, the first image sample contains 256×256 pixels, and the first prediction result contains 256×256 probability distributions, each probability distribution corresponding to one pixel, and each probability distribution contains 5 probability values, each probability value being the probability that the corresponding pixel belongs to a category.

[0108] Similarly, the second prediction result mentioned above can be the prediction result output by the output layer of the second processing branch. For example, the second prediction result can include the probability distribution of each pixel in the first image sample belonging to each preset category.

[0109] Step 220: Based on the first feature map, the second feature map, the first prediction result, and the second prediction result, obtain the first classification information and the second classification information; the first classification information is used to indicate the classification of the first image sample by the first processing branch; the second classification information is used to indicate the classification of the first image sample by the second processing branch.

[0110] Since the first processing branch and the second processing branch are two different network branches, the extraction of features by the first processing branch and the second processing branch are also different. Accordingly, the classification of the same image / image sample by the first processing branch and the second processing branch are also different. In order to support the consideration of different classification distributions in the future, the computer device can obtain the classification of the first image sample by the first processing branch (represented by the first classification information) and the classification of the first image sample by the second processing branch (represented by the second classification information) based on the first feature map, the second feature map, the first prediction result and the second prediction result.

[0111] Step 230: Obtain the loss function value; the loss function value includes a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information.

[0112] In this embodiment, the computer device can calculate a first loss function sub-value based on the difference between the first classification information and the second classification information, and use the first loss function sub-value as part of the loss function value of the image processing model. Since the first loss function sub-value takes into account the rich classification distribution provided by the two processing branches, and can also bring the features extracted by the two processing branches closer together, model training using the first loss function sub-value can enable both processing branches to extract sufficiently rich features, thereby improving the accuracy of the model.

[0113] Step 240: Update the parameters of the image processing model based on the loss function value.

[0114] In some embodiments, the parameters of the image processing model can be updated in a semi-supervised manner based on the loss function value.

[0115] The image processing model trained to convergence is used to process the input target image and predict the classification of each pixel in the target image.

[0116] In some embodiments, since the image processing model contains two processing units with the same function (i.e., a first processing unit and a second processing unit), during the application of the model, the image processing model can process the target image through one of the first processing unit and the second processing unit to predict the classification of each pixel in the target image; or, the image processing model can also process the target image through the first processing unit and the second processing unit respectively, and take the average of the prediction results of the first processing unit and the second processing unit to obtain the classification of each pixel in the target image.

[0117] In one possible implementation, the computer device may also acquire a third prediction result and a fourth prediction result output by the image processing model after processing the second image sample; the third prediction result is used to indicate the classification to which each pixel in the second sample image belongs, as predicted by the first processing branch; the fourth prediction result is used to indicate the classification to which each pixel in the second sample image belongs, as predicted by the second processing branch.

[0118] Step 230 above may also include:

[0119] Based on the third prediction result, the fourth prediction result, and the annotation information of the second image sample, the sixth loss function sub-value is obtained from the loss function value. The sixth loss function sub-value is used to indicate the difference between the third prediction result, the fourth prediction result, and the annotation information, respectively. The annotation information is used to indicate the classification of each pixel in the second image sample.

[0120] In this embodiment of the application, when the computer device updates the parameters of the image processing model, it also uses a second sample image that has been labeled with the classification of each pixel. After the second sample image is input into the image processing model, the two processing branches of the image processing model will output a prediction result (i.e. the third prediction result and the fourth prediction result mentioned above). Then, based on the difference between the prediction results output by the two processing branches and the labeling information of the second sample image, the sixth loss function sub-value for full supervision is calculated.

[0121] For example, the third and fourth prediction results mentioned above are the probability distributions of each pixel in the input second sample image in each category. Taking the possibility that each pixel may have 3 categories (e.g., category 1, category 2, category 3) as an example, the probability distribution of a pixel contains three probability values ​​(e.g., {0.8, 0.1, 0.1}), which represent the probability that the pixel belongs to a category (e.g., the probability of the pixel belonging to category 1 is 0.8, the probability of belonging to category 2 is 0.1, and the probability of belonging to category 3 is 0.1). In addition, the computer device can also obtain the probability distribution of each pixel labeled in the second sample image by the category labeled in the second sample image. For example, if a pixel is labeled as category 1, then the probability distribution of the pixel can be {1, 0, 0}, or if a pixel is labeled as category 2, then the probability distribution of the pixel can be {0, 1, 0}. The computer device can calculate the sixth loss function sub-value based on the difference between the probability distribution of each pixel in the third and fourth prediction results of the second sample image and the probability distribution of each pixel labeled above.

[0122] For example, the third and fourth prediction results mentioned above are the predicted classifications of each pixel in the input second sample image. The computer device can calculate the sixth loss function sub-value based on the difference between the predicted classifications of each pixel in the third and fourth prediction results and the labeled classifications of each pixel.

[0123] In some embodiments, the loss function used by the computer device to calculate the sixth loss function subvalue may include, but is not limited to, at least one of the following loss functions: cross-entropy loss function, Dice loss function, Focal loss function, Tversky loss function, etc.

[0124] In some embodiments, the sixth loss function subvalue described above may further include the loss values ​​corresponding to the two processing branches of the image processing model.

[0125] For example, the computer device can calculate the loss value of the sixth loss function sub-value corresponding to the first processing branch by comparing the probability distribution / predicted classification of each pixel in the third prediction result with the probability distribution / classification of each pixel labeled above; and calculate the loss value of the sixth loss function sub-value corresponding to the second processing branch by comparing the probability distribution / predicted classification of each pixel in the fourth prediction result with the probability distribution / classification of each pixel labeled above.

[0126] In semi-supervised learning, in addition to using unlabeled first image samples, labeled second image samples are also needed for auxiliary learning. This enables the image processing model to accurately predict which of the pre-labeled categories a pixel in the input image belongs to, thus ensuring the accuracy of the model's classification.

[0127] In summary, the scheme shown in this application embodiment, during the training process, extracts features and predicts the first image sample through two processing branches in the image processing model, respectively. Based on the feature maps and prediction results extracted by the two processing branches for the first image sample, the classification of the first image sample by the two processing branches is obtained. The difference between the classifications of the first image sample by the two processing branches is used as part of the loss function to update the parameters of the image processing model. Since the loss calculated by the classifications of the first image sample by the two processing branches is introduced, the optimization process of the image processing model fully considers the rich category distribution, thereby enabling the trained image processing model to extract sufficiently rich features, thus improving the accuracy of the image processing model.

[0128] Based on the above Figure 2 After the image processing model shown in the illustrated embodiment is trained, it can be applied to various tasks that rely on classifying individual pixels in an image, such as image segmentation, overall image classification, and object detection. Specifically, a computer inputs a target image into the trained image processing model. The model predicts the probability distribution of the category to which each pixel in the target image belongs. Based on this probability distribution, the computer outputs a prediction result for the target image, such as image segmentation, image classification, or object detection.

[0129] Based on the above Figure 2 Please refer to the embodiments shown. Figure 3 This illustration shows a flowchart of an image classification processing method provided in an exemplary embodiment of this application. Figure 3 As shown, Figure 2 Step 220 can be implemented as steps 220a to 220e;

[0130] Step 220a: Based on the first prediction result and the second prediction result, obtain the classification label of each pixel in the first image sample; the classification label is used to indicate the classification of the corresponding pixel.

[0131] In some embodiments, since the first and second prediction results typically differ, the computer device can first combine the prediction results from both processing branches to determine a single predicted category label for each pixel in the first image sample. Because this category label is predicted by the model and not manually labeled, and because the label may not be accurate, it can be called a pseudo-label.

[0132] In some embodiments, the computer device may average the first prediction result and the second prediction result to obtain the classification label of each pixel in the first image sample.

[0133] For example, taking the probability distributions of each pixel in the first image sample as included in the first prediction result and the second prediction result respectively, the computer device takes the average of the probability distributions of the same pixel in the first prediction result and the second prediction result to obtain an average probability distribution, and then obtains the classification label of the pixel through the average probability distribution.

[0134] For example, suppose a pixel has a probability distribution of {0.8, 0.2, 0} in the first prediction result and a probability distribution of {0.6, 0.2, 0.2} in the second prediction result. The average probability distribution is obtained by averaging the two probability distributions as {0.7, 0.2, 0.1}. Then, based on the average probability distribution, the category (e.g., category 1) corresponding to the largest probability value (0.7) is determined as the category label of the pixel.

[0135] In one possible implementation, based on the first prediction result and the second prediction result, the classification label of each pixel in the first image sample is obtained, including:

[0136] Based on the first prediction result, obtain the first entropy value of the prediction result of the first processing branch for each pixel in the first image sample;

[0137] Based on the second prediction result, obtain the second entropy value of the prediction result of the second processing branch for each pixel in the first image sample;

[0138] The first entropy value is used as the confidence level of the prediction result for each pixel in the first prediction result, and the second entropy value is used as the confidence level of the prediction result for each pixel in the second prediction result. The classification label of each pixel in the first image sample is extracted from the first prediction result and the second prediction result.

[0139] During model training, the accuracy of the predictions made by the first and second processing branches of the image processing model for the same pixel location in the same image may be uncertain. For example, in a certain training round, the accuracy of the prediction result output by the first processing branch for pixel 1 in the first image sample may be higher than that output by the second processing branch; while for another pixel 2 in the same first image sample, the accuracy of the prediction result output by the first processing branch may be lower than that output by the second processing branch. To address this, the scheme shown in the embodiments of this application uses the entropy value of the prediction results output by the two processing branches to measure the accuracy (i.e., confidence level). By combining the confidence levels of the prediction results from the two processing branches, the classification label of each pixel in the first image sample is determined, improving the accuracy of the classification label and thus improving the accuracy of subsequent parameter updates for the image processing model.

[0140] The entropy value mentioned above can be determined by the processing results of the processing branch on the pixel and the number of classifications.

[0141] For example, the formula for calculating the entropy value mentioned above can be as follows:

[0142]

[0143] in, It is the entropy value at position (h,w) in the first sample image, and the above P h,w It is the prediction result of the first or second processing branch at position (h,w), and C is the number of categories.

[0144] The lower the entropy value, the lower the uncertainty of the prediction, and generally the higher the reliability.

[0145] In some embodiments, during the process of extracting the classification label of each pixel in the first image sample from the first prediction result and the second prediction result, using the first entropy value as the confidence level of the prediction result corresponding to each pixel in the first prediction result and the second entropy value as the confidence level of the prediction result corresponding to each pixel in the second prediction result, the computer device can determine the classification label based on the prediction result with high confidence (low entropy value) among the prediction results output by the two processing branches.

[0146] For example, for unlabeled sample image I u The process of determining category labels (pseudo-labels) can be represented as follows:

[0147]

[0148]

[0149] in, Indicate I u Category labels at positions (h,w) isI u The (h,w) position corresponds to the classification label indicated by the prediction result output by the first processing branch; The (h,w) position corresponds to the classification label indicated by the prediction result output from the second processing branch; isI u The position (h,w) corresponds to the entropy value of the prediction result output by the first processing branch. isI u The position (h, w) corresponds to the entropy value of the prediction result output by the second processing branch. In the above formula, S h,w It can record positions with lower uncertainty in the first processing branch, while (1-S h,w Then, positions with lower uncertainty in the second processing branch can be recorded.

[0150] In other embodiments, during the process of extracting classification labels for each pixel in the first image sample from the first and second prediction results, using a first entropy value as the confidence level of the prediction result for each pixel in the first prediction result and a second entropy value as the confidence level of the prediction result for each pixel in the second prediction result, the computer device can determine the weights of the first and second prediction results based on the confidence levels corresponding to the first and second prediction results. Then, based on the weights of the first and second prediction results, a weighted sum is performed to obtain a weighted prediction result. Based on the weighted prediction result, the classification label for each pixel in the first image sample is determined. The weights of the first and second prediction results are inversely correlated with their corresponding entropy values.

[0151] For example, still using I u Taking the pixel at position (h,w) as an example, the computer device... and Determine I u The position (h,w) corresponds to the weight 1 in the prediction result output by the first processing branch, and I. u The position (h,w) corresponds to weight 2 of the prediction result output by the first processing branch. Weights 1 and 2 are inversely correlated with their corresponding entropy values. In other words, the lower the corresponding entropy value, the higher the weight.

[0152] For example, computer devices can and Normalization is performed to obtain normalized value 1 and normalized value 2. Then, normalized value 1 is assigned as weight 2, and normalized value 2 is assigned as weight 1. For example, computer equipment can... and Input a weight calculation function to obtain the weight 1 and weight 2 mentioned above.

[0153] After determining the weights 1 and 2, the computer device can then use weights 1 and 2 to analyze I. u The position (h,w) corresponds to the prediction result output by the first processing branch, and I u The (h,w) position corresponds to the weighted average of the prediction results output from the second processing branch (e.g., for I). u The position (h,w) corresponds to a weighted average of the probability distributions in the first and second prediction results, which yields the I. u The weighted prediction results (probability distribution after weighted average) corresponding to the position (h,w) are used, and the classification corresponding to the largest probability value in the weighted prediction results is determined as I. u The classification label of the pixel at position (h,w).

[0154] Step 220b: Transform the first feature map into a first manifold feature, the first manifold feature including a feature corresponding to each pixel in the first image sample.

[0155] A manifold space is a characteristic space that locally exhibits properties of Euclidean space.

[0156] From a manifold perspective, the data we observe is actually mapped from a low-dimensional manifold to a high-dimensional space. Due to limitations imposed by the internal characteristics of the data, some high-dimensional data exhibits dimensionality redundancy, and in reality, only lower dimensions are needed for unique representation. Therefore, manifold spaces can serve as a means of data dimensionality reduction.

[0157] The scheme shown in this application reduces the dimensionality of the feature map to the manifold space, thereby enabling the constraint of the model's parameter updates in the manifold space.

[0158] In some embodiments, the process of converting the first feature map into a first manifold feature is as follows:

[0159] The computer device transforms the first feature map into a first image feature, which has the same resolution as the first sample image;

[0160] The first image features are rearranged into an image feature sequence, which contains D second image features, each of which has a size of H×W; where H, W, and D are the height, width, and number of channels of the first image feature, respectively.

[0161] The image feature sequence is processed by a fully connected layer to obtain the first manifold feature; the first manifold feature contains H×W third image features, each third image feature has a dimension of d, where d is the number of channels after the transformation by the fully connected layer.

[0162] For example, a computer device can perform an upsampling operation on the first feature map to obtain the first image feature mentioned above; the first image feature mentioned above can be a three-dimensional feature map of H×W×D; the computer device performs a reshape operation on the three-dimensional feature map to obtain an image feature sequence, and then inputs the image feature sequence into a fully connected layer to obtain the first manifold feature.

[0163] For example, based on the first feature Figure X For example, the computer device first performs an upsampling operation on X, obtaining the upsampled value. (This indicates that X is a feature map of size H×W×D). Then, X is transformed into a sequence of size HW×D through a reshape operation. A fully connected layer is then applied to the rearranged sequence to map it to... That is, the aforementioned first manifold feature, from which the first processing branch can be obtained in, This is a characteristic of the first manifold.

[0164] Step 220c: Based on the first manifold features and the classification labels of each pixel, obtain the first classification information.

[0165] In some embodiments, the first classification information is used to indicate the correspondence between the features extracted by the first processing branch from the first sample image and each category. Alternatively, the first classification information is used to characterize the distribution of features belonging to each category in the feature space corresponding to the first processing branch; more intuitively, the first classification information can be used to characterize which features extracted by the first processing branch from the image will be mapped to which categories.

[0166] In one possible implementation, first classification information is obtained based on the first manifold features and the classification labels of each pixel, including:

[0167] Based on the first manifold features and the classification labels of each pixel, obtain the prototypes of each category in the first manifold features; the prototypes are the average results of the features belonging to the corresponding category.

[0168] Based on the prototypes of each category in the first manifold features, the first classification information is obtained.

[0169] In the above embodiments, the first manifold feature contains H×W d-dimensional features, which correspond to H×W pixels in the first sample image; that is, each pixel in the first sample image corresponds to one d-dimensional feature (also called a pixel feature) in the first manifold feature.

[0170] When acquiring the first classification information, the computer device can add the pixel features corresponding to each pixel in the first manifold features to the feature set of the corresponding classification label according to the classification label (i.e. pseudo label) of each pixel in the first sample image; that is, set up a feature set with the same number of categories, and each feature set contains the pixel features of the pixel corresponding to the same classification label; for each feature set, take the average value of the pixel features in the feature set, and the prototype of the classification corresponding to the feature set can be obtained.

[0171] In other words, in this embodiment of the application, a prototype-based strategy is adopted, in which features belonging to the same category are averaged to represent the corresponding category. This enables the representation of the central features of each category, thereby accurately reflecting the correspondence between the features extracted from the first sample image by the processing branch and each category.

[0172] For example, computer devices can calculate the prototype of a category by averaging the manifold features belonging to that category using the following formula:

[0173]

[0174] in, Representing a manifold The i-th feature, S c Based on tags In the judgment, the feature set with the same category c, |S c | represents the size of the set (that is, the number of pixel features contained in the set).

[0175] In one possible implementation, corresponding to the prototypes of each category, the first category information is obtained, including:

[0176] Based on the features of the first manifold, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the first classification information.

[0177] Covariance matrices are ubiquitous in any statistical domain, reflecting data structure properties. In computer vision and machine learning tasks, covariance matrices support conditional independence and model image texture, particularly in transformer-based deep networks. Introducing covariance matrices into multi-head attention processing demonstrates superior performance, enhancing the discriminative power of learned feature representations. The scheme illustrated in this application utilizes the prototypes corresponding to each category within the first manifold features to construct the covariance matrix. This better reflects the correspondence between features extracted from the first sample image by the processing branch and each category, ensuring the effectiveness of subsequent model training and ultimately improving the accuracy of the image processing model.

[0178] Step 220d: Transform the second feature map into a second manifold feature, which includes a feature corresponding to each pixel in the first image sample.

[0179] The process of transforming the second feature map into the second manifold feature is similar to the process of transforming the first feature map into the first manifold feature in step 220b above, and will not be described again here.

[0180] Step 220e: Obtain second classification information based on the second manifold features and the classification labels of each pixel.

[0181] In one possible implementation, second classification information is obtained based on the second manifold features and the classification labels of each pixel, including:

[0182] Based on the second manifold features and the classification labels of each pixel, obtain the prototypes of each category in the second manifold features;

[0183] Based on the features of the second manifold, the prototypes corresponding to each category are used to obtain the second classification information.

[0184] In one possible implementation, second classification information is obtained based on the prototypes corresponding to each classification in the second manifold features, including:

[0185] Based on the features of the second manifold, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the second classification information.

[0186] The process of obtaining the second category information described above is similar to the process of obtaining the first category information in step 220c, and will not be repeated here.

[0187] In one possible implementation, obtaining the loss function value includes:

[0188] Obtain the F-norm between the first classification information and the second classification information, and use it as a sub-value of the first loss function.

[0189] Considering that the first processing branch and the second processing branch are heterogeneous networks, there are different class distributions in the two processing branches. This application proposes a knowledge transfer loss function between processing branches (that is, the first loss function sub-value mentioned above) to learn complementary class statistics in the manifold space.

[0190] This application embodiment can utilize a symmetric positive definite (SPD) matrix metric to measure class-level differences between two processing branches.

[0191] SPD can be composed of a square matrix M of size d×d, and it is represented as:

[0192]

[0193] in, It is the set of all d-dimensional vectors, 0 d It is a d-dimensional vector of all zeros. This indicates that u is a vector other than a d-dimensional zero vector. T It is the transpose of u. It is a matrix of dimension d×d; the above formula means that if the matrix is ​​regarded as a point in the SPD, then the matrix must be symmetric and have positive eigenvalues, which is a necessary and sufficient condition for the SPD matrix to form a manifold.

[0194] In order to measure the distance between points in the SPD manifold, and considering that the Frobenius norm (F norm) is not limited by the element values ​​in the covariance matrix, it can improve the accuracy and effectiveness of the loss function, thereby ensuring the subsequent model training effect and improving the accuracy of the image processing model. In this embodiment, the Frobenius norm can be used to calculate the above-mentioned first loss function sub-value.

[0195] Specifically, given the manifold in the first processing branch And, the manifold in the second processing branch The covariance matrix can be defined as:

[0196]

[0197] Therefore, the covariance matrix corresponding to the first processing branch and the covariance matrix corresponding to the first processing branch These can be viewed as points in the SPD manifold. Then, using the Frobenius norm between these two matrices, the knowledge transfer loss function between the two processing branches can be calculated.

[0198]

[0199] In one possible implementation, obtaining the loss function value also includes:

[0200] Obtain the second sub-value of the loss function from the loss function values. The second sub-value of the loss function is used to maximize the difference between the prototypes of each category in the first manifold features.

[0201] Obtain the third loss function sub-value from the loss function values. The third loss function sub-value is used to maximize the difference between the prototypes of each category in the second manifold features.

[0202] In this embodiment, in addition to enabling the two processing branches to learn each other's knowledge through the knowledge transfer loss function between the two processing branches, the computer device can also use a second loss function sub-value to make the distance between features belonging to different categories in the features extracted by the first processing branch sufficiently large, and use a third loss function sub-value to make the distance between features belonging to different categories in the features extracted by the second processing branch sufficiently large, thereby ensuring the accuracy of subsequent classification using features extracted by the first and second processing branches, and thus improving the accuracy of the image processing model.

[0203] The solution provided in this application's embodiments can maximize the distance between any two different categories in the latent space. For class-specific prototypes... Reducing inter-class dependencies by maximizing the similarity of differences between prototypes can be expressed as:

[0204]

[0205] Among them, the above and Let C be the c-th and j-th prototypes among the prototypes corresponding to the C categories, where C is the number of categories.

[0206] In one possible implementation, obtaining the loss function value also includes:

[0207] Select the K features with the highest attention scores from the candidate features corresponding to each category and add them to the storage space of the corresponding category; K is an integer greater than or equal to 1; the candidate features are the features extracted by the image processing model from the second image sample and transformed into a manifold; the second image sample is a labeled image sample;

[0208] Obtain the fourth loss function sub-value from the loss function values. The fourth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the first manifold features and the features in the storage space of each category.

[0209] Obtain the fifth loss function sub-value from the loss function values. The fifth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the second manifold features and the features in the storage space of each category.

[0210] In other words, this application employs a prototype-based strategy, which connects unlabeled data to a prototype (i.e., class center) specific to a certain category, with high-quality features extracted from labeled data of the same category. In this way, indistinguishable unlabeled prototypes can be explicitly learned from representative features of labeled data in a class-aware manner, thereby effectively utilizing unlabeled training data, improving model training performance, and ultimately enhancing the accuracy of image processing.

[0211] In some embodiments, a computer device may use a memory bank. This is used to store high-quality features collected from the labeled data. For each class, an attention module is used to obtain a ranking score for candidate features (i.e., the correctly predicted features that are consistent with the ground truth). The attention module consists of two consecutive linear functions and a sigmoid function, ultimately producing a score in the range [0, 1] for each feature. The K features with the highest scores are then selected and added to the memory bank, thus the memory bank has a size of C × K × d.

[0212] Subsequently, the computer equipment allowed the prototype to... Get close to the memory bank Medium-to-high-quality categorical features are used to narrow the intra-class distribution. Specifically, cosine similarity can be used to calculate... and Distance between, intra-class loss It can be defined as:

[0213]

[0214] Where C represents the number of categories, and K represents the number of features in the memory bank. For the prototype of the c-th category, It is the k-th feature in the memory bank of the c-th category.

[0215] In some embodiments, the above and They can be used in combination. In this case, by combining intra-class and inter-class losses, the intra-branch class consistency loss function involved in this application embodiment... This allows the network to separate features belonging to different classes. Among them, It can be applied to both the first and second processing branches simultaneously to improve the performance of each processing branch, which features compact intra-class processing and discrete inter-class processing. It can be represented as:

[0216]

[0217] Where i represents the i-th processing branch in the first processing branch and the second processing branch.

[0218] In one possible implementation, obtaining the loss function value also includes:

[0219] Obtain the seventh sub-value of the loss function; the seventh sub-value of the loss function is used to indicate the difference between the first prediction result and the classification label of each pixel;

[0220] Obtain the eighth loss function subvalue from the loss function values; the eighth loss function subvalue is used to indicate the difference between the second prediction result and the classification label of each pixel.

[0221] In this embodiment of the application, the computer device can also utilize reliable fake tags. And utilize the loss function To supervise the output of the two processing branches, since the pseudo-labels (i.e., the classification labels) are determined by combining the outputs of the two processing branches, the seventh and eighth loss function sub-values ​​can make the prediction results of the two processing branches tend to be consistent at the pixel level (the aforementioned loss functions). This can be referred to as pixel-level consistency loss, thereby further improving the model training effect and ensuring the accuracy of the image processing model. The seventh and eighth loss function sub-values ​​mentioned above can be calculated using at least one of the following loss functions: cross-entropy loss, Dice loss, Focal loss, and Tversky loss. Taking the Dice loss function as an example... It can be represented as:

[0222]

[0223] Where P1 is the first prediction result, and P2 is the second prediction result. For each pixel, a classification label is provided. Corresponding to the seventh loss function sub-value mentioned above, This corresponds to the eighth loss function subvalue mentioned above.

[0224] Taking a CNN and a Transformer network as examples, where the first processing branch and the second processing branch are respectively, the above-mentioned application... Figure 2 or Figure 3The proposed scheme enables a semi-supervised image segmentation technique that combines CNN and Transformer networks, such as... Figure 4 As shown, it illustrates the image segmentation results provided in this application. This scheme can be widely applied to the segmentation of different images (such as multi-category medical images), and its results have been proven applicable to the segmentation of medical images acquired from different datasets and devices.

[0225] Combination Figure 2 and Figure 3 Please refer to the embodiments shown. Figure 5 This illustrates a schematic diagram of a model framework related to this application. Figure 5 The model framework shown can be called the M-CnT framework, which simultaneously takes labeled data I as input. l and unlabeled data I u After feature mining using a parallel CNN-Transformer network, two outputs P can be obtained. CNN and P Trans When inputting labeled data I l When using fully supervised loss Apply constraints. When the input is unlabeled data I u At that time, by utilizing a pixel-level consistency loss function and two manifold constraints and To optimize model parameters.

[0226] Taking the segmentation of medical images as an example, the above Figure 5 The framework shown may include the following steps:

[0227] I. Establishing a training dataset

[0228] This application's embodiments verify the above on two publicly available multi-class datasets. Figure 2 or Figure 3 The methods provided in the illustrated embodiments are Pattern Analysis, Static Modeling and Computational Learning Visual Object Classes (PASCAL VOC) and the Cityscapes dataset, which contain natural images with varying numbers of classes.

[0229] in:

[0230] The Synapse dataset contains 9 categories (8 abdominal organs and 1 background category) and includes 30 CT scan cases, of which 18 cases were used for training and 12 cases were used for testing. In a semi-supervised setting, 15% (3 cases), 10% (6 cases), and 15% (9 cases) of the training data were labeled, while the remaining training cases were unlabeled.

[0231] The ACDC dataset contains four categories (left ventricle, right ventricle, myocardium, and one background category), involving a total of 100 MRI cases. Of these, 70 were used for training, 10 for validation, and 20 for testing. In a semi-supervised setting, 3% (3 cases), 10% (7 cases), and 15% (11 cases) of the training data were labeled, while the remaining training cases were unlabeled.

[0232] The ISIC dataset contains two categories (foreground and background for skin tumors), comprising a total of 2594 pathological images, of which 1838 were used for training and 756 for testing. In a semi-supervised setting, 3% (55 images), 10% (181 images), and 15% (275 images) of the training data were labeled, while the remaining training images were unlabeled.

[0233] II. Experimental Details of this Scheme

[0234] All models were trained using the Stochastic Gradient Descent (SGD) optimizer with an initial learning rate of 0.01, momentum of 0.9, and weight decay of 10⁻⁴. The network converged after 30,000 training iterations. The batch size was set to 16, consisting of 8 labeled images and 8 unlabeled images. In this approach, a 224×224 patch was randomly cropped as input. To avoid overfitting, random rotation and random flipping were used as data augmentation methods. All datasets were evaluated using the Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD) metrics.

[0235] III. Algorithm Design and Steps

[0236] (1) CNN-Transformer network architecture

[0237] The framework proposed in this application utilizes the complementary characteristics of CNN and Transformer. The various components of the framework will be described in detail below.

[0238] The M-CnT proposed in this application mines complementary features of two student networks, including the CNN student network f(I; θ). C ) and Transformer student network f(I; θ T The following section will detail the various components of the CNN-Transformer network architecture.

[0239] CNN Student Network: For the CNN student network, this application chooses the traditional UNet with a feature pyramid structure to extract local hierarchical contextual information. UNet consists of an encoder and a decoder with skip connections. The encoder is divided into five stages. As the encoder progresses, the feature map sizes of the five stages gradually decrease to {1 / 1, 1 / 2, 1 / 4, 1 / 8, 1 / 16}; while the channel dimensions correspondingly increase to {16, 32, 64, 128, 256}. This allows for the extraction of features at different granular levels in stages. The decoder gradually restores the size of the feature maps and ultimately produces a segmentation result with the same resolution as the input image.

[0240] Transformer Student Network: Unlike CNN student networks that learn local cues, Transformers aim to extract global feature information. Specifically, SwinUNet can be used as the backbone network of the Transformer, featuring spatial reduction and window-based attention to reduce computational complexity. The SwinUNet encoder is also divided into five stages. The resolution of the feature maps in the five stages gradually decreases to {1 / 4, 1 / 8, 1 / 16, 1 / 32, 1 / 32}, while the number of channels increases accordingly to {96, 192, 384, 768, 768}. The decoder gradually increases the resolution of the feature maps, generating the segmentation result.

[0241] (2) Loss Function

[0242] 1) Pixel-level consistency loss

[0243] Based on the Dual-student structure, input an unlabeled dataset I. u Pixel-level segmentation results can be obtained from CNN and Transformer networks respectively: P CNN and P Trans Furthermore, their respective pseudo-labels can be obtained:

[0244]

[0245] As mentioned earlier, CNNs with local convolution operations and Transformers with long-range self-attention have different learning abilities, leading to different prediction results. This scheme further considers the uncertainties in CNNs and Transformers, rather than directly averaging the predictions of two students to obtain pseudo-labels. This effectively eliminates unreliable and noisy pseudo-labels. Specifically, the scheme shown in this application can calculate P... CNN and P Trans The entropy of the probability distribution at the mid-pixel level is used to obtain higher quality pseudo-labels.

[0246] Next, using reliable pseudo-labels And using the Dice loss function To supervise the output of two students, this process can be represented as:

[0247]

[0248] The goal of this approach is to leverage manifold constraints to fully utilize the key category statistics embedded in unlabeled data. Therefore, this approach incorporates features prior to the final segmentation layer. Figure X This is mapped to a feature representation in the manifold space. Specifically, the features are first upsampled... Figure X Transformed to the same resolution as the original input image, i.e. Then, it is rearranged into a sequence of size HW×D, that is, the 3D feature map (H×W×D) is transformed into a 2D sequence (HW×D) through a reshape operation. A fully connected layer is then applied to the rearranged sequence (that is, a fully connected layer processes the previously rearranged sequence), mapping it to... Therefore, the same size can be obtained from both the CNN and Transformer student networks. In this context, H represents the length, W represents the width, D represents the number of channels before transformation, and d represents the number of channels after transformation through the fully interconnected layer.

[0249] 2) Intra-class consistency loss function for students

[0250] This section primarily addresses methods for generating separable features for each student branch in the manifold space. Specifically, two loss functions are defined: intra-class clustering loss to achieve more compact features, and inter-class discretization loss to further generate relatively large inter-class differences.

[0251] Intraclass aggregation loss In the method of this application, the prototype of a class is first calculated by averaging the manifold characteristics belonging to that class. Then, a memory bank is used... This is used to store high-quality features collected from the labeled data. For each class, an attention module is used to obtain a ranking score for the candidate features. Then, the K features with the highest scores are selected and added to the memory bank.

[0252] Then let the prototype Get close to the memory bank Medium to high-quality categorical features for calculating intra-class loss

[0253] Inter-class discrete loss Another efficient way to achieve separable boundaries between classes is to maximize the distance between any two distinct classes in the latent space. Therefore, for class-specific prototypes... Reduce inter-class dependencies by maximizing the similarity of differences between prototypes.

[0254] The intra-class consistency loss function for students is obtained by combining intra-class and inter-class losses. This allows the network to separate features belonging to different classes. Simultaneously applied to both CNN and Transformer student networks to improve the performance of each student's intra-class compact and inter-class discrete networks, it can be expressed as:

[0255]

[0256] 3) Knowledge transfer loss function among students

[0257] In this section, considering the different class distributions in the CNN and Transformer student networks, a knowledge transfer loss function between students is proposed to learn complementary class statistics in the manifold space. Given the manifolds in the two student networks... as well as covariance matrix These can be viewed as points in the SPD manifold. Then, the Frobenius norm between these two matrices can be used to calculate the knowledge transfer loss function between students.

[0258]

[0259] 4) Overall SemiCVT optimization function:

[0260] like Figure 5 As shown, the overall optimization function of SemiCVT can be:

[0261]

[0262] in It applies to labeled data I l The fully supervised loss function can be, specifically, the commonly used cross-entropy loss function and the Dice loss function. and Applying to unlabeled data I u These are the pixel-level unsupervised loss, the intra-student class consistency loss function, and the inter-student knowledge transfer loss function, respectively, thus fully learning the information from a large amount of unlabeled data. λ p and λ m This is used to balance the various loss functions. For example, λ can be chosen. p =1 and Where t is the current training phase, t max This is the largest training phase.

[0263] The solution shown in this application embodiment can be applied to cloud scenarios, that is, to provide users with services such as image segmentation through the cloud.

[0264] For example, please refer to Figure 6 This diagram illustrates the system framework of the image segmentation service involved in an embodiment of this application. Figure 6 As shown, front-end A receives the data (input image) that needs to be segmented, uploads it to the back-end, and then the back-end uses this technical solution to segment it and outputs the segmented image to front-end B.

[0265] The proposed method evaluates its segmentation performance on three publicly available medical datasets, including quantitative results comparing it with other state-of-the-art methods, as well as visualizations of the segmentation results. For details, please refer to [link to relevant documentation]. Figures 7 to 11 The diagram illustrates the experimental structure involved in this application. Figures 7 to 11 As shown, first in Figure 7 and Figure 8 This paper demonstrates the quantitative performance of M-CnT and other state-of-the-art methods on different datasets. DSC and HD metrics were used as model evaluation methods on all three datasets. Figure 7 and Figure 8 As can be seen, the M-CnT method provided in this application consistently outperforms other methods and significantly improves performance on all datasets. Notably, compared to CTCT, which uses the same CNN-Transformer architecture, the M-CnT method in this application significantly outperforms CTCT, especially in the challenging 3% labeled data setting of the ACDC dataset (DSC: +4.9%, HD: -9.0mm). Furthermore, in Figure 9 , Figure 10 and Figure 11The visualization results are presented, showcasing the top 5 segmentation results on their respective datasets. Observation reveals that the method provided in this application can accurately segment target objects of different sizes, shapes, and locations. Furthermore, the segmentation results of the method provided in this application have clearer boundaries than other results, reducing the probability of missegmentation.

[0266] In the embodiments described above, a novel semi-supervised segmentation technique (M-CnT) combining CNN-Transformer under manifold constraints is proposed. This technique addresses the intra-student and inter-student problems faced by CNN-based semi-supervised methods. Specifically, M-CnT follows a dual-student architecture, employing both CNN and Transformer heterogeneous network structures, and mines complementary class features under manifold constraints: intra-student class consistency loss and inter-student knowledge transfer loss. The former helps generate compact and easily distinguishable features in each student branch, while the latter transfers complementary class knowledge between students. Our method has been extensively evaluated on three public datasets and consistently outperforms other SSL methods. This method is based on medical image segmentation techniques, but in practice, it can be applied to segmentation tasks of other images, such as natural images.

[0267] Figure 12 This is a structural block diagram illustrating an image classification processing apparatus according to an exemplary embodiment. The apparatus can implement... Figure 2 or Figure 3 The illustrated embodiment provides all or part of the steps in the method for image classification processing, which includes:

[0268] Image input module 1201 is used to input a first image sample into an image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result; the first feature map is a feature map obtained by a first processing branch in the image processing model performing feature extraction processing on the first image sample, and the second feature map is a feature map obtained by a second processing branch in the image processing model performing feature extraction processing on the first image sample; the first prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the first processing branch; the second prediction result is used to indicate the classification to which each pixel in the first sample image belongs, as predicted by the second processing branch; the first image sample is an unlabeled image sample.

[0269] The classification information acquisition module 1202 is used to acquire first classification information and second classification information based on the first feature map, the second feature map, the first prediction result, and the second prediction result; the first classification information is used to indicate the classification of the first image sample by the first processing branch; the second classification information is used to indicate the classification of the first image sample by the second processing branch.

[0270] The loss function value acquisition module 1203 is used to acquire loss function values; the loss function values ​​include a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information.

[0271] The parameter update module 1204 is used to update the parameters of the image processing model based on the loss function value.

[0272] The image processing model, trained to convergence, is used to process the input target image and predict the classification of each pixel in the target image.

[0273] In one possible implementation, the classification information acquisition module 1202 is used to,

[0274] Based on the first prediction result and the second prediction result, the classification label of each pixel in the first image sample is obtained; the classification label is used to indicate the classification of the corresponding pixel;

[0275] The first feature map is transformed into a first manifold feature, wherein the first manifold feature includes a feature corresponding to each pixel in the first image sample;

[0276] Based on the first manifold feature and the classification label of each pixel, the first classification information is obtained;

[0277] The second feature map is transformed into a second manifold feature, the second manifold feature including a feature corresponding to each pixel in the first image sample;

[0278] Based on the second manifold feature and the classification label of each pixel, the second classification information is obtained.

[0279] In one possible implementation, the classification information acquisition module 1202 is used to,

[0280] Based on the first manifold feature and the classification label of each pixel, the prototype corresponding to each category in the first manifold feature is obtained; the prototype is the average result of the feature belonging to the corresponding category.

[0281] Based on the prototypes of each category in the first manifold features, obtain the first category information;

[0282] The step of obtaining the second classification information based on the second manifold feature and the classification label of each pixel includes:

[0283] Based on the second manifold feature and the classification label of each pixel, obtain the prototype of each category in the second manifold feature;

[0284] Based on the prototypes of each category in the second manifold features, the second classification information is obtained.

[0285] In one possible implementation, the classification information acquisition module 1202 is used to,

[0286] Based on the features of the first manifold, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the first classification information;

[0287] The step of obtaining the second classification information based on the prototypes corresponding to each classification in the second manifold features includes:

[0288] Based on the second manifold features, a covariance matrix is ​​constructed corresponding to the prototypes of each category to obtain the second classification information.

[0289] In one possible implementation, the loss function value acquisition module 1203 is used to acquire the F-norm between the first classification information and the second classification information, as a sub-value of the first loss function.

[0290] In one possible implementation, the loss function value acquisition module 1203 is further configured to:

[0291] Obtain the second loss function sub-value from the loss function values. The second loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the first manifold features.

[0292] Obtain the third loss function sub-value from the loss function values. The third loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the second manifold features.

[0293] In one possible implementation, the loss function value acquisition module 1203 is further configured to:

[0294] From each of the categories, select the K features with the highest attention scores from the candidate features corresponding to each category and add them to the storage space of the corresponding category; K is an integer greater than or equal to 1; the candidate features are the features extracted by the image processing model from the second image sample and transformed into a manifold; the second image sample is a labeled image sample;

[0295] Obtain the fourth loss function sub-value from the loss function values. The fourth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the first manifold features and the features in the storage space of each category.

[0296] Obtain the fifth loss function sub-value from the loss function values. The fifth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the second manifold features and the features in the storage space of each category.

[0297] In one possible implementation, the device further includes:

[0298] The result acquisition module is used to acquire the third prediction result and the fourth prediction result output by the image processing model after processing the second image sample; the third prediction result is used to indicate the category to which each pixel in the second sample image belongs, as predicted by the first processing branch; the fourth prediction result is used to indicate the category to which each pixel in the second sample image belongs, as predicted by the second processing branch.

[0299] The loss function value acquisition module 1203 is further configured to acquire a sixth loss function sub-value in the loss function value based on the third prediction result, the fourth prediction result, and the annotation information of the second image sample. The sixth loss function sub-value is used to indicate the difference between the third prediction result, the fourth prediction result, and the annotation information. The annotation information is used to indicate the classification of each pixel in the second image sample.

[0300] In one possible implementation, the loss function value acquisition module 1203 is further configured to:

[0301] Obtain the seventh loss function sub-value from the loss function values; the seventh loss function sub-value is used to indicate the difference between the first prediction result and the classification label of each pixel;

[0302] Obtain the eighth loss function sub-value from the loss function values; the eighth loss function sub-value is used to indicate the difference between the second prediction result and the classification label of each pixel.

[0303] In one possible implementation, the classification information acquisition module 1202 is used to,

[0304] Based on the first prediction result, obtain the first entropy value of the prediction result of the first processing branch for each pixel in the first image sample;

[0305] Based on the second prediction result, obtain the second entropy value of the prediction result of the second processing branch for each pixel in the first image sample;

[0306] The first entropy value is used as the confidence level of the prediction result corresponding to each pixel in the first prediction result, and the second entropy value is used as the confidence level of the prediction result corresponding to each pixel in the second prediction result. The classification label of each pixel in the first image sample is extracted from the first prediction result and the second prediction result.

[0307] In one possible implementation, the first processing branch and the second processing branch are heterogeneous networks.

[0308] In summary, the scheme shown in this application embodiment, during the training process, extracts features and predicts the first image sample through two processing branches in the image processing model, respectively. Based on the feature maps and prediction results extracted by the two processing branches for the first image sample, the classification of the first image sample by the two processing branches is obtained. The difference between the classifications of the first image sample by the two processing branches is used as part of the loss function to update the parameters of the image processing model. Since the loss calculated by the classifications of the first image sample by the two processing branches is introduced, the optimization process of the image processing model fully considers the rich category distribution, thereby enabling the trained image processing model to extract sufficiently rich features, thus improving the accuracy of the image processing model.

[0309] Figure 13This is a schematic diagram of a computer device according to an exemplary embodiment. The computer device can be implemented as a computer device for training image processing models in the various method embodiments described above, or it can be implemented as a computer device for performing image classification processing using image processing models in the various method embodiments described above. The computer device 1300 includes a central processing unit (CPU) 1301, a system memory 1304 including random access memory (RAM) 1302 and read-only memory (ROM) 1303, and a system bus 1305 connecting the system memory 1304 and the central processing unit 1301. The computer device 1300 also includes a basic input / output system 1306 to facilitate information transfer between various devices within the computer, and a mass storage device 1307 for storing the operating system 1313, application programs 1314, and other program modules 1315.

[0310] The mass storage device 1307 is connected to the central processing unit 1301 via a mass storage controller (not shown) connected to the system bus 1305. The mass storage device 1307 and its associated computer-readable media provide non-volatile storage for the computer device 1300. That is, the mass storage device 1307 may include computer-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0311] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1304 and mass storage device 1307 described above can be collectively referred to as memory.

[0312] Computer device 1300 can be connected to the Internet or other network devices via network interface unit 1311 connected to the system bus 1305.

[0313] The memory also includes one or more programs, which are stored in the memory, and the central processing unit 1301 implements the functions by executing the one or more programs. Figure 2 or Figure 3 All or part of the steps of any of the methods shown.

[0314] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including a computer program (instructions) that can be executed by a processor of a computer device to perform the methods shown in the various embodiments of this application. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0315] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods shown in the various embodiments described above.

[0316] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the solutions disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0317] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for image classification processing, characterized in that, The method includes: A first image sample is input into an image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result. The first feature map is obtained by a first processing branch in the image processing model performing feature extraction on the first image sample. The second feature map is obtained by a second processing branch in the image processing model performing feature extraction on the first image sample. The first prediction result indicates the classification to which each pixel in the first image sample belongs, as predicted by the first processing branch. The second prediction result indicates the classification to which each pixel in the first image sample belongs, as predicted by the second processing branch. The first image sample is an unlabeled image sample. Based on the first prediction result and the second prediction result, the classification label of each pixel in the first image sample is obtained; the classification label is used to indicate the classification of the corresponding pixel; the first feature map is reduced to a manifold space to obtain a first manifold feature, the first manifold feature includes a feature corresponding to each pixel in the first image sample; based on the first manifold feature and the classification label of each pixel, the prototype corresponding to each classification in the first manifold feature is obtained; the prototype is the average result of the feature belonging to the corresponding classification; based on the prototype corresponding to each classification in the first manifold feature, a covariance matrix is ​​constructed to obtain first classification information; the second feature map is reduced to a manifold space to obtain a second manifold feature, the second manifold feature includes a feature corresponding to each pixel in the first image sample; based on the second manifold feature and the classification label of each pixel, the prototype corresponding to each classification in the second manifold feature is obtained; based on the prototype corresponding to each classification in the second manifold feature, a covariance matrix is ​​constructed to obtain second classification information; Obtain the loss function value; the loss function value includes a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information; The image processing model is updated based on the loss function value; The image processing model, trained to convergence, is used to process the input target image and predict the classification of each pixel in the target image.

2. The method according to claim 1, characterized in that, The process of obtaining the loss function value includes: Obtain the F-norm between the first classification information and the second classification information, and use it as a sub-value of the first loss function.

3. The method according to claim 2, characterized in that, The process of obtaining the loss function value also includes: Obtain the second loss function sub-value from the loss function values. The second loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the first manifold features. Obtain the third loss function sub-value from the loss function values. The third loss function sub-value is used to maximize the difference between the prototypes corresponding to each category in the second manifold features.

4. The method according to claim 1, characterized in that, The process of obtaining the loss function value also includes: From each of the categories, select the K features with the highest attention scores from the candidate features corresponding to each category and add them to the storage space of the corresponding category; K is an integer greater than or equal to 1; the candidate features are the features extracted by the image processing model from the second image sample and transformed into a manifold; the second image sample is a labeled image sample; Obtain the fourth loss function sub-value from the loss function values. The fourth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the first manifold features and the features in the storage space of each category. Obtain the fifth loss function sub-value from the loss function values. The fifth loss function sub-value is used to indicate the difference between the prototype corresponding to each category in the second manifold features and the features in the storage space of each category.

5. The method according to claim 4, characterized in that, The method further includes: The image processing model outputs a third prediction result and a fourth prediction result after processing the second image sample. The third prediction result indicates the classification to which each pixel in the second image sample belongs, as predicted by the first processing branch. The fourth prediction result indicates the classification to which each pixel in the second image sample belongs, as predicted by the second processing branch. The process of obtaining the loss function value also includes: Based on the third prediction result, the fourth prediction result, and the annotation information of the second image sample, a sixth loss function sub-value is obtained from the loss function values. The sixth loss function sub-value is used to indicate the differences between the third prediction result, the fourth prediction result, and the annotation information. The annotation information is used to indicate the classification of each pixel in the second image sample.

6. The method according to claim 1, characterized in that, The process of obtaining the loss function value also includes: Obtain the seventh loss function sub-value from the loss function values; the seventh loss function sub-value is used to indicate the difference between the first prediction result and the classification label of each pixel; Obtain the eighth loss function sub-value from the loss function values; the eighth loss function sub-value is used to indicate the difference between the second prediction result and the classification label of each pixel.

7. The method according to any one of claims 1 to 6, characterized in that, The step of obtaining the classification label of each pixel in the first image sample based on the first prediction result and the second prediction result includes: Based on the first prediction result, obtain the first entropy value of the prediction result of the first processing branch for each pixel in the first image sample; Based on the second prediction result, obtain the second entropy value of the prediction result of the second processing branch for each pixel in the first image sample; The first entropy value is used as the confidence level of the prediction result corresponding to each pixel in the first prediction result, and the second entropy value is used as the confidence level of the prediction result corresponding to each pixel in the second prediction result. The classification label of each pixel in the first image sample is extracted from the first prediction result and the second prediction result.

8. The method according to claim 1, characterized in that, The first processing branch and the second processing branch are heterogeneous networks.

9. An image classification processing apparatus, characterized in that, The device includes: An image input module is used to input a first image sample into an image processing model to obtain a first feature map, a second feature map, a first prediction result, and a second prediction result. The first feature map is a feature map obtained by a first processing branch in the image processing model performing feature extraction on the first image sample. The second feature map is a feature map obtained by a second processing branch in the image processing model performing feature extraction on the first image sample. The first prediction result indicates the classification to which each pixel in the first image sample belongs, as predicted by the first processing branch. The second prediction result indicates the classification to which each pixel in the first image sample belongs, as predicted by the second processing branch. The first image sample is an unlabeled image sample. A classification information acquisition module is used to: acquire classification labels for each pixel in the first image sample based on the first prediction result and the second prediction result; the classification labels indicate the classification of the corresponding pixel; reduce the dimensionality of the first feature map to a manifold space to obtain a first manifold feature, the first manifold feature including a feature corresponding to each pixel in the first image sample; based on the first manifold feature and the classification labels of each pixel, acquire prototypes corresponding to each classification in the first manifold feature; the prototypes are the average results of features belonging to the corresponding classification; construct a covariance matrix based on the prototypes corresponding to each classification in the first manifold feature to obtain first classification information; reduce the dimensionality of the second feature map to a manifold space to obtain a second manifold feature, the second manifold feature including a feature corresponding to each pixel in the first image sample; based on the second manifold feature and the classification labels of each pixel, acquire prototypes corresponding to each classification in the second manifold feature; construct a covariance matrix based on the prototypes corresponding to each classification in the second manifold feature to obtain second classification information; The loss function value acquisition module is used to acquire loss function values; the loss function value includes a first loss function sub-value, which is used to indicate the difference between the first classification information and the second classification information; The parameter update module is used to update the parameters of the image processing model based on the loss function value; The image processing model, trained to convergence, is used to process the input target image and predict the classification of each pixel in the target image.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer instruction, which is loaded and executed by the processor to implement the image classification processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer instruction, which is loaded and executed by a processor to implement the image classification processing method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the image classification processing method as described in any one of claims 1 to 8.

Images