Image classification method

Abstract

Embodiments of the present specification provide an image classification method, which comprises: acquiring an image to be classified, wherein the image to be classified comprises global image information of a target object; inputting the image to be classified into an image classification model to obtain type information of the target object, wherein the image classification model is obtained by training based on feature correlation information of a sample image, the sample image comprises global image information composed of multiple local features of a sample object, and the feature correlation information is obtained by performing correlation analysis on the multiple local features of the sample object. By training the image classification model based on the feature correlation information of the sample image, the image classification model is enabled to have the function of classifying images based on the feature correlation information between local features, thereby improving the accuracy of the type of the target object output by the image classification model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to an image classification method. BACKGROUND

[0002] With the continuous development of computer technology, users have more and more demands for image classification, and image classification is applied to more and more scenarios. At present, in order to improve the efficiency of image classification, the local features of an image can be extracted, and then the classification of the image is realized based on the local features.

[0003] However, in actual application scenarios, the method of classifying only considering local features can lead to inaccurate classification.

[0004] Therefore, it is necessary to provide a more reliable solution to solve the problem of classifying images. SUMMARY

[0005] Therefore, the embodiments of the present specification provide an image classification method. One or more embodiments of the present specification also relate to an image classification apparatus, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0006] According to a first aspect of the embodiments of the present specification, an image classification method is provided, comprising:

[0007] obtaining a to-be-classified image, wherein the to-be-classified image comprises global image information of a target object;

[0008] inputting the to-be-classified image into an image classification model to obtain type information of the target object, wherein the image classification model is obtained by training based on feature correlation information of a sample image, the sample image comprises global image information composed of multiple local features of a sample object, and the feature correlation information is obtained by correlation analysis between the multiple local features of the sample object.

[0009] According to a second aspect of the embodiments of the present specification, another image classification method is provided, applied to a cloud-side device, comprising:

[0010] obtaining a to-be-classified image sent by an edge-side device, wherein the to-be-classified image comprises global image information of a target object;

[0011] inputting the to-be-classified image into an image classification model to obtain type information of the target object, wherein the image classification model is obtained by training based on feature correlation information of a sample image, the sample image comprises global image information composed of multiple local features of a sample object, and the feature correlation information is obtained by correlation analysis between the multiple local features of the sample object.

[0012] returning the type information to the end-side device.

[0013] According to a third aspect of the embodiments of the present specification, a training method of an image classification model is provided, applied to a cloud-side device, comprising:

[0014] obtaining a sample image in a sample set, wherein the sample image contains global image information of a sample object and an object classification label;

[0015] inputting the sample image into an image classification model, analyzing the global image information, and obtaining a plurality of local features;

[0016] calculating feature correlation information between the plurality of local features based on the plurality of local features;

[0017] predicting a predicted object classification label corresponding to the sample object according to the feature correlation information;

[0018] calculating a model loss value based on the predicted object classification label and the object classification label;

[0019] training the image classification model according to the model loss value until a model training stop condition is reached, obtaining a trained image classification model;

[0020] returning model parameters of the image classification model to an end-side device.

[0021] According to a fourth aspect of the embodiments of the present specification, a classification method of a lesion in a human body image is provided, comprising:

[0022] obtaining a three-dimensional human body image, wherein the three-dimensional human body image includes global image information of a human body;

[0023] inputting the three-dimensional human body image into an image classification model to obtain a lesion type corresponding to the human body, wherein the image classification model is obtained by training based on feature correlation information of a sample image, the sample image includes global image information composed of a plurality of local features of a sample object, and the feature correlation information is obtained by correlating and analyzing the plurality of local features of the sample object.

[0024] According to a fifth aspect of the embodiments of the present specification, an image classification device is provided, comprising:

[0025] an obtaining module configured to obtain an image to be classified, wherein the image to be classified includes global image information of a target object;

[0026] The input module is configured to input the image to be classified into an image classification model to obtain the type information of the target object. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between the multiple local features of the sample object.

[0027] According to a sixth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0028] Memory and processor;

[0029] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described image classification method.

[0030] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the image classification method described above.

[0031] According to an eighth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the image classification method described above.

[0032] One embodiment of this specification implements the following: acquiring an image to be classified, wherein the image to be classified includes global image information of a target object; inputting the image to be classified into an image classification model to obtain type information of the target object, wherein the image classification model is trained based on feature association information of sample images, and the sample images include global image information composed of multiple local features of the sample object, the feature association information being obtained through association analysis between the multiple local features of the sample object. The image classification model performs association analysis based on multiple local features to obtain feature association information, thereby enabling the image classification model to be trained based on the feature association information of sample images. This allows the image classification model to perform image classification based on feature association information between local features, thereby improving the accuracy of the target object type output by the image classification model, i.e., improving the accuracy of the type information output by the image classification model based on the image to be classified. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating a scenario of image classification model training and application method provided in one embodiment of this specification;

[0034] Figure 2 This is a flowchart illustrating an image classification method provided in one embodiment of this specification;

[0035] Figure 3 This is a flowchart of another image classification method provided in one embodiment of this specification;

[0036] Figure 4 This is a flowchart illustrating an image classification model training method provided in one embodiment of this specification;

[0037] Figure 5 This is a flowchart illustrating the processing steps of a method for classifying lesions in human images, provided in one embodiment of this specification.

[0038] Figure 6 This is a flowchart illustrating a method for diagnosing Parkinson's disease according to one embodiment of this specification;

[0039] Figure 7 This is a schematic diagram of the structure of an image classification device provided in one embodiment of this specification;

[0040] Figure 8 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0041] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0042] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0044] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0045] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0046] CNN (Convolutional Neural Network) is a multi-layered supervised learning neural network. The hidden convolutional layers and pooling layers are the core modules that enable feature extraction. This network model uses gradient descent to minimize the loss function and adjusts the weight parameters layer by layer inversely, improving the network's accuracy through frequent iterative training. A convolutional neural network consists of one or more convolutional layers and a fully connected layer at the top (corresponding to a classic neural network), and also includes associated weights and pooling layers.

[0047] Transformer: The Transformer model is a neural network that learns context and thus meaning by tracking relationships in sequence data (such as words in a sentence).

[0048] Convolutional layer: In a convolutional neural network, each convolutional layer consists of several convolutional units, and the parameters of each convolutional unit are obtained through the backpropagation algorithm. The purpose of convolution is to extract different features from the input. The first convolutional layer may only extract some low-level features such as edges, lines, and corners, while more layers of the network can iteratively extract more complex features from low-level features.

[0049] This specification provides an image classification method, and also relates to an image classification device, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0050] See Figure 1 , Figure 1 The illustration shows a scenario diagram of an image classification model training and application method provided according to an embodiment of this specification, specifically including:

[0051] Figure 1The model architecture for the image classification model is as follows: Sample images, specifically 3D brain images, are obtained from the sample set. These sample images are input into part (a) of the image classification model, carrying object classification labels. Part (a) in the figure represents the prediction module. The sample images are input into the prediction module, where the convolutional stem of the prediction module performs convolution processing on the global image information of the sample images to obtain multiple local features. These local features are then input into global determination module 1 to calculate the feature relationship information 1 between each local feature. The feature relationship information 1 output by global determination module 1 is then input into global determination module 2 to obtain feature relationship information 2. This information is then input into global determination module 3 to obtain feature relationship information 3, and finally into global determination module 4 to obtain feature relationship information 4. Each global determination module has the same structure. Figure 1 The processing of part (3) is as follows: In part (3), the local features obtained by convolution are input into the input layer, and then input into the convolution layer, block layer, normalization, self-attention layer, feedforward layer, and recovery layer. Thus, the output layer outputs the feature relationship information before the local features. The feature relationship information can show the correlation between the local features, that is, it can represent the metabolic effects between different regions of the brain.

[0052] The feature relationship information 4 output by the global determination module 4 is then processed by the pooling layer, fully connected layer and classification layer in the prediction module to obtain the predicted object classification label; then the classification loss value can be calculated based on the object classification label and the predicted object classification label.

[0053] To improve the classification accuracy of the image classification model, a further feature was added to the image classification model. Figure 1 The image reconstruction module in section (b) combines the output of the global determination module 1-4 to reconstruct the image, obtaining the predicted reconstructed image and the predicted object local label; a portion of the sample image is masked to obtain the mask image and the object local label of the mask image. Here, the masking of the image is to add weight features to the portion of the image; the reconstruction loss value is calculated based on the object local label, i.e. the features of the masked portion and the predicted object local label, i.e. the features of the predicted masked portion.

[0054] The reconstruction loss and classification loss are used as model loss values, and the model parameters of the image classification model are adjusted based on the model loss values ​​until the training of the image classification model is complete.

[0055] When applying an image classification model, the image to be classified can be input into the prediction module in part (a), without needing to perform the image reconstruction operation in part (b). For example, inputting a brain image to be classified into a trained image classification model will result in the brain object in the brain image being classified as type A.

[0056] The image classification method described in this specification involves acquiring an image to be classified, wherein the image to be classified includes global image information of a target object; inputting the image to be classified into an image classification model to obtain type information of the target object; wherein the image classification model is trained based on feature association information of sample images, and the sample images include global image information composed of multiple local features of the sample object, and the feature association information is obtained by performing association analysis between the multiple local features of the sample object. The image classification model performs association analysis based on multiple local features to obtain feature association information, thereby enabling the image classification model to be trained based on the feature association information of sample images. This allows the image classification model to perform image classification based on feature association information between local features, thereby improving the accuracy of the target object type output by the image classification model, i.e., improving the accuracy of the type information output by the image classification model based on the image to be classified.

[0057] See Figure 2 , Figure 2 The illustration shows a scene diagram of an image classification method according to an embodiment of this specification, which specifically includes the following steps.

[0058] Step 202: Obtain the image to be classified, wherein the image to be classified contains global image information of the target object.

[0059] Among them, the image to be classified refers to an image that has a classification requirement, such as an animal image in which the classification of animals in the image needs to be determined; the image to be classified contains a target object, which refers to the main body in the image to be classified, such as the main body of the person in the image of people, or the main body of the tree in the image of trees; global image information refers to the overall image information of the target object in the image to be classified.

[0060] Specifically, the process involves receiving an image processing request; parsing the image processing request to obtain an image classification identifier; obtaining an image to be classified based on the image classification identifier, which contains global image information of the target object; or obtaining an image to be classified based on a classification task.

[0061] In one specific embodiment of this specification, an image classification request is received; the image classification request is parsed to obtain the image to be classified, "animal image"; and global image information of the animal object is included in the image to be classified.

[0062] By acquiring the images to be classified, we can then classify the identified images.

[0063] Step 204: Input the image to be classified into the image classification model to obtain the type information of the target object. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between multiple local features of the sample object.

[0064] Here, an image classification model refers to a model that can output corresponding type information based on an input image; type information refers to the type of the image. For example, if the type of the animal object in image A is 'a', then the type information of image A is "a"; if the type of the animal object in image B is 'b', then the type information of image B is "b"; a sample image refers to an image used to train the image classification model; in practical applications, sample images carry classification labels, and the classification loss value can be calculated based on the predicted classification and classification labels output by the image classification model to adjust the model parameters of the image classification model; feature association information refers to the association information between multiple local features; a sample object refers to the main body in the sample image; local features refer to the information obtained by locally extracting the global image information of the sample object; global image information refers to the overall image information of the sample object in the sample image.

[0065] Specifically, before inputting the image to be classified into the image classification model, the image classification model needs to be trained. That is, the sample image is input into the image classification model to determine the global image information corresponding to the sample object in the sample image; multiple local features corresponding to the sample object are collected from the global image information; correlation analysis is performed between multiple local features to obtain feature correlation information; the image classification model is trained based on the feature correlation information until the trained image classification model is obtained; the image to be classified is input into the trained image classification model to obtain the type information of the target object output by the image classification model.

[0066] In one specific embodiment of this specification, the image to be classified is determined to be a plant image; the plant image is input into the trained image classification model; and the type information of the pine tree in the plant image output by the image classification model based on the plant image is obtained as "terrestrial plant".

[0067] In practical applications, methods for inputting the image to be classified into an image classification model to obtain the type information of the target object may include:

[0068] The image to be classified is input into the prediction module of the image classification model to obtain the predicted classification weight information;

[0069] Based on the predicted classification weight information, the type information of the target object is determined.

[0070] The prediction module refers to the module used to classify and predict the image type in the image; the prediction classification weight information refers to the prediction weight of the target object for each preset type. For example, the prediction classification weight information output by the image classification model for plant images is "the probability of pine trees being classified as terrestrial plants is 2%, and the probability of pine trees being classified as aquatic plants is 98%"; the preset classification refers to the classification set in advance for the classification task of the image classification model. For example, the plant images can be pre-set to be classified as terrestrial plant types and aquatic plant types.

[0071] Specifically, the method for obtaining the predicted classification weight information output by the prediction module of the image classification model and determining the type information of the target object based on the predicted classification weight information is not specifically limited in this specification. For example, preset types with weight information greater than or equal to the weight threshold can be selected from the preset classification weight information as the type information of the target object; or the weight information corresponding to each predicted type can be arranged in descending order, and the preset type with the largest weight can be selected as the type information of the target object, etc.

[0072] In a specific embodiment of this specification, the image of the plant to be classified is input into the prediction module of the image classification model to obtain the prediction weight classification information; if the prediction classification information is determined to include "the probability of pine trees being classified as terrestrial plants is 2% and the probability of pine trees being classified as aquatic plants is 98%", then the type with the highest classification probability, the terrestrial plant type, is selected as the classification information corresponding to the pine trees in the image of the plant to be classified.

[0073] In practical applications, image classification models can be trained using the feature association information of sample images, thereby improving the classification accuracy of the image classification models.

[0074] Specifically, before inputting the image to be classified into the image classification model, the following steps are also included:

[0075] Obtain sample images from the sample set, wherein the sample images contain global image information of the sample objects and object classification labels;

[0076] The sample image is input into an image classification model to analyze the global image information and obtain multiple local features.

[0077] Calculate the feature association information between the multiple local features based on the multiple local features;

[0078] Predict the predicted object classification label corresponding to the sample object based on the feature association information;

[0079] The model loss value is calculated based on the predicted object classification label and the object classification label;

[0080] The image classification model is trained based on the model loss value until the model training stops, thus obtaining a trained image classification model.

[0081] Here, the sample set refers to the collection of sample images, which contain labels used to train the image classification model; the object classification label refers to the label carried by the sample image for supervised training of the model, for example, a terrestrial plant label is set in advance for pine tree objects in a plant image; the predicted object classification label refers to the object classification label output by the image classification model based on the sample image; and the model loss value refers to the loss value used to train the image generation model.

[0082] Specifically, a sample set for training the image generation model is obtained, containing sample images carrying object classification labels. Sample images are randomly selected from the sample set and input into the image classification model. The image classification model extracts multiple local features from the global image information corresponding to the samples in the sample images. Feature association information between these local features is calculated, and the predicted object classification label corresponding to the sample object is predicted based on this feature association information. The model loss value is calculated based on a preset loss function, a preset object classification label, and the object classification label. The model parameters of the image classification model are adjusted using the calculated model loss value. The above steps are repeated until the model training stopping condition is met, resulting in a trained image classification model.

[0083] By analyzing the global image information of the sample object, multiple local features are obtained; then the feature correlation between each local feature is calculated, thereby avoiding the problem of inaccurate prediction caused by classification prediction based solely on local features. In other words, the feature correlation between each local feature is used to train the image classification model, enabling the image classification model to classify target objects by combining the relationship between local features, thus improving the classification accuracy of the image generation model.

[0084] In practical applications, in order to further improve the classification accuracy of image classification models, the loss generated by image reconstruction can be combined with the model training process.

[0085] Specifically, before calculating the model loss value based on the predicted object classification label and the object classification label, the following steps are also included:

[0086] Obtain sample reconstructed images from the sample set, wherein the sample reconstructed images carry object local labels;

[0087] Accordingly, the model loss value is calculated based on the predicted object classification label and the object classification label, including:

[0088] Calculate the classification loss value based on the predicted object classification label and the object classification label;

[0089] A predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local label and the object local label of the predicted reconstructed image;

[0090] The model loss value is determined based on the classification loss value and the reconstruction loss value.

[0091] Among them, the sample reconstructed image refers to the image obtained by locally annotating the sample image, which is used to calculate the reconstruction loss value; the object local label refers to the label added to the weight of some features in the sample reconstructed image; the classification loss value refers to the loss value between the object classification label and the predicted object classification label; the predicted reconstructed image refers to the image generated by reconstructing the image based on the feature association information of the sample image; the predicted object local label refers to the object local label carried in the predicted reconstructed image; and the reconstruction loss value refers to the loss value between the predicted object local label and the object local label.

[0092] Specifically, sample reconstructed images can be pre-added to the sample set; the sample reconstructed images in the sample set can be input into the image classification model; based on the feature association information of the sample images in the image classification model, the pre-reconstructed images can be constructed; the reconstruction loss value can be calculated based on the predicted object local label and the object local label of the constructed pre-reconstructed images; the classification loss value can be calculated based on the predicted object classification label and the object classification label; the model loss value can be calculated based on the classification loss value and the reconstruction loss value, for example, the sum of the classification loss value and the reconstruction loss value can be used as the model loss value.

[0093] In a specific embodiment of this specification, sample pairs are collected in a sample set, each containing a sample image and a reconstructed sample image. The sample pairs are input into an image classification model, and multiple local features are extracted from the global image information of the sample objects. Feature association information between the multiple local features is calculated. Based on the feature association information, the predicted object classification label of the sample image is predicted, and a classification loss value is calculated based on the object classification label of the sample image and the predicted object classification label. A predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local label of the predicted reconstructed image and the object local label of the sample image. The difference between the reconstruction loss value and the classification loss value is used as the model loss value, and the model parameters of the image classification model are adjusted based on the model loss value until the model training stopping condition is met, thus obtaining a trained image classification model.

[0094] In practical applications, the local labels of objects in the reconstructed images are pre-set. Specifically, before acquiring sample images from the sample set, the following steps are also taken:

[0095] Obtain the image to be labeled and the image object template corresponding to the image to be labeled;

[0096] Add local object labels to the objects in the image to be labeled based on the image object template;

[0097] The images to be labeled, carrying the local labels of the objects, are used as sample reconstructed images and added to the sample set.

[0098] Here, the image to be labeled refers to an image that can be locally labeled; the image object template refers to a template used to determine the sub-objects to be labeled on the object to be labeled in the image to be labeled, for example, determining the cerebellum and brainstem in a human brain image based on the image object template.

[0099] Specifically, based on the image object template of the image to be labeled, the labelable regions of the objects to be labeled in the image are determined; further, labels are added to the labelable regions to generate local labels for the objects in the image to be labeled; in practical applications, the labeling operation can be performed by relevant technical personnel or by a pre-trained labeling model, and this specification does not make specific limitations; by adding labels to some sub-objects in the target object, that is, increasing the feature weight of some sub-objects in the sample reconstructed image, so that the image classification model can be strengthened to process the features of these sub-objects through the reconstruction loss value, thereby improving the classification accuracy of the image classification model.

[0100] In one specific embodiment of this specification, the image to be labeled is determined to be a brain image; based on the image object template, the brain region and brainstem region in the image to be labeled are determined; a doctor labels the brain region in the image to be labeled to obtain an image to be labeled carrying local object labels; the image to be labeled is used as a sample to reconstruct an image and added to the sample set for training the image classification model.

[0101] By adding labels to certain regions of the image to be labeled, i.e. increasing the feature weight of these regions in the reconstructed image, the image classification model can be trained using the calculated reconstruction loss value. This strengthens the image classification model's processing of the labeled features and improves its classification accuracy.

[0102] In practical applications, image classification models contain multiple modules, each performing different data processing.

[0103] Specifically, the image classification model includes a prediction module, which in turn includes a local extraction module and a global extraction module, wherein the global extraction module is composed of multiple neural networks;

[0104] The sample image is input into an image classification model to analyze the global image information and obtain multiple local features, including:

[0105] The global image information of the sample image is input into the local extraction module to obtain multiple local features corresponding to the sample object;

[0106] Calculating feature association information between multiple local features based on the aforementioned multiple local features includes:

[0107] The multiple local features are input into the global extraction module to obtain feature association information between the multiple local features.

[0108] The prediction module is a module that outputs the corresponding prediction type information based on the input sample image. The prediction module includes a local extraction module and a global extraction module. The local extraction module is a module that outputs multiple local features based on the input global image information. The global extraction module is a module that outputs the feature association information between multiple local features based on the input multiple local features.

[0109] In a specific embodiment of this specification, the framework of the image classification module is determined to consist of a CNN model and a transformer model; the local extraction module in the image classification model is a convolutional layer of the CNN model, which is used to extract multiple local features from the global image information; the global extraction module in the image classification model is a transformer model, which is used to calculate the feature relationship information between multiple local features.

[0110] After local features are extracted by the local extraction module of the image classification model, the feature relationship information between local features is calculated by the global extraction module. This reduces the limitations caused by using only local features for image classification and allows the image classification model to be trained based on data containing the relationship between features.

[0111] Furthermore, the image classification model can also include an image reconstruction module to reconstruct the image, and then the model parameters of the image classification model can be adjusted based on the reconstruction loss value.

[0112] Specifically, a predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local labels and the object local labels of the predicted reconstructed image, including:

[0113] The feature association information is input into the image reconstruction module of the image classification model to obtain the predicted reconstructed image and the local labels of the predicted objects corresponding to the preset reconstructed image;

[0114] The reconstruction loss value is calculated based on the predicted object's local label and the object's local label.

[0115] The image reconstruction module refers to the module that outputs a predicted reconstructed image and the corresponding local labels of the predicted objects based on the input feature relationship information.

[0116] Specifically, the feature association relationship is input into the image reconstruction module of the image classification model to obtain the predicted local label of the object output by the image reconstruction module; based on the object local label and the predicted object local label of the reconstructed image, the reconstruction loss value is calculated.

[0117] In a specific embodiment of this specification, the image reconstruction module of the image classification model consists of an upsampling layer and two convolutional layers. Feature relationship information is input into the image reconstruction module, the upsampling layer amplifies the local features, and the amplified features are further processed based on the two convolutional layers to obtain reconstructed image features. Based on the reconstructed image features, the local label of the predicted object is determined, that is, the local label of the predicted object is output by the image reconstruction module. The reconstruction loss value is calculated based on the local label of the predicted object and the local label of the object.

[0118] By utilizing the image reconstruction module of the image classification model during the image classification process, the local labels of the predicted objects can be obtained. Based on the predicted local labels and the local labels of the objects, the reconstruction loss value can be calculated, and the model parameters of the image classification model can be adjusted based on the reconstruction loss value to improve the classification efficiency of the image classification model.

[0119] In practical applications, there may be uncertainties when setting object classification labels for sample images. For example, it may be impossible to determine whether a plant in an image is a terrestrial or aquatic plant. Therefore, a presumed classification label can be set for the sample image, and other pre-trained image classification models can be used to predict the classification label, thereby selecting the label that appears most frequently from multiple labels.

[0120] Specifically, before acquiring sample images from the sample set, the following steps are also included:

[0121] Acquire a target image, wherein the target image contains a presumed classification label;

[0122] The target image is input into the first image classification model to obtain the first predicted classification label;

[0123] The target image is input into the second image classification model to obtain the second predicted classification label;

[0124] Select a target classification label from the inferred classification label, the first predicted classification label, and the second predicted classification label, and add it to the target image;

[0125] The target image carrying the target classification label is added to the sample set as a sample image.

[0126] Here, the target image refers to an image whose category cannot be clearly defined; the first image classification model refers to a model that can output corresponding classification labels based on the input image, and its model architecture is different from that of the second image classification model; the second image classification model refers to a model that can output corresponding classification labels based on the input image, and its model architecture is different from that of the second image classification model.

[0127] Specifically, a speculative classification label is set for the target image; the target image is input into image classification models with different model architectures to obtain different predicted classification labels; among the predicted and speculative classification labels, the target classification label with the highest frequency of occurrence is selected as the object classification label of the target image, and the target image carrying the target classification label is added to the sample set as a sample image, thereby increasing the number of samples in the sample set.

[0128] In one specific embodiment of this specification, a plant image is acquired, carrying a predicted label "aquatic plant type"; a first image classification model with a CNN model architecture and a second image classification model with a Transformer model architecture are determined; the plant image is input into the first image classification model to obtain the first predicted classification label "aquatic plant type" output by the model, and the plant image is input into the second image classification model to obtain the second predicted classification label "terrestrial plant type" output by the model; if it is determined that "aquatic plant type" has more entries among the three classification labels, then the plant image carrying the label "aquatic plant type" is added to the sample set.

[0129] By adding speculative classification labels to objects in images with uncertain classifications, and then predicting the classification labels of images based on image classification models with different model architectures, the label with the highest probability of occurrence among multiple labels is selected as the label of the image, and the image is added to the sample set, thereby enriching the number of samples. Training the image classification model with a richer sample set of sample images is beneficial to improving the classification accuracy of the image classification model.

[0130] The image classification method described in this specification involves acquiring an image to be classified, wherein the image to be classified includes global image information of a target object; inputting the image to be classified into an image classification model to obtain type information of the target object; wherein the image classification model is trained based on feature association information of sample images, and the sample images include global image information composed of multiple local features of the sample object, and the feature association information is obtained by performing association analysis between the multiple local features of the sample object. The image classification model performs association analysis based on multiple local features to obtain feature association information, thereby enabling the image classification model to be trained based on the feature association information of sample images. This allows the image classification model to perform image classification based on feature association information between local features, thereby improving the accuracy of the target object type output by the image classification model, i.e., improving the accuracy of the type information output by the image classification model based on the image to be classified.

[0131] See Figure 3 , Figure 3 A flowchart of another image classification method according to an embodiment of this specification is shown, applied to a cloud-side device, and specifically includes the following steps.

[0132] Step 302: Obtain the image to be classified sent by the end device, wherein the image to be classified includes global image information of the target object.

[0133] Step 304: Input the image to be classified into the image classification model to obtain the type information of the target object. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between multiple local features of the sample object.

[0134] Step 306: Return the type information to the end-side device.

[0135] This specification describes an image classification method applied to cloud-side devices. It acquires an image to be classified sent by an end-side device, wherein the image to be classified includes global image information of a target object; inputs the image to be classified into an image classification model to obtain type information of the target object, wherein the image classification model is trained based on feature association information of sample images, the sample images including global image information composed of multiple local features of the sample object, and the feature association information is obtained by performing association analysis between the multiple local features of the sample object; and returns the type information to the end-side device.

[0136] By receiving the image to be classified from the edge device and classifying the objects in the image based on the image classification model on the cloud device, the type information corresponding to the objects is obtained, thereby avoiding the application of the image classification model on the edge device and saving the computing resources of the edge device.

[0137] See Figure 4 , Figure 4 A flowchart illustrating a training method for an image classification model provided according to an embodiment of this specification, applied to a cloud-side device, is shown, specifically including the following steps.

[0138] Step 402: Obtain sample images from the sample set, wherein the sample images contain global image information of the sample objects and object classification labels.

[0139] Step 404: Input the sample image into the image classification model, analyze the global image information, and obtain multiple local features.

[0140] Step 406: Calculate the feature association information between the multiple local features based on the multiple local features.

[0141] Step 408: Predict the predicted object classification label corresponding to the sample object based on the feature association information.

[0142] Step 410: Calculate the model loss value based on the predicted object classification label and the object classification label.

[0143] Specifically, before calculating the model loss value based on the predicted object classification label and the object classification label, it is necessary to obtain sample reconstruction images in the sample set, wherein the sample reconstruction images carry object local labels; further, a classification loss value is calculated based on the predicted object classification label and the object classification label; a predicted reconstruction image is constructed according to the feature association information, and a reconstruction loss value is calculated based on the predicted object local labels and the object local labels of the predicted reconstruction image; the model loss value is determined based on the classification loss value and the reconstruction loss value.

[0144] Step 412: Train the image classification model based on the model loss value until the model training stops, and obtain the trained image classification model.

[0145] Step 414: Return the model parameters of the image classification model to the edge device.

[0146] The image classification model training method described in this specification involves: acquiring sample images from a sample set, wherein the sample images contain global image information of the sample objects and object classification labels; inputting the sample images into an image classification model, analyzing the global image information to obtain multiple local features; calculating feature association information between the multiple local features; predicting the predicted object classification label corresponding to the sample object based on the feature association information; calculating the model loss value based on the predicted object classification label and the object classification label; training the image classification model based on the model loss value until the model training stopping condition is met to obtain a trained image classification model; and returning the model parameters of the image classification model to the edge device.

[0147] By training the image classification model on the cloud-side device, and then feeding back the model parameters to the edge device upon receiving an image classification model request, the cloud-side device completes the training of the model, thus saving the computing resources of the edge device.

[0148] The following is in conjunction with the appendix Figure 5 Taking the application of the image classification method provided in this specification to human body images as an example, the image classification method will be further explained. Among other things, Figure 5 The flowchart illustrates a method for classifying lesions in human images according to an embodiment of this specification, which specifically includes the following steps.

[0149] Step 502: Obtain a three-dimensional human body image, wherein the three-dimensional human body image includes global image information of the human body.

[0150] Specifically, a three-dimensional human body image refers to a human body image in 3D file format.

[0151] Step 504: Input the three-dimensional human body image into the image classification model to obtain the lesion type corresponding to the human body. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between multiple local features of the sample object.

[0152] Specifically, before inputting the three-dimensional human image into the image classification model, the image classification model can be trained based on a sample set containing human sample images. The specific training process is the same as the model training process in the above embodiment, and will not be repeated here.

[0153] This specification describes a method for classifying lesions in human images. The method involves acquiring a three-dimensional human image, which includes global image information of the human body. The three-dimensional human image is then input into an image classification model to obtain the corresponding lesion type. The image classification model is trained based on feature association information from sample images. The sample images include global image information composed of multiple local features of the sample object. The feature association information is obtained through correlation analysis between the multiple local features of the sample object.

[0154] By training the image classification model based on the feature association information of sample images, the image classification model is equipped with the function of classifying images based on the feature association information between local features, thereby improving the accuracy of the lesion type output by the image classification model.

[0155] See Figure 6 , Figure 6 A flowchart of a Parkinson's disease diagnostic method according to an embodiment of this specification is shown, specifically including the following steps:

[0156] Step 602: Train the image classification model.

[0157] Specifically, brain images are acquired from a brain image set, and inferred classification labels are added to these images to obtain target images. These inferred classification labels can be added by doctors diagnosing Parkinson's disease. The target image is then input into a first image classification model to obtain a first predicted classification label. Next, the target image is input into a second image classification model to obtain a second predicted classification label. A target classification label is selected from the target label, the first predicted label, and the second predicted label, and added to the target image. If the target classification label matches the inferred label, no new label needs to be added to the target image. If the target classification label is either the first or second predicted label, the inferred classification label carried by the target image needs to be replaced with the target classification label to obtain a target image carrying the target classification label. The target image carrying the target classification label is then added to the sample set as a sample image.

[0158] Among them, the first image classification model and the second image classification model are two classification models with different model architectures; by selecting the target classification label from the first predicted classification label, the second predicted classification label, and the inferred classification label, the accuracy of the label carried by the sample image is guaranteed, thereby improving the accuracy of training the image classification model.

[0159] The above is one way to generate sample images. In practical applications, images carrying speculative classification labels, first predicted classification labels, and second predicted classification labels can also be used as sample images, or images carrying classification labels generated based on other methods can be used as sample images to generate sample sets.

[0160] Sample images are acquired from a sample set, wherein the sample images contain global image information of the sample objects and object classification labels; the image classification model includes a prediction module, which includes a local extraction module and a global extraction module, wherein the global extraction module is composed of multiple neural networks; the sample images are input into the image classification model to analyze the global image information and obtain multiple local features, specifically: the global image information of the sample images is input into the local extraction module to obtain multiple local features corresponding to the sample objects; based on the multiple local features, feature association information between the multiple local features is calculated, including: the multiple local features are input into the global extraction module to obtain feature association information between the multiple local features.

[0161] Calculate the feature association information between the multiple local features; predict the prediction object classification label corresponding to the sample object based on the feature association information.

[0162] The model loss value is calculated based on the predicted object classification label and the object classification label. Specifically, sample reconstructed images are obtained from the sample set, wherein the sample reconstructed images carry object local labels, and a classification loss value is calculated based on the predicted object classification label and the object classification label. The feature association information is input into the image reconstruction module of the image classification model to obtain the predicted object local labels corresponding to the predicted reconstructed image and the preset reconstructed image, and a reconstruction loss value is calculated based on the predicted object local labels of the predicted reconstructed image and the object local labels. The model loss value is determined based on the classification loss value and the reconstruction loss value.

[0163] The image classification model is trained based on the model loss value until the model training stops, thus obtaining a trained image classification model.

[0164] Step 604: Obtain the brain image to be classified, wherein the brain image to be classified includes global image information of the brain.

[0165] Step 606: Input the brain image to be classified into the trained image classification model to obtain the Parkinson's disease type information corresponding to the brain. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between multiple local features of the sample object.

[0166] Specifically, the brain image to be classified is input into the prediction module of the image classification model to obtain predicted classification weight information; based on the predicted classification weight information, the Parkinson's disease type information corresponding to the brain is determined.

[0167] See Table 1 below, which outputs the performance of the Parkinson's disease diagnostic method in this specification and compares the diagnostic effectiveness of different baselines for Parkinson's disease. The mean accuracy (Acc.), standard deviation (std), F1 score, sensitivity, and specificity of the nested five-fold cross-validation are reported. The last column shows the model size in terms of the number of trainable parameters (M). * indicates a p-value <0.001 compared to the reference value (i.e., our method in this specification); in Table 1, multiple system atrophy (MSA), progressive superannuclear palsy (PSP), and idiopathic Parkinson's disease (IPD) represent different types of Parkinson's disease.

[0168] Table 1

[0169]

[0170] See Table 2 below for ablation studies. The mean accuracy of standard deviation (std), p-value, F1 score, sensitivity, and specificity in nested five-fold cross-validation with different configurations. Where B (backbone), EF (early fusion), IR (image reconstruction), MIR (masked image reconstruction), SL (self-learning), and SLE (self-learning with ensemble) are used.

[0171] Table 2

[0172]

[0173] Ablation experiments were conducted to validate the efficacy of each component in the proposed framework. The mean accuracy, F1 score, sensitivity, and specificity of nested five-fold cross-validation with different configurations are reported in Table 2. Compared to the baseline methods in Table 1, the a) MetaViT backbone of this specification outperforms both CNN-based and ViT-based methods, demonstrating the clear advantage of modeling metabolic interactions in relevant brain regions. To evaluate the effectiveness of the masked image reconstruction task, it was compared with two other methods: b) Early fusion, which fuses the mask as an additional channel of the input image, is the simplest way to integrate prior knowledge with the mask; c) Image reconstruction, which reconstructs the original image instead of the masked image, observed that the early fusion strategy did not help diagnostic performance, and original image reconstruction even reduced accuracy. Conversely, d) The masked image reconstruction task significantly improved accuracy from 95.5% to 96.4% (p = 0.06), implying the effectiveness of integrating prior knowledge into model training.

[0174] The Parkinson's disease diagnosis method in this specification uses an image classification model to perform correlation analysis based on multiple local features to obtain feature correlation information. This information allows the image classification model to be trained based on the feature correlation information of sample images, enabling it to classify images based on the feature correlation information between local features. This improves the accuracy of the target object type output by the image classification model. The trained image classification model is then used to process the brain image to be classified to determine the Parkinson's disease type information corresponding to the brain, thus improving the accuracy of the Parkinson's disease type information output by the image classification model based on the brain image to be classified.

[0175] Corresponding to the above method embodiments, this specification also provides embodiments of image classification devices. Figure 7 A schematic diagram of an image classification device according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0176] The acquisition module 702 is configured to acquire an image to be classified, wherein the image to be classified includes global image information of the target object;

[0177] The input module 704 is configured to input the image to be classified into an image classification model to obtain the type information of the target object. The image classification model is trained based on the feature association information of the sample image. The sample image includes global image information composed of multiple local features of the sample object. The feature association information is obtained by performing association analysis between the multiple local features of the sample object.

[0178] Optionally, the device further includes a training module, the training module being configured to:

[0179] Obtain sample images from the sample set, wherein the sample images contain global image information of the sample objects and object classification labels;

[0180] The sample image is input into an image classification model to analyze the global image information and obtain multiple local features.

[0181] Calculate the feature association information between the multiple local features based on the multiple local features;

[0182] Predict the predicted object classification label corresponding to the sample object based on the feature association information;

[0183] The model loss value is calculated based on the predicted object classification label and the object classification label;

[0184] The image classification model is trained based on the model loss value until the model training stops, thus obtaining a trained image classification model.

[0185] Optionally, the apparatus further includes an acquisition submodule, the acquisition submodule being configured to:

[0186] Obtain sample reconstructed images from the sample set, wherein the sample reconstructed images carry object local labels;

[0187] Accordingly, the model loss value is calculated based on the predicted object classification label and the object classification label, including:

[0188] Calculate the classification loss value based on the predicted object classification label and the object classification label;

[0189] A predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local label and the object local label of the predicted reconstructed image;

[0190] The model loss value is determined based on the classification loss value and the reconstruction loss value.

[0191] Optionally, the image classification model includes a prediction module, which includes a local extraction module and a global extraction module, wherein the global extraction module is composed of multiple neural networks;

[0192] The training module is further configured as follows:

[0193] The global image information of the sample image is input into the local extraction module to obtain multiple local features corresponding to the sample object;

[0194] Calculating feature association information between multiple local features based on the aforementioned multiple local features includes:

[0195] The multiple local features are input into the global extraction module to obtain feature association information between the multiple local features.

[0196] Optionally, the training module is further configured to:

[0197] The feature association information is input into the image reconstruction module of the image classification model to obtain the predicted reconstructed image and the local labels of the predicted objects corresponding to the preset reconstructed image;

[0198] The reconstruction loss value is calculated based on the predicted object's local label and the object's local label.

[0199] Optionally, the apparatus further includes a sample reconstruction image adding module, configured to:

[0200] Obtain the image to be labeled and the image object template corresponding to the image to be labeled;

[0201] Add local object labels to the objects in the image to be labeled based on the image object template;

[0202] The images to be labeled, carrying the local labels of the objects, are used as sample reconstructed images and added to the sample set.

[0203] Optionally, the apparatus further includes a sample image adding module, configured to:

[0204] Acquire a target image, wherein the target image contains a presumed classification label;

[0205] The target image is input into the first image classification model to obtain the first predicted classification label;

[0206] The target image is input into the second image classification model to obtain the second predicted classification label;

[0207] Select a target classification label from the inferred classification label, the first predicted classification label, and the second predicted classification label, and add it to the target image;

[0208] The target image carrying the target classification label is added to the sample set as a sample image.

[0209] Optionally, the input module 704 is further configured to:

[0210] The image to be classified is input into the prediction module of the image classification model to obtain the predicted classification weight information;

[0211] Based on the predicted classification weight information, the type information of the target object is determined.

[0212] The image classification device of this specification includes an acquisition module configured to acquire an image to be classified, wherein the image to be classified includes global image information of a target object; and an input module configured to input the image to be classified into an image classification model to obtain type information of the target object, wherein the image classification model is trained based on feature association information of a sample image, the sample image including global image information composed of multiple local features of the sample object, and the feature association information being obtained by performing association analysis between the multiple local features of the sample object.

[0213] Image classification models perform correlation analysis based on multiple local features to obtain feature correlation information. This information can then be used to train the image classification model, enabling it to classify images based on the feature correlation information between local features. This improves the accuracy of the target object type output by the image classification model, i.e., it improves the accuracy of the type information output by the image classification model based on the image to be classified.

[0214] The above is a schematic scheme of an image classification device according to this embodiment. It should be noted that the technical solution of this image classification device and the technical solution of the image classification method described above belong to the same concept. For details not described in detail in the technical solution of the image classification device, please refer to the description of the technical solution of the image classification method described above.

[0215] Figure 8 A structural block diagram of a computing device 800 according to one embodiment of this specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0216] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0217] In one embodiment of this specification, the above-described components of the computing device 800 and Figure 8 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 8 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0218] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0219] The processor 820 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described image classification method.

[0220] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the image classification method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the image classification method described above.

[0221] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the image classification method described above.

[0222] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the image classification method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the image classification method described above.

[0223] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described image classification method.

[0224] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the image classification method described above belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the image classification method described above.

[0225] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0226] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0227] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0228] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0229] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image classification method, comprising: Obtain the image to be classified, wherein the image to be classified includes global image information of the target object; The image to be classified is input into an image classification model to obtain the type information of the target object. The image classification model is trained as follows: Sample images are acquired from a sample set, each containing global image information and object classification labels for the sample object. These sample images are input into the image classification model, and the global image information is analyzed to obtain multiple local features. Feature association information between these local features is calculated, and a predicted object classification label corresponding to the sample object is predicted based on the feature association information. A classification loss value is calculated based on the predicted object classification label and the object classification label. A sample reconstruction image is acquired from the sample set, carrying local object labels. A predicted reconstruction image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local labels and the object local labels. A model loss value is determined based on the classification loss value and the reconstruction loss value. The image classification model is trained based on the model loss value until the model training stops, resulting in a trained image classification model.

2. The method as described in claim 1, wherein the image classification model includes a prediction module, and the prediction module includes a local extraction module and a global extraction module, wherein, The global extraction module consists of multiple neural networks; The sample image is input into an image classification model to analyze the global image information and obtain multiple local features, including: The global image information of the sample image is input into the local extraction module to obtain multiple local features corresponding to the sample object; Calculating feature association information between multiple local features based on the aforementioned multiple local features includes: The multiple local features are input into the global extraction module to obtain feature association information between the multiple local features.

3. The method as described in claim 1, comprising constructing a predicted reconstructed image based on the feature association information, and calculating a reconstruction loss value based on the predicted object local labels of the predicted reconstructed image and the object local labels, comprising: The feature association information is input into the image reconstruction module of the image classification model to obtain the predicted reconstructed image and the local label of the predicted object corresponding to the predicted reconstructed image; The reconstruction loss value is calculated based on the predicted object's local label and the object's local label.

4. The method of claim 1, further comprising, before acquiring sample images from the sample set: Obtain the image to be labeled and the image object template corresponding to the image to be labeled; Add local object labels to the objects in the image to be labeled based on the image object template; The images to be labeled, carrying the local labels of the objects, are used as sample reconstructed images and added to the sample set.

5. The method of claim 1, further comprising, before acquiring sample images from the sample set: Acquire a target image, wherein the target image contains a presumed classification label; The target image is input into the first image classification model to obtain the first predicted classification label; The target image is input into the second image classification model to obtain the second predicted classification label; Select a target classification label from the inferred classification label, the first predicted classification label, and the second predicted classification label, and add it to the target image; The target image carrying the target classification label is added to the sample set as a sample image.

6. The method as described in claim 1, wherein the image to be classified is input into an image classification model to obtain the type information of the target object, comprising: The image to be classified is input into the prediction module of the image classification model to obtain the predicted classification weight information; Based on the predicted classification weight information, the type information of the target object is determined.

7. An image classification method applied to cloud-side devices, comprising: Acquire the image to be classified sent by the end device, wherein the image to be classified includes global image information of the target object; The image to be classified is input into an image classification model to obtain the type information of the target object. The image classification model is trained in the following manner: sample images are obtained from a sample set, the sample images containing global image information of the sample object and object classification labels; the sample images are input into the image classification model, the global image information is analyzed to obtain multiple local features, feature association information between the multiple local features is calculated based on the multiple local features, the predicted object classification label corresponding to the sample object is predicted based on the feature association information, and a classification loss value is calculated based on the predicted object classification label and the object classification label; a sample reconstruction image is obtained from the sample set, the sample reconstruction image carrying object local labels; a predicted reconstruction image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local labels of the predicted reconstruction image and the object local labels; a model loss value is determined based on the classification loss value and the reconstruction loss value; the image classification model is trained based on the model loss value until the model training stopping condition is reached, and a trained image classification model is obtained. The type information is returned to the terminal device.

8. A method for training an image classification model, applied to a cloud-based device, for training the image classification model according to any one of claims 1-6, comprising: Obtain sample images from the sample set, wherein the sample images contain global image information of the sample objects and object classification labels; The sample image is input into an image classification model to analyze the global image information and obtain multiple local features. Calculate the feature association information between the multiple local features based on the multiple local features; Predict the predicted object classification label corresponding to the sample object based on the feature association information; Calculate the classification loss value based on the predicted object classification label and the object classification label; Obtain sample reconstructed images from the sample set, wherein the sample reconstructed images carry object local labels; A predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local label and the object local label of the predicted reconstructed image; The model loss value is determined based on the classification loss value and the reconstruction loss value; The image classification model is trained based on the model loss value until the model training stops, thus obtaining a trained image classification model. The model parameters of the image classification model are returned to the edge device.

9. A method for classifying lesions in human images, comprising: Acquire a three-dimensional human body image, wherein the three-dimensional human body image includes global image information of the human body; The three-dimensional human image is input into an image classification model to obtain the lesion type corresponding to the human body. The image classification model is trained as follows: sample images are acquired from a sample set, the sample images containing global image information and object classification labels of the sample objects. The sample images are input into the image classification model, the global image information is analyzed to obtain multiple local features, feature association information between the multiple local features is calculated based on the multiple local features, the predicted object classification label corresponding to the sample object is predicted based on the feature association information, and a classification loss value is calculated based on the predicted object classification label and the object classification label. Sample reconstructed images are acquired from the sample set, the sample reconstructed images carrying object local labels. A predicted reconstructed image is constructed based on the feature association information, and a reconstruction loss value is calculated based on the predicted object local label and the object local label of the predicted reconstructed image. The model loss value is determined based on the classification loss value and the reconstruction loss value. The image classification model is trained based on the model loss value until the model training stopping condition is reached, and a trained image classification model is obtained.

10. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

Images