A training method for classification models based on adaptive attention

Abstract

The present invention provides a training method for a classification model based on adaptive attention, comprising the steps of: obtaining original image set data containing skin diseases; extracting original feature vector data and original output vector data from the original image set data, and obtaining classification loss set data; constructing an image according to the original feature vector data, obtaining reconstructed image set data, and extracting reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data; obtaining consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data, and the original output vector data; training the training parameters of the classification model according to the classification loss set data and the consistency loss set data, and obtaining a trained classification model. Through the training method, device, equipment, and medium of a classification model based on adaptive attention disclosed by the present invention, the present invention can significantly improve the generalization and robustness of the classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a training method, device, equipment and medium for a classification model based on adaptive attention. Background Art

[0002] For image classification problems, using convolutional neural networks to train a classifier is already a robust and excellent solution. Under the conditions of sufficient data and annotation, a baseline model with good accuracy can usually be obtained through transfer learning. Existing high-precision classification models are all based on the optimal models of the ImageNet dataset. When there is a significant difference in the data distribution of skin disease data and ImageNet, simply selecting a classification model for transfer learning cannot directly obtain satisfactory accuracy.

[0003] The differences between skin disease data and natural image data from ImageNet are: different granularity, different object scale diversity, and the presence of dirty data. Fine granularity, object scale diversity, and dirty data are the main characteristics of the data set in the skin disease scenario, and the existing classification model cannot identify the special data in the skin disease scenario. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a training method, device, equipment and medium for a classification model based on adaptive attention, which can significantly improve the generalization and robustness of the classification model.

[0005] To achieve the above objectives and other related objectives, the present invention provides a training method for a classification model based on adaptive attention, which is applied to the classification model and includes:

[0006] Acquire original image set data containing skin diseases;

[0007] Extracting original feature vector data and original output vector data from the original image set data, and obtaining classification loss set data;

[0008] Constructing an image according to the original feature vector data to obtain reconstructed image set data, and extracting reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data;

[0009] Obtaining consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data, and the original output vector data;

[0010] The training parameters of the classification model are trained according to the classification loss set data and the consistency loss set data to obtain a trained classification model.

[0011] In one embodiment of the present invention, the step of extracting original feature vector data and original output vector data from the original image set data and obtaining classification loss set data includes:

[0012] Inputting the original image set data into the convolutional neural network in the classification model to extract original feature vector data of different original images in the original image set data;

[0013] Inputting the original feature vector data into a fully connected network in the classification model to obtain the original output vector data;

[0014] According to the original image set data, to obtain the classification loss of the corresponding original image;

[0015] A plurality of the classification losses are combined to obtain classification loss set data; wherein the classification loss l(f,y) of the original image is expressed as:

[0016] Among them, x represents the original image,

[0017] y represents the true label,

[0018] n represents the total number of original images,

[0019] m represents the number of categories,

[0020] f represents the network parameters.

[0021] In one embodiment of the present invention, the steps of constructing an image according to the original feature vector data, obtaining reconstructed image set data, and extracting reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data include:

[0022] Inputting the original feature vector data into a decoder network in the classification model to reconstruct the original image to obtain a reconstructed image;

[0023] Aggregating the plurality of reconstructed images to obtain reconstructed image set data;

[0024] The reconstructed image set data is input into the convolutional neural network in the classification model to extract the reconstructed feature vector data and the reconstructed output vector data of different reconstructed images in the reconstructed image set data.

[0025] In one embodiment of the present invention, the step of obtaining the consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data comprises:

[0026] Obtaining consistency loss according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data, and the original output vector data;

[0027] Aggregating a plurality of the consistency losses to obtain consistency loss set data;

[0028] Among them, the consistency loss D kl (p / q), expressed as:

[0029] Among them, p represents the reconstructed feature vector data and the reconstructed output vector data,

[0030] q represents the original feature vector data and the original output vector data,

[0031] p i / q i It represents the ratio of the reconstructed feature vector data to the original feature vector data, and the ratio of the reconstructed output vector data to the original output vector data.

[0032] The present invention also provides a training device for a classification model based on adaptive attention, which is applied to the classification model and includes:

[0033] An image acquisition module, used for acquiring original image set data containing skin diseases;

[0034] A classification loss module is used to extract original feature vector data and original output vector data from the original image set data, and obtain classification loss set data;

[0035] An image reconstruction module is used to construct an image based on the original feature vector data, obtain reconstructed image set data, and extract reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data;

[0036] A consistency loss module, used to obtain consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data; and

[0037] The training module is used to train the training parameters of the classification model according to the classification loss set data and the consistency loss set data to obtain a trained classification model.

[0038] In one embodiment of the present invention, the classification loss module includes:

[0039] A feature vector data extraction module, used for inputting the original image set data into the convolutional neural network in the classification model to extract original feature vector data of different original images in the original image set data;

[0040] an output vector data extraction module, used for inputting the original feature vector data into a fully connected network in the classification model to obtain the original output vector data; and

[0041] The classification loss calculation module is used to obtain the classification loss of the corresponding original image according to the original image set data.

[0042] In one embodiment of the present invention, the training parameters of the classification model include: training parameters of a convolutional neural network, training parameters of a fully connected network, and training parameters of a decoder network.

[0043] In one embodiment of the present invention, the classification model is an efficientNet-v2s model, and the efficientNet-v2s model is trained based on the classification loss set data and the consistency loss set data as data sets.

[0044] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the training method of the adaptive attention-based classification model when executing the computer program.

[0045] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the adaptive attention-based classification model are implemented.

[0046] As described above, the present invention provides a training method, device, equipment and medium for a classification model based on adaptive attention, which can make full use of all images and corresponding annotation information, and can significantly improve the generalization and robustness of the classification model. The classification loss and consistency loss weaken the overall loss of outliers, so that the classification model converges more smoothly during the training process. The present invention adds an image reconstruction process on the basis of the classification model, which can promote the convolutional neural network to learn fine-grained features and retain important underlying information during the classification model training stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0048] Figure 1 Shown is a flowchart of a training method for a classification model based on adaptive attention of the present invention.

[0049] Figure 2 Shown is a sub-step flowchart of step S2 in a training method for a classification model based on adaptive attention of the present invention.

[0050] Figure 3 Shown is a sub-step flowchart of step S3 in a training method for a classification model based on adaptive attention of the present invention.

[0051] Figure 4 Shown is a sub-step flowchart of step S4 in a training method for a classification model based on adaptive attention of the present invention.

[0052] Figure 5 Shown is a schematic diagram of a training device for an adaptive attention-based classification model of the present invention.

[0053] Figure 6 Shown is a schematic diagram of a classification loss module in a training device for an adaptive attention-based classification model of the present invention.

[0054] Figure 7 Shown is a schematic diagram of an image reconstruction module in a training device for a classification model based on adaptive attention according to the present invention.

[0055] Figure 8 Shown is a schematic diagram of a consistency loss module in a training device for a classification model based on adaptive attention according to the present invention.

[0056] Fig. 9 Shown is a schematic diagram of a processor and a memory in an electronic device of the present invention.

[0057] Fig.10 A schematic diagram of a storage medium in a computer-readable storage medium of the present invention is shown.

[0058] Component number description:

[0059] 10. Image acquisition module; 20. Classification loss module; 21. Feature vector data extraction module; 22. Output vector data extraction module; 23. Classification loss calculation module; 30. Image reconstruction module; 31. Reconstruction module; 32. Reconstruction extraction module; 40. Consistency loss module; 41. Consistency loss calculation module; 42. Consistency loss set data collection module; 50. Training module; 60. Processor; 70. Memory; 80. Computer program; 90. Storage medium. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1-10 It should be noted that the illustrations provided in this embodiment are only used to illustrate the basic concept of the present invention in a schematic manner, and the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0062] See also Figure 1 As shown, the present invention provides a training method for a classification model based on adaptive attention, which can be applied to a classification model. The specific steps of the training method for a classification model based on adaptive attention may include:

[0063] Step S1, obtaining original image set data containing skin diseases;

[0064] Step S2, extracting original feature vector data and original output vector data from the original image set data, and obtaining classification loss set data;

[0065] Step S3: construct an image according to the original feature vector data to obtain reconstructed image set data, and extract reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data.

[0066] Step S4: Obtain consistency loss set data based on the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data.

[0067] Step S5: train the training parameters of the classification model according to the classification loss set data and the consistency loss set data to obtain a trained classification model.

[0068] In one embodiment of the present invention, when step S1 is performed, the original image set data containing skin diseases is obtained. The original image set data may come from a set of images collected by a cooperative unit (such as a hospital, etc.) during a consultation process. The original image set data may also come from images obtained by a patient's selfie, and thus a set obtained after the patient uploads the image. The original image set data may also actively search for various public images through the Internet to form a set of images. The original image set data may contain multiple images, and the images may contain different types of skin diseases, so that each image has corresponding skin disease data. Skin disease data has the characteristics of fine granularity, target scale diversity, and dirty data.

[0069] In one embodiment of the present invention, specifically, fine-grainedness can be a more detailed sub-classification of image data in images of the same basic category. For example, image network data focuses on the division of large categories in nature, such as people, cars, ships, fish, dogs, etc. Skin disease data focuses on various types of skin lesions on human skin, which is a more detailed sub-classification in images of the same basic category (i.e., images containing skin). The pre-weighting of image network cannot extract such fine-grained features well.

[0070] In one embodiment of the present invention, specifically, the diversity of target scales can be the distribution of image data. For example, the foreground targets in most images of the image network data are relatively large, and the skin disease data comes from the self-portraits of patients with handheld devices. The scale of the foreground target is affected by the shooting angle and the shooting environment. At the same time, the skin disease data is also affected by the manifestation of the skin disease lesions themselves (there are ringworm diseases that spread over a large area, and there are also very small and often single pigmented nevi), and the scale distribution range of the lesion foreground is very wide and uneven. The existing classification model can easily lose the features of some underlying extremely small targets in the process of downsampling layer by layer, expanding the receptive field, and integrating global information.

[0071] In one embodiment of the present invention, specifically, dirty data can be data with low relevance. For example, from the perspective of data labels, the annotation quality of image network data is very high, and data with different data labels are highly independent and not easily mixed. Dermatology data can come from massive data collected by cooperative units (such as hospitals, etc.) during the consultation process, so there is dirty data (relevant / irrelevant, skin / non-skin, diseased / non-disease images may all be considered as skin disease data). Therefore, using a single cross entropy loss function to train a classifier can easily make dirty data a difficult sample, so that dirty data will cause the classification model to overfit to a certain extent, affecting the classification accuracy of the classification model.

[0072] See also Figure 2As shown, in one embodiment of the present invention, when performing step S2, that is, extracting the original feature vector data and the original output vector data from the original image set data, and obtaining the classification loss set data, wherein the classification model is an efficientNet-v2s classification model, and the efficientNet-v2s model is trained based on the classification loss set data and the consistency loss set data as a data set. The step of extracting the original feature vector data and the original output vector data from the original image set data and obtaining the classification loss set data may include:

[0073] Step S21, inputting the original image set data into the convolutional neural network in the classification model to extract original feature vector data of different original images in the original image set data, wherein the classification model is an efficientNet-v2s classification model;

[0074] Step S22, inputting the original feature vector data into the fully connected network in the classification model to obtain the original output vector data;

[0075] Step S23: According to the original image set data, the classification loss l(f,y) of the corresponding original image is obtained, which is expressed as:

[0076] Among them, x represents the original image,

[0077] y represents the true label,

[0078] n represents the total number of original images,

[0079] m represents the number of categories,

[0080] f represents the network parameters;

[0081] Step S24, aggregating the multiple classification losses to obtain classification loss set data.

[0082] In one embodiment of the present invention, when performing step S21, the original image set data is input into the convolutional neural network in the classification model to extract the original feature vector data of different original images in the original image set data. The convolutional neural network scans and reads each original image in the original image set data in a certain order, and extracts the original feature vector data in each original image respectively. The classification model can be a model that predicts the category of new data by learning and training existing data. Specifically, the classification model can be an efficientNet-v2s classification model.

[0083] In one embodiment of the present invention, specifically, the convolutional neural network is a type of feedforward neural network that includes convolution calculations and has a deep structure. The convolutional neural network has representational learning capabilities and can perform translation-invariant classification of input information according to its hierarchical structure. The convolutional neural network may include an input layer, a hidden layer, and an output layer, and the input layer, the hidden layer, and the output layer are connected in sequence, and the hidden layer may include a convolution layer, a pooling layer, and a fully connected layer. The convolutional neural network is constructed by imitating the visual perception mechanism of organisms, and can perform supervised learning and unsupervised learning. The convolution kernel parameter sharing in the hidden layer and the sparsity of the inter-layer connection enable the convolutional neural network to grid features with a smaller amount of computation.

[0084] In one embodiment of the present invention, when performing step S22, the original feature vector data is input into the fully connected network in the classification model to obtain the original output vector data. The fully connected network will calculate the original feature vector data in each image in a certain order and obtain the corresponding original output vector data. The fully connected network can be a single switch that connects all inputs and outputs, and has the characteristics of high throughput, high reliability and low latency. Fully connected networks are actually not scalable. Independent switches are usually fully connected internally, providing basic building blocks for larger networks.

[0085] In one embodiment of the present invention, when performing step S23 and step S24, the classification loss of the corresponding original image is obtained according to the original image set data. Specifically, the classification loss of the original image can be calculated according to the multiple original images, true labels, the total number of original images, the number of categories and network parameters in the original image set data. Among them, the original image can be each image in the original image set data. The true label can be a label in the original image. For example, when the original image set data is an image set data of skin diseases, the true label can be a different skin disease, for example, it can be eczema, urticaria, onychomycosis, onychomycosis, tinea corporis, psoriasis and other skin diseases. The total number of original images can be the total number of original images contained in the original image set data. The number of categories can be the total number of types of skin diseases in the original image. The network parameters can be the network parameters of the convolutional neural network. Thus, the classification loss of the corresponding original image can be obtained by taking the original image, the true label, the total number of original images, the number of categories and the network parameters as input, and the classification loss of each original image is aggregated to obtain the classification loss set data of the original image.

[0086] See also Figure 3As shown, in one embodiment of the present invention, when performing step S3, that is, constructing an image based on the original feature vector data to extract the reconstructed feature vector data and the reconstructed output vector data. The step of constructing an image based on the original feature vector data to extract the reconstructed feature vector data and the reconstructed output vector data may include:

[0087] Step S31, inputting the original feature vector data into the decoder network in the classification model to reconstruct the original image to obtain a reconstructed image;

[0088] Step S32, aggregating the plurality of reconstructed images to obtain reconstructed image set data;

[0089] Step S33: input the reconstructed image set data into the convolutional neural network in the classification model to extract the reconstructed feature vector data and the reconstructed output vector data of different reconstructed images in the reconstructed image set data.

[0090] In one embodiment of the present invention, when performing step S31 and step S32, the original feature vector data is input into the decoder network in the classification model to reconstruct the original image and obtain a reconstructed image. Specifically, the decoder network can be used to reconstruct the image, that is, the original feature vector data can be decoded and compressed, uploaded, transmitted over the network, stored and forwarded by the encoder, and then restored by the decoder to obtain a reconstructed image. The reconstructed image corresponds to the original image, so that when the obtained reconstructed images are assembled, reconstructed image set data can be obtained, and the reconstructed image set data can correspond to the original image set data.

[0091] In one embodiment of the present invention, when performing step S33, the reconstructed image set data is input into the convolutional neural network in the classification model to extract the reconstructed feature vector data and the reconstructed output vector data of different reconstructed images in the reconstructed image set data. Specifically, the convolutional neural network used in this step can be the same as the convolutional neural network used in step S21, and can be the same convolutional neural network. The convolutional neural network will scan and read each reconstructed image in the reconstructed image set data in a certain order, and extract the reconstructed feature vector data and the reconstructed output vector data in each reconstructed image.

[0092] See also Figure 4As shown, in one embodiment of the present invention, when performing step S4, that is, according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data, the consistency loss set data is obtained. The step of obtaining the consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data may include:

[0093] Step S41: Obtain consistency loss D according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data. kl (p / q), expressed as:

[0094] Among them, p represents the reconstructed feature vector data and the reconstructed output vector data,

[0095] q represents the original feature vector data and the original output vector data,

[0096] p i / q i Represents the ratio of the reconstructed feature vector data to the original feature vector data, and the ratio of the reconstructed output vector data to the original output vector data;

[0097] Step S42: Aggregate the plurality of consistency losses to obtain consistency loss set data.

[0098] In one embodiment of the present invention, when performing step S41 and step S42, the consistency loss is obtained according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data. Specifically, the consistency loss of the original image can be calculated according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the output vector data. Thus, the consistency loss of the reconstructed feature vector data and the original feature vector data, and the consistency loss of the reconstructed output vector data and the original output vector data of a single original image can be obtained. At this time, the consistency losses of the original images are aggregated to obtain consistency loss set data.

[0099] In one embodiment of the present invention, when performing step S5, the training parameters of the classification model are trained according to the classification loss set data and the consistency loss set data to obtain a trained classification model, wherein the training parameters include the training parameters of the convolutional neural network, the training parameters of the fully connected network, and the training parameters of the decoder network. Specifically, the classification loss set data and the consistency loss set data can be input into the classification model, and the training parameters of the convolutional neural network, the training parameters of the fully connected network, and the training parameters of the decoder network in the classification model are iteratively trained, and the training parameters of the convolutional neural network, the training parameters of the fully connected network, and the training parameters of the decoder network are optimized based on the results of the iterative training, thereby obtaining a trained classification model.

[0100] In one embodiment of the present invention, when an image is reconstructed based on the original feature vector data, most conventional image reconstruction tasks use the consistency loss function of the original image and the reconstructed image as direct supervision information to drive the classification model to learn a reconstructed image that is as similar to the original image as possible. However, in the present invention, a certain amount of information loss can be allowed in the reconstructed image at the pixel level, as long as the reconstructed feature vector data of the reconstructed image is as close as possible to the original feature vector data of the original image. This ensures that sufficient information for performing classification tasks is retained in the reconstructed image, that is, sufficient information for performing classification tasks is retained in the feature vector data from the original image. Therefore, it can effectively promote the classification model to mine fine-grained features in the original image and retain the most valuable feature information as much as possible.

[0101] In one embodiment of the present invention, when reconstructing an image, the original feature vector data is first gradually restored, thereby effectively alleviating the problem of disappearance of small-scale targets. Compared with the attention module deployed in a single network layer in a separation model, the entire network feature layer can be adaptively screened, thereby directly modeling the features of the front and back layers. Secondly, additional supervisory information is also provided, such as consistency loss, which can drive the reconstructed image learned by the classification model to be as similar as possible to the original image. When there is a certain amount of dirty data in the original image set data, this part of the data can easily become an outlier, resulting in a large loss and gradient, thereby misleading the convergence direction of the classification model. The classification loss set data and the consistency loss set data assist in measuring the consistency of the original features and the reconstructed features without considering the category, thereby weakening the loss of outliers and making the convergence of the classification model smoother.

[0102] See also Figure 5As shown, the present invention also provides a training device for a classification model based on adaptive attention, which can be applied to the classification model, and the training device for the classification model based on adaptive attention may include: an image acquisition module 10, a classification loss module 20, an image reconstruction module 30, a consistency loss module 40 and a training module 50. Among them, the module referred to in the present invention can be a series of computer program segments that can be executed by a processor 60 and can complete fixed functions, which are stored in a memory 70.

[0103] In one embodiment of the present invention, the image acquisition module 10 can be used to obtain raw image set data containing skin diseases. Specifically, the raw image set data can come from a collection of images collected by a cooperative unit (such as a hospital, etc.) during a consultation process. The raw image set data can also come from images obtained by the patient's self-portrait, so that the patient uploads the image and obtains the collection. The raw image set data can also actively search for various public images through the Internet to form a collection of images. The raw image set data can contain multiple images, and the images can contain different types of skin diseases, so that each image has corresponding skin disease data. Skin disease data has the characteristics of fine granularity, target scale diversity, dirty data, etc.

[0104] See also Figure 6 As shown, in one embodiment of the present invention, the classification loss module 20 can be used to extract the original feature vector data and the original output vector data in the original image set data, and obtain the classification loss set data. The classification loss module 20 may include: a feature vector data extraction module 21, an output vector data extraction module 22 and a classification loss calculation module 23. Among them, the feature vector data extraction module 21 can be used to input the original image set data into the convolutional neural network in the classification model to extract the original feature vector data of different original images in the original image set data. Specifically, the convolutional neural network will scan and read each original image in the original image set data in a certain order, and extract the original feature vector data in each original image respectively. The classification model can be to predict the category of new data by learning and training existing data.

[0105] In one embodiment of the present invention, the output vector data extraction module 22 can be used to input the original feature vector data into the fully connected network in the classification model to obtain the original output vector data. Specifically, the fully connected network will calculate the original feature vector data in each image in a certain order and obtain the corresponding original output vector data. The fully connected network can be a single switch that connects all inputs and outputs, with the characteristics of high throughput, high reliability and low latency. Fully connected networks are actually not scalable. Independent switches are usually fully connected internally, providing a basic building block for larger networks.

[0106] In one embodiment of the present invention, the classification loss calculation module 23 can be used to obtain the classification loss of the corresponding original image according to the original image set data. Specifically, the classification loss of the original image can be calculated according to the multiple original images, true labels, the total number of original images, the number of categories and network parameters in the original image set data. Among them, the original image can be each image in the original image set data. The true label can be a label in the original image. For example, when the original image set data is an image set data of skin diseases, the true label can be a different skin disease, for example, it can be eczema, urticaria, onychomycosis, onychomycosis, tinea corporis, psoriasis and other skin diseases. The total number of original images can be the total number of original images contained in the original image set data. The number of categories can be the total number of types of skin diseases in the original image. The network parameters can be the network parameters of the convolutional neural network. Therefore, the classification loss of the corresponding original image can be obtained by taking the original image, the true label, the total number of original images, the number of categories and the network parameters as input, and the classification loss of each original image is aggregated to obtain the classification loss set data of the original image.

[0107] See also Figure 7 As shown, in one embodiment of the present invention, the image reconstruction module 30 can be used to construct an image based on the original feature vector data to extract the reconstructed feature vector data and the reconstructed output vector data. The image reconstruction module 30 may include: a reconstruction module 31 and a reconstruction extraction module 32. Among them, the reconstruction module 31 can be used to input the original feature vector data into the decoder network in the classification model to reconstruct the original image and obtain a reconstructed image. Specifically, the decoder network can be used to reconstruct the image, that is, the original feature vector data can be decoded and compressed, uploaded, transmitted over the network, stored and forwarded by the encoder, and then restored by the decoder to obtain a reconstructed image. Among them, the reconstructed image corresponds to the original image, so that when the obtained reconstructed images are assembled, reconstructed image set data can be obtained, and the reconstructed image set data can correspond to the original image set data.

[0108] In one embodiment of the present invention, the reconstruction extraction module 32 may be used to input the reconstructed image set data into the convolutional neural network in the classification model to extract the reconstruction feature vector data and the reconstruction output vector data of different reconstructed images in the reconstructed image set data. Specifically, the convolutional neural network will scan and read each reconstructed image in the reconstructed image set data in a certain order, and respectively extract the reconstruction feature vector data and the reconstruction output vector data in each reconstructed image.

[0109] See also Figure 8 As shown, in one embodiment of the present invention, the consistency loss module 40 can be used to obtain consistency loss set data according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data. The consistency loss module 40 may include: a consistency loss calculation module 41 and a consistency loss set data combination module 42.

[0110] In one embodiment of the present invention, the consistency loss calculation module 41 can be used to obtain the consistency loss based on the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data. Specifically, the consistency loss of the original image can be calculated based on the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the output vector data. Thereby, the consistency loss between the reconstructed feature vector data and the original feature vector data, and the consistency loss between the reconstructed output vector data and the original output vector data of a single original image can be obtained. The consistency loss set data combination module 42 can be used to combine multiple consistency losses to obtain consistency loss set data.

[0111] In one embodiment of the present invention, the training module 50 can be used to train the training parameters of the classification model according to the classification loss set data and the consistency loss set data to obtain a trained classification model. Specifically, the classification loss set data and the consistency loss set data can be input into the classification model, and the training parameters of the convolutional neural network, the training parameters of the fully connected network, and the training parameters of the decoder network in the classification model are iteratively trained, and the training parameters of the convolutional neural network, the training parameters of the fully connected network, and the training parameters of the decoder network are optimized based on the results of the iterative training, so as to obtain a trained classification model.

[0112] It should be noted that the training device based on the adaptive attention classification model of the present embodiment is a device corresponding to the training method based on the adaptive attention classification model mentioned above, and the functional modules in the training device based on the adaptive attention classification model respectively correspond to the corresponding steps in the training method based on the adaptive attention classification model. The training device based on the adaptive attention classification model of the present embodiment can be implemented in coordination with the training method based on the adaptive attention classification model. Accordingly, the relevant technical details mentioned in the training device based on the adaptive attention classification model of the present embodiment can also be applied to the training method based on the adaptive attention classification model mentioned above.

[0113] It should be noted that the above-mentioned functional modules can be fully or partially integrated into one physical entity during actual implementation, or they can be physically separated, and these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, some or all steps of the above method, or the above functional modules can be completed by hardware integrated logic circuits in the processor 60 element or software instructions.

[0114] The present invention also provides an electronic device. The above-mentioned training method based on the adaptive attention classification model and / or the training device based on the adaptive attention classification model can be applied to the electronic device. The electronic device can be a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware can include but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0115] In one embodiment of the present invention, the electronic device may be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), an intelligent wearable device, etc. The electronic device may also include a network device and / or a user device. Among them, the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing (Cloud Computing). The network in which the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0116] See also Fig. 9 As shown, in one embodiment of the present invention, the electronic device may include a memory 70, a processor 60 and a bus, and may also include a computer program 80 stored in the memory 70 and executable on the processor 60, such as a text recognition program based on direction detection. Among them, the memory 70 may include at least one type of readable storage medium 90, and the readable storage medium 90 may include a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 70 may be an internal storage unit of the electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory 70 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 70 may also include both an internal storage unit of the electronic device and an external storage device. The memory 70 may be used not only to store application software and various types of data installed in the electronic device, but also to temporarily store data that has been output or is to be output.

[0117] In one embodiment of the present invention, the processor 60 may be composed of an integrated circuit in some embodiments, for example, it may be composed of a single packaged integrated circuit, or it may be composed of multiple integrated circuits with the same function or different functions. The processor 60 may include one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 60 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes programs or modules stored in the memory 70 (for example, executing a physical examination report verification program, etc.), and calls the data stored in the memory 70 to execute various functions of the electronic device and process data. The processor 60 executes the operating device of the electronic device and various installed applications. The processor 60 executes the application to implement the steps in the above-mentioned training method embodiments of the classification model based on adaptive attention.

[0118] See also Fig.10 As shown, in one embodiment of the present invention, the computer program 80 can be divided into one or more modules, one or more modules are stored in the memory 70, and are executed by the processor 60 to complete the present invention. One or more modules can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 80 in the electronic device. For example, the computer program 80 can be divided into an image acquisition module 10, a classification loss module 20, an image reconstruction module 30, a consistency loss module 40, and a training module 50. The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium 90. The above-mentioned software function module is stored in a storage medium 90, including a number of instructions to enable a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor 60 (processor) to perform part of the functions of the training method of the classification model based on adaptive attention in various embodiments of the present invention.

[0119] In one embodiment of the present invention, the bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between a memory 70 and at least one processor 60, etc.

[0120] In summary, the training method, device, equipment and medium of a classification model based on adaptive attention provided by the present invention can make full use of all images and corresponding annotation information, significantly improve the generalization and robustness of the model, and the classification loss and consistency loss weaken the overall loss of outliers, so that the classification model converges more smoothly during the training process. The present invention adds an image reconstruction process on the basis of the classification model, which can promote the convolutional neural network to learn fine-grained features and retain important underlying information during the classification model training stage.

[0121] In the description of this specification, the description with reference to the terms "this embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0122] The embodiments of the present invention disclosed above are only used to help illustrate the present invention. The embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A training method for a classification model based on adaptive attention, characterized in that: It is applied to the classification model and includes the following steps: Acquire original image set data containing skin diseases; the original image set data includes a plurality of images, the plurality of images contain different skin disease data, and the skin disease data is concentrated on various skin lesion representations on human skin; Extracting original feature vector data and original output vector data from the original image set data, and obtaining classification loss set data; Constructing an image according to the original feature vector data to obtain reconstructed image set data, and extracting reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data; According to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data, a consistency loss is obtained; the consistency loss , expressed as: ;in, Represents reconstructed feature vector data and reconstructed output vector data; Represents original feature vector data and original output vector data; Represents the ratio of the reconstructed feature vector data to the original feature vector data, and the ratio of the reconstructed output vector data to the original output vector data; Aggregating a plurality of the consistency losses to obtain consistency loss set data; Training the training parameters of the classification model according to the classification loss set data and the consistency loss set data to obtain the trained classification model; The step of extracting original feature vector data and original output vector data from the original image set data and obtaining classification loss set data includes: Inputting the original image set data into the convolutional neural network in the classification model to extract the original feature vector data of different original images in the original image set data; Inputting the original feature vector data into a fully connected network in the classification model to obtain the original output vector data; According to the original image set data, to obtain the classification loss of the corresponding original image; A plurality of the classification losses are combined to obtain classification loss set data.

2. The training method of the classification model based on adaptive attention according to claim 1, characterized in that: The classification loss of the original image , expressed as: , in, represents the original image, represents the true label, represents the total number of original images, represents the number of categories, Represents network parameters.

3. The training method of the classification model based on adaptive attention according to claim 1, characterized in that: The steps of constructing an image according to the original feature vector data, obtaining reconstructed image set data, and extracting reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data include: Inputting the original feature vector data into a decoder network in the classification model to reconstruct the original image to obtain a reconstructed image; Aggregating the plurality of reconstructed images to obtain reconstructed image set data; The reconstructed image set data is input into the convolutional neural network in the classification model to extract the reconstructed feature vector data and the reconstructed output vector data of different reconstructed images in the reconstructed image set data.

4. The training method of the classification model based on adaptive attention according to claim 1, characterized in that: The training parameters of the classification model include: training parameters of a convolutional neural network, training parameters of a fully connected network, and training parameters of a decoder network.

5. The training method of the classification model based on adaptive attention according to claim 1, characterized in that: The classification model is an efficientNet-v2s model, and the efficientNet-v2s model is trained using the classification loss set data and the consistency loss set data as a data set.

6. A training device for a classification model based on adaptive attention, characterized in that: It is used in classification models, including: An image acquisition module is used to acquire original image set data containing skin diseases; the original image set data includes a plurality of images, the plurality of images contain different skin disease data, and the skin disease data is concentrated on various skin lesion representations on human skin; A classification loss module is used to extract original feature vector data and original output vector data from the original image set data, and obtain classification loss set data; An image reconstruction module is used to construct an image based on the original feature vector data, obtain reconstructed image set data, and extract reconstructed feature vector data and reconstructed output vector data of the reconstructed image set data; A consistency loss module is used to obtain a consistency loss according to the reconstructed feature vector data, the reconstructed output vector data, the original feature vector data and the original output vector data; a plurality of the consistency losses are aggregated to obtain consistency loss set data; the consistency loss , expressed as: ;in, Represents reconstructed feature vector data and reconstructed output vector data; Represents original feature vector data and original output vector data; represents the ratio of the reconstructed feature vector data to the original feature vector data, and the ratio of the reconstructed output vector data to the original output vector data; and A training module is used to train the training parameters of the classification model according to the classification loss set data and the consistency loss set data to obtain a trained classification model; Wherein, the classification loss module includes: A feature vector data extraction module, used for inputting the original image set data into the convolutional neural network in the classification model to extract original feature vector data of different original images in the original image set data; an output vector data extraction module, used for inputting the original feature vector data into a fully connected network in the classification model to obtain the original output vector data; and The classification loss calculation module is used to obtain the classification loss of the corresponding original image according to the original image set data.

7. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images