Classification model training method, device, equipment, storage medium and program product

Abstract

The present application relates to a training method, apparatus, device, storage medium and program product for a classification model. The method comprises: obtaining unlabeled first image data and labeled second image data, inputting the first image data and the second image data into a preset classification model for training, outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data, determining a first loss based on the first classification result and the second classification result output by the first classification sub-model, determining a second loss based on the third image data and the first image data, and training the preset classification model based on the second loss and the first loss to obtain a target classification model; the third image data is an image after mask processing and completion processing. The method provided by the above method for training a classification model based on labeled images and unlabeled images solves the problem of high difficulty in training the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, device, storage medium, and program product for a classification model. Background Art

[0002] The fundus (including the retina and optic nerve) is the only tissue in the body that can directly observe blood vessels and nerves non-invasively. Fundus images can not only diagnose eye diseases such as diabetic retinopathy and retinal detachment, but also observe systemic diseases such as hypertension, lupus erythematosus, and Alzheimer's disease.

[0003] With the development of artificial intelligence, training computer vision models using ultra-wide-angle fundus images to identify various types of diseases has become a hot research topic. However, in the medical field, the labels of collected ultra-wide-angle fundus images are often quite sparse, making it difficult to train computer vision models that rely on labeled data.

[0004] Therefore, how to reduce the training difficulty of computer vision models in eye disease recognition applications has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] Based on this, it is necessary to provide a training method, device, equipment, storage medium and program product for a classification model that can reduce the training difficulty of a computer vision model in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for training a classification model, comprising:

[0007] Acquire unlabeled first image data and labeled second image data;

[0008] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0009] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0010] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0011] In one embodiment, the preset classification model includes a first initial classification sub-model and a second initial classification sub-model. The first image data and the second image data are input into the preset classification model for training, and the first classification result of the first image data, the first classification sub-model corresponding to the second image data, and the third image data are output, including:

[0012] Inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0013] Inputting the first image data into the first classification sub-model for classification to obtain a second classification result;

[0014] The first image data is input into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0015] In one embodiment, the second initial classification sub-model includes an encoder and a decoder, and the first image data is input into the second initial classification sub-model for processing to obtain third image data, including:

[0016] performing mask processing on the first image data to obtain a masked image;

[0017] Inputting the masked image into the encoder for encoding to obtain an encoded image;

[0018] The encoded image is input into the decoder for completion processing to obtain third image data.

[0019] In one embodiment, the second initial classification sub-model further includes a perceptron, and the first image data is input into the second initial classification sub-model for processing to obtain a first classification result, including:

[0020] The encoded image is input into the perceptron for classification to obtain the first classification result.

[0021] In one embodiment, performing mask processing on the first image data to obtain a masked image includes:

[0022] Performing block processing on the first image data to obtain a plurality of image blocks;

[0023] Masking is performed on some image blocks among the multiple image blocks to obtain a masked image.

[0024] In one embodiment, determining the second loss based on the third image data and the first image data includes:

[0025] determining a corresponding third image pixel value according to the third image data;

[0026] Determining a corresponding first image pixel value according to the first image data;

[0027] Determine a second loss based on the third image pixel value and the first image pixel value. Step A;

[0028] In a second aspect, the present application also provides a training device for a classification model, comprising:

[0029] an acquisition module, configured to acquire unlabeled first image data and labeled second image data;

[0030] a training module, configured to input the first image data and the second image data into a preset classification model for training, and output a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0031] a determination module, configured to determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and to determine a second loss based on the third image data and the first image data;

[0032] The training module is used to train the preset classification model according to the second loss and the first loss to obtain a target classification model.

[0033] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Acquire unlabeled first image data and labeled second image data;

[0035] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0036] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0037] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0039] Acquire unlabeled first image data and labeled second image data;

[0040] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0041] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0042] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0043] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0044] Acquire unlabeled first image data and labeled second image data;

[0045] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0046] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0047] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0048] The training method, apparatus, device, storage medium, and program product of the above-mentioned classification model first obtain unlabeled first image data and labeled second image data, then input the first image data and the second image data into a preset classification model for training, outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data. Then, based on the first classification result and the second classification result output by the first classification sub-model, a first loss is determined, and based on the third image data and the first image data, a second loss is determined, and based on the second loss and the first loss, the preset classification model is trained to obtain a target classification model; the third image data is an image after masking and completion processing. The above-mentioned method provides a method for training a classification model based on labeled and unlabeled images, which solves the problem of the difficulty of training computer vision models in current eye disease identification applications. In addition, the preset classification model is trained based on the first loss determined based on the classification result corresponding to the preset classification model and the second loss determined based on the image information corresponding to the preset classification model, thereby ensuring the accuracy of the classification model obtained after training. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 FIG. 1 is an application environment diagram of a classification model training method in one embodiment;

[0051] Figure 2 Schematic diagram of a process for training a classification model in one embodiment;

[0052] Figure 3 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0053] Figure 4 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0054] Figure 5 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0055] Figure 6 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0056] Figure 7 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0057] Figure 8 Schematic diagram of a flow chart of a training method for a classification model in another embodiment;

[0058] Figure 9 is a structural block diagram of a training device for a classification model in one embodiment;

[0059] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] The fundus (including the retina and optic nerve) is the only tissue in the body where blood vessels and nerves can be directly and noninvasively observed. Fundus images can be used to diagnose not only eye diseases such as diabetic retinopathy and retinal detachment, but also systemic diseases such as hypertension, lupus erythematosus, and Alzheimer's disease. Compared to standard fundus color photography, ultra-wide-angle fundus imaging samples the fundus from a wider angle. This not only encompasses the sampling range of traditional fundus color photography but also allows for a clearer observation of the fundus periphery, facilitating more accurate and comprehensive screening and diagnosis.

[0062] Artificial intelligence is now widely used in medical image interpretation. Artificial intelligence is an emerging technical discipline that studies and develops theories, methods, technologies, and applications for simulating, extending, and expanding human intelligence. Machine learning refers to the process of using algorithms to guide computers to derive appropriate models from known data and use these models to make judgments about new situations. Deep learning is a subfield of machine learning research, the concept of which originates from the study of artificial neural networks. The multilayer perceptron with multiple hidden layers is an example of a deep learning architecture. Its motivation is to build artificial neural networks that mimic the analytical learning of the human brain. They imitate the human brain's mechanisms to interpret data modalities, including images, sound, and text. By combining lower-level features to form more abstract higher-level features, they discover associations and distributions in the data.

[0063] With the development of artificial intelligence, training computer vision models using ultra-wide-angle fundus images to identify various types of diseases has become a hot research topic. However, in the medical field, the labels of collected ultra-wide-angle fundus images are often quite sparse, making it difficult to train computer vision models that rely on labeled data.

[0064] Therefore, how to reduce the difficulty of training computer vision models in eye disease recognition applications has become a technical problem that needs to be solved urgently. This application aims to solve this problem.

[0065] After introducing the background technology of the training method of the classification model provided by the embodiment of the present application, the following briefly describes the implementation environment involved in the training method of the classification model provided by the embodiment of the present application. The training method of the classification model provided by the embodiment of the present application can be applied to Figure 1 In the illustrated implementation environment, the implementation environment includes a server 104, which can be implemented as a standalone server 104 or a server cluster consisting of multiple servers 104. A data storage system 102 can store data that the server 104 needs to process. The data storage system 102 can be integrated with the server 104, or it can be placed on a cloud or other network server. The server 104 can obtain labeled fundus image data and unlabeled fundus image data, and train a classification model based on the labeled fundus image data and the unlabeled fundus image data to obtain a trained classification model. The trained classification model can determine the disease type corresponding to the fundus image.

[0066] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0067] After introducing the application scenarios of the training method of the classification model provided in the embodiments of the present application, the following focuses on the training method of the classification model described in the present application.

[0068] In one embodiment, Figure 2 As shown, a training method for a classification model is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0069] S201: Acquire first image data without labels and second image data with labels.

[0070] Among them, the first image data and the second image data are both fundus images, specifically ultra-wide-angle fundus images. The unlabeled first image data refers to the fundus image marked with the disease type, and the labeled second image data refers to the fundus image without the disease type.

[0071] In an embodiment of the present application, when it is necessary to train a classification model based on fundus images and to identify the disease type corresponding to the fundus images based on the trained classification model, it is necessary to first collect unlabeled first image data and labeled second image data within a historical time period; optionally, the unlabeled first image data can be directly obtained from the fundus image scanning device, and the labeled second image data can be obtained from the doctor's disease diagnosis data.

[0072] S202: Input the first image data and the second image data into a preset classification model for training, and output a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data.

[0073] The preset classification model refers to an initial classification model that has not been trained.

[0074] The third image data is an image after mask processing and completion processing.

[0075] In an embodiment of the present application, after the first image data and the second image data are acquired as described above, the first image data and the second image data can be input into a preset classification model, and the preset classification model can be trained to obtain a first classification result corresponding to the first image data, a first classification sub-model corresponding to the second image data, and third image data.

[0076] Optionally, the preset classification model may include a first initial classification sub-model and a second initial classification sub-model. The first initial classification sub-model can be trained based on the second image data to obtain a first classification sub-model corresponding to the second image data, and the first image data can be input into the second initial classification sub-model for initial classification to obtain a first classification result, and the first image data can be masked and completed in sequence to obtain third image data.

[0077] S203: Determine a first loss based on the first classification result and the second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data.

[0078] In an embodiment of the present application, the first image data can be input into the first classification sub-model for classification to obtain a second classification result, and then the first loss can be determined based on the difference between the first classification result and the second classification result. After the third image data is determined as above, the second loss can be determined based on the difference between the first image data and the third image data.

[0079] S204: Train the preset classification model based on the second loss and the first loss to obtain a target classification model.

[0080] In the embodiment of the present application, after the first loss and the second loss are obtained as described above, the target loss can be determined based on the second loss and the first loss, and the preset classification model can be trained based on the target loss to obtain the target classification model.

[0081] Optionally, the process of determining the target loss based on the first loss and the second loss is as follows:

[0082]

[0083] in, is the target loss, is the first loss, is the loss coefficient, For the second loss.

[0084] It should be noted that the Adam optimizer can be used to perform gradient descent optimization on the model parameters of the preset classification model. When the target loss value is less than the preset loss threshold, it means that the preset classification model has been trained. At this time, the training process can be stopped in time.

[0085] The training method of the classification model provided in the embodiment of the present application first obtains unlabeled first image data and labeled second image data, then inputs the first image data and the second image data into a preset classification model for training, outputs a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data, then, based on the first classification result and the second classification result output by the first classification sub-model, determines a first loss, and based on the third image data and the first image data, determines a second loss, and based on the second loss and the first loss, trains the preset classification model to obtain a target classification model; the third image data is an image after masking and completion processing. The above method provides a method for training a classification model based on labeled images and unlabeled images, which solves the problem of the difficulty of training computer vision models in current eye disease recognition applications; in addition, the preset classification model is trained based on the first loss determined based on the classification result corresponding to the preset classification model and the second loss determined based on the image information corresponding to the preset classification model, thereby ensuring the accuracy of the classification model obtained after training.

[0086] In one embodiment, Figure 2 Based on the embodiment shown, the above-mentioned preset classification model includes a first initial classification sub-model and a second initial classification sub-model, and the process of determining the first classification result, the first classification sub-model corresponding to the second image data, and the third image data can be described, such as Figure 3As shown, the above S202 "inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and the third image data" includes:

[0087] S301: Input the second image data into the first initial classification sub-model for training to obtain a first classification sub-model.

[0088] The first initial classification sub-model refers to an initial type of the first classification sub-model that can be trained based on the labeled second image data.

[0089] The first initial classification sub-model can be a ResNet-50 model pre-trained based on the ImageNet dataset. The first classification sub-model can classify fundus images. Optionally, the last layer of the neural network of the first classification sub-model can be replaced with an input dimension of 1024 and an output dimension of , so that each dimension of the output corresponds to the judgment probability of the corresponding disease.

[0090] In the embodiment of the present application, after the labeled second image data is obtained as described above, the second image data can be input into the first initial classification sub-model for training to obtain the first classification sub-model after training.

[0091] S302: Input the first image data into the first classification sub-model for classification to obtain a second classification result.

[0092] In the embodiment of the present application, after the trained first classification sub-model is obtained, the first image data can be input into the trained first classification sub-model for classification to obtain a second classification result.

[0093] Optionally, the first image data can be input into the first classification sub-model after training to obtain the output vector , the output vector The value of each dimension represents the probability of the corresponding disease predicted by the first classification sub-model based on the first image data, and sets the disease label with a probability greater than 50% as positive, otherwise it is set to negative.

[0094] S303: Input the first image data into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0095] The second initial classification sub-model may include a classification sub-unit and an image processing sub-unit.

[0096] In an embodiment of the present application, the first image data can be input into the classification subunit for classification processing to obtain a first classification result, and the first image data can be input into the image processing subunit for image processing to obtain third image data after image processing.

[0097] Optionally, the process of obtaining the first classification result and the process of obtaining the third image data may be described in detail respectively.

[0098] First, the process of obtaining the third image data is described, see Figure 4 The second initial classification sub-model includes an encoder and a decoder. The above S303 “inputting the first image data into the second initial classification sub-model for processing to obtain third image data” includes:

[0099] S401 : Perform mask processing on first image data to obtain a masked image.

[0100] In the embodiment of the present application, after the first image data is acquired as described above, mask processing may be performed on part of the first image data to obtain a masked image.

[0101] Optionally, the process of performing mask processing on the first image data may be described, see Figure 5 , that is, the above-mentioned S401 “performing mask processing on the first image data to obtain a masked image” includes:

[0102] S4011. Perform block processing on the first image data to obtain multiple image blocks.

[0103] In the embodiment of the present application, after the first image data is obtained, each image in the first image data may be divided into blocks to obtain a plurality of image blocks corresponding to each image. Optionally, the first image data may be input into the input processing layer to divide each image in the first image data into a plurality of image blocks. For example, according to the length of the image He Kuan , divide the image into image blocks.

[0104] Optionally, before the first image data is divided into blocks, the resolution of the first image data may be adjusted to Then, the adjusted first image data is subjected to data enhancement processing to obtain the first image data after data enhancement processing, and then the first image data after data enhancement processing is subjected to block processing. Optionally, the data enhancement processing method includes standardizing color distribution, randomly cropping to Resolution size, as well as randomly adding horizontal flip, vertical flip and random angle rotation, etc.

[0105] S4012: Perform mask processing on some image blocks among the multiple image blocks to obtain a masked image.

[0106] In the embodiment of the present application, after obtaining the multiple image blocks corresponding to an image, a portion of the multiple image blocks may be masked to obtain a masked image. For example, 75% of the multiple image blocks corresponding to an image may be masked, i.e., all pixel values of the masked image blocks are set to 0, thereby obtaining a masked image.

[0107] Thus, the masked image corresponding to the first image data is obtained.

[0108] S402: Input the masked image into an encoder for encoding to obtain an encoded image.

[0109] The encoder may be an encoder composed of a multi-layer Transformer network.

[0110] Where the coded image can be of length The vector contains high-level semantic information about the ultra-wide-angle fundus image.

[0111] In the embodiment of the present application, after the masked image is obtained, the masked image can be input into an encoder for encoding to obtain an encoded image.

[0112] Optionally, after obtaining the masked image above, you can first use a size of , the step length is The 2D convolutional model maps the masked image to a latent representation space ,in , obtain the vector of the representation space output, and then input the vector of the representation space output into the encoder composed of multiple layers of Transformer for encoding to obtain the encoded image.

[0113] S403: Input the encoded image into the decoder for completion processing to obtain third image data.

[0114] The decoder may be a decoder composed of multiple layers of Transformer.

[0115] In the embodiment of the present application, after the encoded image is obtained as described above, the encoded image can be input into a decoder for completion processing to obtain completed third image data.

[0116] Optionally, in order to apply to the mask image completion task, the above-obtained coded image needs to be input into the decoder for completion processing to obtain a third image data with the same resolution as the first image data. .

[0117] The above is the process of acquiring the third image data.

[0118] Second, the process of obtaining the first classification result is described, see Figure 6 The second initial classification sub-model further includes a perceptron. The above S303 “inputting the first image data into the second initial classification sub-model for processing to obtain a first classification result” includes:

[0119] S404: Input the encoded image into a perceptron for classification to obtain a first classification result.

[0120] In the embodiment of the present application, after the encoded image is obtained as described above, the encoded image can be input into the perception machine. The first classification result is obtained by memory classification.

[0121] The training method of the classification model provided in the embodiment of the present application fully utilizes massive unlabeled image data to train the classification model, so that the model after training can have better generalization and robustness in distinguishing various diseases; in addition, the semi-supervised learning method using the first classification sub-model for knowledge distillation can further enhance the efficiency of unlabeled data in the pre-training process, and can further enhance the performance of the base model.

[0122] In one embodiment, Figure 2-Figure 6 Based on any of the embodiments shown, a method for determining the second loss can be described as follows: Figure 7 As shown, the above S203 “determining the second loss according to the third image data and the first image data” includes:

[0123] S501: Determine corresponding third image pixel values according to third image data.

[0124] In the embodiment of the present application, after the third image data is acquired as described above, the pixel value corresponding to the third image data, that is, the third image pixel value, can be determined.

[0125] S502: Determine a corresponding first image pixel value according to the first image data.

[0126] In the embodiment of the present application, after the first image data is acquired, the pixel value corresponding to the first image data, that is, the first image pixel value, can be determined.

[0127] S503: Determine a second loss according to the third image pixel value and the first image pixel value.

[0128] In an embodiment of the present application, after the first image pixel value and the third image pixel value are obtained as described above, the mean square error between the first image pixel value and the third image pixel value can be determined, and the second loss can be determined based on the mean square error between the first image pixel value and the third image pixel value.

[0129] The method for obtaining the second loss provided in this application determines the second loss by the difference between the pixel values corresponding to the third image data and the first image data, providing a data basis for subsequently determining the target loss and training the target classification model based on the target loss.

[0130] In one embodiment, see Figure 8 , also provides a complete classification model training method, including:

[0131] S10, acquiring unlabeled first image data and labeled second image data;

[0132] S11, inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0133] S12, inputting the first image data into the first classification sub-model for classification to obtain a second classification result;

[0134] S13, performing block processing on the first image data to obtain multiple image blocks;

[0135] S14, performing mask processing on some image blocks among the multiple image blocks to obtain a masked image;

[0136] S15, inputting the masked image into an encoder for encoding to obtain an encoded image;

[0137] S16, inputting the encoded image into the decoder for completion processing to obtain third image data;

[0138] S17, inputting the encoded image into the perceptron for classification to obtain a first classification result;

[0139] S18. Determine a first loss based on the first classification result and the second classification result output by the first classification sub-model;

[0140] S19. Determine a corresponding third image pixel value according to the third image data;

[0141] S20. Determine a corresponding first image pixel value according to the first image data;

[0142] S21, determining a second loss according to the pixel value of the third image and the pixel value of the first image;

[0143] S22. Train the preset classification model based on the second loss and the first loss to obtain a target classification model.

[0144] The above method provides a method for training a classification model based on labeled images and unlabeled images, which solves the problem that training computer vision models is difficult in current eye disease recognition applications. In addition, the preset classification model is trained based on a first loss determined by the classification result corresponding to the preset classification model and a second loss determined based on the image information corresponding to the preset classification model, thereby ensuring the accuracy of the classification model obtained after training.

[0145] In one embodiment, to ensure privacy in real-world settings, especially in the medical field, user data often correlates with or involves user privacy. Directly centralizing a wide range of user data for model training often leads to the risk of privacy leakage. To address this issue, the field of federated learning has proposed a method that uses data from various parties separately for model training and then centralizes the trained models. This method formally addresses the privacy issues caused by data centralization.

[0146] Therefore, this application also provides a decentralized federated learning privacy protection mechanism based on the MQTT protocol. The MQTT protocol adopts a publish-subscribe mechanism. Participants need to agree on an MQTT message server that is reachable by all participants) and a root topic. Then, federated learning can be carried out according to the following steps, including:

[0147] 1) Sharing of model parameters among participants: Taking a single participant as an example, after each round of training, testing, and parameter synchronization, some model training parameters are selected and published to the sub-topic with the corresponding parameter name under the root topic of the MQTT message server with network service quality QoS = 0. After that, all participants who subscribe to the root topic will receive the updated parameter values and the corresponding parameter names.

[0148] 2) Receiving and synchronizing model parameters from other participants:

[0149] The federated learning method provided in this application does not require the training time of each participant to be completely synchronized. ), for example, during the model training process, it is possible to receive model parameters, sample quantity and model performance information from other participants. At this time, the participant will temporarily store this information without affecting the normal progress of its own training process. After the training and testing process of this round is completed, the participant will then use the number of its own test samples to And the performance of the test , and the relevant information received from other participants is calculated using the federated average algorithm to obtain a weighted average, see formula (2):

[0150]

[0151] in, Known as The set of participants in the parameter update of For participants The name is The model parameters, The name obtained after weighted average is The model parameters.

[0152] Optionally, after obtaining the weighted average, the participant's model training can be performed based on the weighted average.

[0153] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0154] Based on the same inventive concept, the present application also provides a classification model training device for implementing the classification model training method mentioned above. The solution to the problem provided by the device is similar to the solution described in the above method. Therefore, the specific limitations of the one or more classification model training device embodiments provided below can be found in the above-mentioned limitations on the classification model training method, and will not be repeated here.

[0155] In an exemplary embodiment, Figure 9 As shown, a training device for a classification model is provided, comprising: an acquisition module 10, an output module 11, a determination module 12 and a training module 13, wherein:

[0156] The acquisition module 10 is configured to acquire first image data without labels and second image data with labels.

[0157] The output module 11 is used to input the first image data and the second image data into a preset classification model for training, and output the first classification result of the first image data, the first classification sub-model corresponding to the second image data, and the third image data; the third image data is the image after mask processing and completion processing.

[0158] The determination module 12 is configured to determine a first loss based on the first classification result and the second classification result output by the first classification sub-model, and to determine a second loss based on the third image data and the first image data.

[0159] The training module 13 is used to train the preset classification model according to the second loss and the first loss to obtain a target classification model.

[0160] In an exemplary embodiment, the preset classification model includes a first initial classification sub-model and a second initial classification sub-model, and the output module 11 includes: a training unit, a classification unit and a processing unit, wherein:

[0161] a training unit, specifically configured to input the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0162] a classification unit, specifically configured to input the first image data into the first classification sub-model for classification to obtain a second classification result;

[0163] The processing unit is specifically configured to input the first image data into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0164] In an exemplary embodiment, the above-mentioned second initial classification sub-model includes an encoder and a decoder, and the above-mentioned processing unit is specifically used to perform mask processing on the first image data to obtain a masked image; input the masked image into the encoder for encoding to obtain an encoded image; input the encoded image into the decoder for completion processing to obtain third image data.

[0165] In an exemplary embodiment, the second initial classification sub-model further includes a perceptron, and the processing unit is further configured to input the encoded image into the perceptron for classification to obtain a first classification result.

[0166] In an exemplary embodiment, the processing unit is further configured to perform block processing on the first image data to obtain a plurality of image blocks; and perform mask processing on some of the plurality of image blocks to obtain a masked image.

[0167] In an exemplary embodiment, the determination module 12 includes: a first determination unit, a second determination unit, and a third determination unit, wherein:

[0168] A first determining unit is specifically configured to determine a corresponding third image pixel value according to the third image data;

[0169] A second determining unit is specifically configured to determine a corresponding first image pixel value according to the first image data;

[0170] The third determining unit is specifically configured to determine the second loss according to the third image pixel value and the first image pixel value.

[0171] Each module in the above-mentioned classification model training device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0172] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store first image data and second image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for a classification model is implemented.

[0173] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0174] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0175] Acquire unlabeled first image data and labeled second image data;

[0176] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0177] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0178] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0179] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0180] Inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0181] Inputting the first image data into the first classification sub-model for classification to obtain a second classification result;

[0182] The first image data is input into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0183] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0184] performing mask processing on the first image data to obtain a masked image;

[0185] Inputting the masked image into the encoder for encoding to obtain an encoded image;

[0186] The encoded image is input into the decoder for completion processing to obtain third image data.

[0187] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0188] The encoded image is input into the perceptron for classification to obtain the first classification result.

[0189] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0190] Performing block processing on the first image data to obtain a plurality of image blocks;

[0191] Masking is performed on some image blocks among the multiple image blocks to obtain a masked image.

[0192] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0193] determining a corresponding third image pixel value according to the third image data;

[0194] Determining a corresponding first image pixel value according to the first image data;

[0195] A second loss is determined based on the third image pixel value and the first image pixel value.

[0196] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0197] Acquire unlabeled first image data and labeled second image data;

[0198] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0199] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0200] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0201] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0202] Inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0203] Inputting the first image data into the first classification sub-model for classification to obtain a second classification result;

[0204] The first image data is input into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0205] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0206] performing mask processing on the first image data to obtain a masked image;

[0207] Inputting the masked image into the encoder for encoding to obtain an encoded image;

[0208] The encoded image is input into the decoder for completion processing to obtain third image data.

[0209] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0210] The encoded image is input into the perceptron for classification to obtain the first classification result.

[0211] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0212] Performing block processing on the first image data to obtain a plurality of image blocks;

[0213] Masking is performed on some image blocks among the multiple image blocks to obtain a masked image.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0215] determining a corresponding third image pixel value according to the third image data;

[0216] Determining a corresponding first image pixel value according to the first image data;

[0217] A second loss is determined based on the third image pixel value and the first image pixel value.

[0218] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0219] Acquire unlabeled first image data and labeled second image data;

[0220] Inputting the first image data and the second image data into a preset classification model for training, and outputting a first classification result of the first image data, a first classification sub-model corresponding to the second image data, and third image data; the third image data is an image after masking and completion processing;

[0221] Determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determine a second loss based on the third image data and the first image data;

[0222] According to the second loss and the first loss, the preset classification model is trained to obtain the target classification model.

[0223] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0224] Inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model;

[0225] Inputting the first image data into the first classification sub-model for classification to obtain a second classification result;

[0226] The first image data is input into the second initial classification sub-model for processing to obtain a first classification result and third image data.

[0227] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0228] performing mask processing on the first image data to obtain a masked image;

[0229] Inputting the masked image into the encoder for encoding to obtain an encoded image;

[0230] The encoded image is input into the decoder for completion processing to obtain third image data.

[0231] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0232] The encoded image is input into the perceptron for classification to obtain the first classification result.

[0233] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0234] Performing block processing on the first image data to obtain a plurality of image blocks;

[0235] Masking is performed on some image blocks among the multiple image blocks to obtain a masked image.

[0236] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0237] determining a corresponding third image pixel value according to the third image data;

[0238] Determining a corresponding first image pixel value according to the first image data;

[0239] A second loss is determined based on the third image pixel value and the first image pixel value.

[0240] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0241] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0242] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0243] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A training method for a classification model, characterized in that: The method comprises: Acquire unlabeled first image data and labeled second image data; Inputting the second image data into the first initial classification sub-model for training to obtain a first classification sub-model; inputting the first image data into the first classification sub-model for classification to obtain a second classification result; inputting the first image data into the second initial classification sub-model for processing to obtain a first classification result and third image data; determining a first loss based on the first classification result and a second classification result output by the first classification sub-model, and determining a second loss based on the third image data and the first image data; Training a preset classification model based on the second loss and the first loss to obtain a target classification model; The second initial classification sub-model includes an encoder and a decoder, and the first image data is input into the second initial classification sub-model for processing to obtain the third image data, including: performing mask processing on the first image data to obtain a masked image; Inputting the masked image into the encoder for encoding to obtain an encoded image; The encoded image is input into the decoder for completion processing to obtain the third image data.

2. The method according to claim 1, characterized in that The second initial classification sub-model further includes a perceptron, which inputs the first image data into the second initial classification sub-model for processing to obtain the first classification result, including: The encoded image is input into the perceptron for classification to obtain the first classification result.

3. The method according to claim 1, characterized in that The performing mask processing on the first image data to obtain a masked image includes: performing block processing on the first image data to obtain a plurality of image blocks; Masking is performed on some of the multiple image blocks to obtain the masked image.

4. The method according to any one of claims 1 to 3, characterized in that The determining of the second loss according to the third image data and the first image data includes: determining a corresponding third image pixel value according to the third image data; determining a corresponding first image pixel value according to the first image data; The second loss is determined based on the third image pixel value and the first image pixel value.

5. A training device for a classification model, characterized in that: The device comprises: an acquisition module, configured to acquire unlabeled first image data and labeled second image data; a training module, configured to input the second image data into the first initial classification sub-model for training to obtain a first classification sub-model; input the first image data into the first classification sub-model for classification to obtain a second classification result; and input the first image data into the second initial classification sub-model for processing to obtain a first classification result and third image data; a determination module, configured to determine a first loss based on the first classification result and a second classification result output by the first classification sub-model, and to determine a second loss based on the third image data and the first image data; A training module, configured to train a preset classification model based on the second loss and the first loss to obtain a target classification model; The second initial classification sub-model includes an encoder and a decoder, and the training module includes: a masking unit, specifically configured to perform masking on the first image data to obtain a masked image; an encoding unit, inputting the masked image into the encoder for encoding to obtain an encoded image; The complementing unit inputs the encoded image into the decoder for complement processing to obtain the third image data.

6. The device according to claim 5, characterized in that The second initial classification sub-model also includes a perceptron, and the training module is further configured to input the encoded image into the perceptron for classification to obtain the first classification result.

7. The device according to claim 5, characterized in that The mask unit is further configured to perform block processing on the first image data to obtain a plurality of image blocks; and perform mask processing on some of the plurality of image blocks to obtain the masked image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images