Efficient and high-compatibility semi-supervised learning model training method and device

Abstract

The invention provides an efficient and high-compatibility semi-supervised learning model training method and device. The method comprises the following steps: combining labeled and unlabeled image data into a plurality of batches according to a certain proportion; inputting the data into to-be-trained models on two branches in the ACCT semi-supervised learning framework batch by batch; calculating the prediction loss according to the prediction results output by the two models in a cross pseudo-supervision mode, and reversely propagating the prediction loss for updating the parameters of the two models at the same time; and iteratively training for multiple rounds according to the steps until the loss curves of the two models are converged. The training method of the semi-supervised learning model is optimized based on deep learning, the training effect is improved, the training steps are simplified, and the training cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing data processing, and in particular relates to an efficient and highly compatible semi-supervised learning model training method and equipment. Background Art

[0002] Semi-supervised learning is a branch of deep learning and has been widely used in the field of intelligent processing of remote sensing image data. Semi-supervised learning is a technology that uses a small number of labeled samples and a large number of unlabeled samples to learn the inherent laws and representation levels of sample data, ultimately enabling computers to analyze and process image, text, sound and other data like humans. The current training method of semi-supervised learning models is complicated, costly and has limited training accuracy, and is not well suited for specific tasks in the remote sensing field. Therefore, it is extremely important to train semi-supervised learning models with high accuracy efficiently and at low cost.

[0003] Remote sensing data has the characteristics of large quantity, complex data, and high difficulty in labeling. It has always been processed by a limited number of professionals. Therefore, the amount of labeled remote sensing data is much smaller than the amount of unlabeled remote sensing data. Using only a small amount of labeled data to train the model can easily cause the model to fall into the dilemma of overfitting, thereby reducing the generalization of the model and limiting the scope of application of the model. Therefore, fully mining the information features in unlabeled samples on the basis of labeled samples is the key to improving the accuracy of remote sensing data processing models.

[0004] The current semi-supervised learning model training methods mainly include self-training, collaborative training and consistency learning. Among them, self-training and collaborative training require multiple iterative updates of the data set according to certain rules. The training steps are relatively cumbersome and the training cost is high, and the feature learning of pseudo-label samples with low confidence is abandoned. Consistency learning is to improve the anti-interference ability of a single model by adding noise perturbations. It does not fully explore the feature information of unlabeled samples and has limited data utilization. The huge amount of remote sensing data determines a higher iterative update cost of the data set. The complex data characteristics of remote sensing data determine that the confidence of the pseudo-label samples generated by the model is generally low, and the noise perturbation cannot enable the model to fully learn the deep features of the data. Therefore, the current mainstream semi-supervised learning model training methods cannot be well used for the training of remote sensing data intelligent processing models with large data volumes and complex data characteristics. Summary of the invention

[0005] In order to overcome the problem that the above-mentioned existing semi-supervised learning model training methods cannot meet the cost requirements and accuracy requirements of remote sensing data processing, the present invention provides an efficient and highly compatible semi-supervised learning model training method and device.

[0006] According to a first aspect of an embodiment of the present invention, there is provided an efficient and highly compatible semi-supervised learning model training method, comprising the following steps:

[0007] (1) Combine labeled and unlabeled image data into several batches according to a certain ratio;

[0008] (2) Input the data batch by batch into the models to be trained on the two branches in the ACCT semi-supervised learning framework;

[0009] (3) The prediction results output by the two models are used to calculate the prediction loss through cross-pseudo-supervision, and the prediction loss is back-propagated to simultaneously update the parameters of the two models;

[0010] (4) Follow the above steps to iterate and train for multiple rounds until the loss curves of the two models converge.

[0011] Among them, the model to be trained in the present invention is a deep learning model that takes image data as input and a predicted entropy map as output.

[0012] Furthermore, the two models are two deep learning models with different structures or model parameter quantities, including:

[0013] Simultaneously train two models with the same structure but different initial model parameters;

[0014] Simultaneously train two models with different structures and similar model parameters;

[0015] Simultaneously train two models with different structures and model parameters that differ by two to a hundred times;

[0016] Simultaneously train two models with different structures and model parameters that differ by more than a hundred times.

[0017] Preferably, the sample data used to train the model in step (2) includes:

[0018] Labeled sample data. Each set of labeled sample data consists of a piece of data to be predicted in the form of a picture or text and the label corresponding to the data.

[0019] Unlabeled sample data: Each set of unlabeled sample data consists of data to be predicted in the form of a picture or text, etc., and has no corresponding label.

[0020] Among them, the sample data are combined into sample batches in a specific way according to the proportion of labeled samples and unlabeled samples, and are input into ACCT batch by batch for training two models.

[0021] Furthermore, the ratio of labeled samples to unlabeled samples should be preprocessed into a power of 2 distribution as much as possible to match the computer hardware, such as 1 / 4:3 / 4, 1 / 8:7 / 8, etc.

[0022] The method for calculating the loss in step (3) is:

[0023] When the sample data is a labeled sample, the sample loss is calculated by the prediction results of the two models and the sample label;

[0024] When the sample data is an unlabeled sample, the sample loss is calculated by the prediction results of the two models;

[0025] The loss is calculated through the loss function. You can choose classic loss functions such as MSE, CrossEntropy, MSE, MAE, etc. according to the specific task type, or design a new loss function for loss calculation.

[0026] In step (3), the losses calculated by the two models should be calculated batch by batch using the following formula and used for back propagation:

[0027] L=L s +αL c

[0028] Where L is the loss calculated for the batch for back propagation, L s is the sum of the losses calculated for the labeled samples in the batch, L c is the sum of the losses calculated for the unlabeled samples in the batch, and α is a manually set weight.

[0029] Furthermore, after forming a training sample batch, the model training needs to go through multiple rounds of iterations until the loss converges. Each round of iteration includes the steps of sample batch input, loss calculation, back propagation and updating model parameters.

[0030] According to the second aspect of an embodiment of the present invention, there is provided an efficient and highly compatible semi-supervised learning model training electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program instructions, is capable of executing a semi-supervised learning model training method provided by any possible implementation of the various possible implementations of the first aspect.

[0031] According to the third aspect of an embodiment of the present invention, a non-transitory computer-readable storage medium is also provided, which stores computer instructions so that the computer executes the semi-supervised learning model training method provided by any possible implementation of the various possible implementations of the first aspect.

[0032] The embodiment of the present invention provides an efficient and highly compatible semi-supervised learning model training method and device. The method improves the existing semi-supervised learning model training method, and the improved ACCT model training framework includes two models with different structures and a supervision module. First, the labeled data and unlabeled data are placed in two models and the results predicted by the two models from their respective perspectives are output. Then, the supervision module calculates the loss batch by batch based on whether the input data has a label, and back-propagates the final loss to synchronously update the parameters of the two models. During the training process. The model utilizes the potential information in the unlabeled data, and through mutual learning, it continuously corrects the parameters using the potential information in the unlabeled data, thereby effectively improving the prediction accuracy of the model. In addition, the ACCT model training process does not require manual intervention, and the graphics card memory and training time required for training are also less, which significantly saves training costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Here, the road extraction task in remote sensing images is taken as an example for introduction. The drawings described below are some embodiments of the present invention. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A schematic diagram of the overall process of the ACCT semi-supervised model training method provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the overall framework of the semi-supervised model training method ACCT provided in an embodiment of the present invention;

[0036] Figure 3 A sample data set formation process of the semi-supervised model training method ACCT provided in an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of the overall structure of a semi-supervised model training device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In one embodiment of the present invention, an efficient and highly compatible semi-supervised learning model training method is provided, which is introduced here by taking the road extraction task in remote sensing images as an example. Figure 1 The overall process diagram of the ACCT semi-supervised model training method provided by the embodiment of the present invention includes the following steps:

[0039] 1) Collect data according to specific tasks and organize them into sample data sets. Figure 3The sample data set formation process of the semi-supervised model training method ACCT provided by the embodiment of the present invention. First, the data to be analyzed needs to be collected according to the specific task, and the data needs to be annotated with a small amount of annotations. If the collected data is not annotated at all, it needs to be manually annotated appropriately. After this step, a small amount of labeled sample data and a large amount of unlabeled sample data are obtained. According to the proportion of labeled data and unlabeled data in the collected data, the closest power ratio of 2 is selected and used as the proportion of labeled data and unlabeled data in the subsequent sample set. According to the ratio, the collected samples are screened to obtain an unbatched sample data set. According to the ratio of labeled and unlabeled data in the unbatched sample data set, the samples are randomly combined batch by batch to ensure that the number of data in each batch of samples is the same and is the power of 2. Finally, a batched sample set is obtained. According to the needs of model training, the batched sample set can be further divided into a training set and a test set in a ratio of 8:2.

[0040] 2) The training set data is input into the two models on the two branches of ACCT in batches and two prediction results are obtained. Figure 2 The overall framework diagram of the semi-supervised model training method ACCT provided by the embodiment of the present invention includes a main model F p , Sub-model F v and a supervision module. Taking the road extraction task in remote sensing images as an example, each remote sensing image in a batch of sample data needs to be input into the main model F p and sub-model F v And get the prediction results P respectively p and P v .

[0041] 3) Calculate the prediction loss and update the model parameters. p and sub-model F v Different loss calculation methods are selected depending on whether the data in the model is labeled or not. The loss function can be the current mainstream loss function or a new loss function designed according to the task type. Taking the cross entropy loss function as an example, when the data input to the model is labeled data, the prediction loss is calculated according to the following formula:

[0042]

[0043] Among them, L s is the prediction loss of labeled data in a batch of samples, D l is the number of labeled samples in the batch, X is the data input to the model, W is the width of the input data, H is the height of the input data, l ce is the cross entropy function, p pi and p viThe main model F p and sub-model F v Output prediction results.

[0044] When the data input into the model has labeled data, the prediction loss is calculated according to the following formula:

[0045]

[0046] Among them, L c is the prediction loss of unlabeled data in a batch of samples, D u is the number of unlabeled samples in the batch, X is the data input to the model, W is the width of the input data, H is the height of the input data, l ce is the cross entropy function, p pi and p vi The main model F p and sub-model F v Output prediction results.

[0047] The final loss function L used for back propagation is calculated as follows:

[0048] L=L s +αL c ;

[0049] The ACCT method inputs samples into the model batch by batch, calculates the loss batch by batch and uses it to update the model parameters in back propagation.

[0050] Record the predicted loss value of the model in each round of iteration, and draw a loss curve with the number of iterations as the horizontal axis and the predicted loss value as the vertical axis. Observe the loss curve. When the curve has not converged, keep iterating the model; when the curve has converged, stop iterating and complete the model training.

[0051] This embodiment provides an electronic device, Figure 4 The overall structure diagram of the electronic device provided by the embodiment of the present invention includes: at least one processor 401, at least one memory 402 and a bus 403; wherein,

[0052] The processor 401 and the memory 402 complete information exchange with each other through the bus 403;

[0053] The memory 402 stores program instructions that can be executed by the processor 401. The processor calls the program instructions to execute the methods provided by the above-mentioned method embodiments, for example, including: inputting the data into the model batch by batch for prediction; updating the model parameters according to the predicted loss, and saving the model parameters to the memory; continuously updating the loss curve as the number of iterations increases, and storing the latest loss curve in the memory.

[0054] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable the computer to execute the methods provided by the above-mentioned method embodiments, for example, including: inputting the data into the model batch by batch for prediction; updating the model parameters according to the predicted loss, and saving the model parameters to the memory; continuously updating the loss curve as the number of iterations increases, and storing the latest loss curve in the memory.

[0055] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0056] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0057] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An efficient and highly compatible semi-supervised learning model training method, characterized in that: The following steps are involved: (1) Combine labeled and unlabeled image data into several batches according to a certain ratio; (2) Input the data batch by batch into the models to be trained on the two branches in the ACCT semi-supervised learning framework; (3) The prediction results output by the two models are used to calculate the prediction loss through cross-pseudo-supervision, and the prediction loss is back-propagated to simultaneously update the parameters of the two models; (4) Follow the above steps to iterate and train for multiple rounds until the loss curves of the two models converge.

2. The efficient and highly compatible semi-supervised learning model training method according to claim 1, characterized in that: The model to be trained is a deep learning model that takes image data as input and predicts entropy maps as output.

3. The efficient and highly compatible semi-supervised learning model training method according to claim 1, characterized in that: The two models are two deep learning models with different structures or model parameter quantities, including: Simultaneously train two models with the same structure but different initial model parameters; Simultaneously train two models with different structures and similar model parameters; Simultaneously train two models with different structures and model parameters that differ by two to a hundred times; Simultaneously train two models with different structures and model parameters that differ by more than a hundred times.

4. The efficient and highly compatible semi-supervised learning model training method according to claim 1, characterized in that: The sample data used to train the model in step (2) includes: Labeled sample data. Each set of labeled sample data consists of a piece of data to be predicted in the form of a picture or text and the label corresponding to the data. Unlabeled sample data: Each set of unlabeled sample data consists of data to be predicted in the form of a picture or text, etc., and has no corresponding label.

5. The efficient and highly compatible semi-supervised learning model training method according to claim 4, characterized in that: The sample data are combined into sample batches in a specific way according to the ratio of labeled samples to unlabeled samples, and are input into ACCT batch by batch for training the two models.

6. The efficient and highly compatible semi-supervised learning model training method according to claim 5, characterized in that: The ratio of labeled samples to unlabeled samples is preprocessed into a power of 2 distribution.

7. The efficient and highly compatible semi-supervised learning model training method according to claim 1, characterized in that: The method for calculating the loss in step (3) is: When the sample data is a labeled sample, the sample loss is calculated by the prediction results of the two models and the sample label; When the sample data is an unlabeled sample, the sample loss is calculated by the prediction results of the two models; The loss is calculated using the loss function; In step (3), the losses calculated by the two models should be calculated batch by batch using the following formula and used for back propagation: L=L s +αL c Where L is the loss calculated for the batch for back propagation, L s is the sum of the losses calculated for the labeled samples in the batch, L c is the sum of the losses calculated for the unlabeled samples in the batch, and α is a manually set weight.

8. The efficient and highly compatible semi-supervised learning model training method according to any one of claims 1 to 7, characterized in that: After forming a training sample batch, the model training needs to go through multiple rounds of iterations until the loss converges. Each round of iteration includes the steps of sample batch input, loss calculation, back propagation and updating model parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the program, the steps of the efficient and highly compatible semi-supervised learning model training method as described in any one of claims 1 to 8 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the efficient and highly compatible semi-supervised learning model training method as described in any one of claims 1 to 8 are implemented.

Images