Deep learning model training method, device, electronic device and storage medium

Abstract

The present application proposes a deep learning model training method, device, electronic device and storage medium, which relate to the field of computer technology. A first training image set including multiple unlabeled images and a second training image set including multiple labeled images are used as inputs of the deep learning model, and a first loss function and a second loss function corresponding to the first training image set and the second training image set are used to update the model parameters of the deep learning model. Compared with the prior art, when the deep learning model is trained, there is no need to consider the model structure characteristics, but the model parameters are updated by combining unsupervised learning and supervised learning, so that the model training method is no longer limited to a specific model structure, thereby improving the generalization of model training.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a deep learning model training method, device, electronic device and storage medium. Background Art

[0002] With the rise of deep learning, the performance of supervised learning has become increasingly powerful, but its powerful performance relies on massive amounts of manually labeled training data, and labeling massive amounts of image data is a task that requires a lot of manpower, material resources and time. On the other hand, although unsupervised learning does not require data labeling, because there are no real labels for training, it can only rely on some prior knowledge to design algorithms or manually give some labels with weak supervision information. There is a clear gap between its performance and supervised learning.

[0003] Semi-supervised learning uses both labeled image data and unlabeled image data to train the model, which can effectively reduce the model's demand for labeled image data. Among them, although unlabeled image data cannot provide real labels for supervised learning, they can reflect the data distribution of real images and help the network learn a good representation of image data. By mining the features of image data hidden in the data distribution information or better representing image data, the model can be helped to better learn image recognition tasks.

[0004] However, the current semi-supervised learning schemes for deep learning models, such as those based on Generative Adversarial Networks (GAN), have highly targeted model training schemes and are difficult to apply to the training of other models. Summary of the invention

[0005] The purpose of this application is to provide a deep learning model training method, device, electronic device and storage medium to improve the generalization of model training.

[0006] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a deep learning model training method, the method comprising:

[0008] Obtaining a first training image set and a second training image set, wherein the first training image set includes a plurality of unlabeled images, and the second training image set includes a plurality of labeled images;

[0009] Respectively using the first training image set and the second training image set as inputs of the deep learning model to obtain a first loss function and a second loss function, wherein the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model;

[0010] Update the model parameters of the deep learning model according to the first loss function and the second loss function.

[0011] In a second aspect, an embodiment of the present application provides a deep learning model training device, the device comprising:

[0012] A processing module, configured to obtain a first training image set and a second training image set, wherein the first training image set includes a plurality of unlabeled images, and the second training image set includes a plurality of labeled images;

[0013] The processing module is also used to respectively use the first training image set and the second training image set as inputs of the deep learning model to obtain a first loss function and a second loss function, wherein the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model;

[0014] An updating module is used to update the model parameters of the deep learning model according to the first loss function and the second loss function.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory for storing one or more programs; and a processor. When the one or more programs are executed by the processor, the above-mentioned deep learning model training method is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned deep learning model training method when executed by a processor.

[0017] The embodiments of the present application provide a deep learning model training method, device, electronic device and storage medium, which use a first training image set including multiple unlabeled images and a second training image set including multiple labeled images as inputs of the deep learning model, and use the first loss function and the second loss function corresponding to the first training image set and the second training image set to update the model parameters of the deep learning model. Compared with the prior art, when the deep learning model is trained, there is no need to consider the model structure characteristics, but the model parameters are updated by combining unsupervised learning and supervised learning, so that the model training method is no longer limited to a specific model structure, thereby improving the generalization of model training.

[0018] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic framework diagram of a semi-supervised learning method based on a generative adversarial network;

[0021] Figure 2 A schematic structural block diagram of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 A schematic flowchart of a deep learning model training method provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a training scenario for a deep learning model provided in an embodiment of the present application;

[0024] Figure 5 for Figure 3 A schematic flow chart of the sub-steps of S203;

[0025] Figure 6 for Figure 3 Another schematic flow chart of the sub-steps of S203;

[0026] Figure 7 for Figure 3 A schematic flow chart of the sub-steps of S205;

[0027] Figure 8A schematic structural diagram of a deep learning model training device provided in an embodiment of the present application;

[0028] In the figure: 100-electronic device; 101-memory; 102-processor; 103-communication interface; 300-deep learning model training device; 301-processing module; 302-update module. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0030] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0031] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0032] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0033] In the semi-supervised learning scheme based on the generative adversarial network mentioned above, the main idea of ​​the learning scheme is to use the generative adversarial network to learn the distribution of the real input image data (for example, including labeled images and unlabeled images), and then the generative adversarial network updates the distribution information of the image data in the weight parameters of the discriminator. The discriminator shares the weight parameters with the classifier, and then passes the distribution information of the image data to the classifier, so that the classifier can better predict the category of the image.

[0034] For example, see Figure 1 , Figure 1 This is a schematic framework diagram of a semi-supervised learning method based on a generative adversarial network. The semi-supervised learning framework based on a generative adversarial network mainly includes a generator, a discriminator and a classifier.

[0035] Among them, the generator is composed of a multi-layer deconvolution model, which can generate fake images using N-dimensional random noise sampled from a Gaussian distribution; the discriminator contains a multi-layer convolution model, which is trained to identify fake images from real images and fake images generated by the generator; the classifier and the discriminator share weight parameters and network structure, and the classifier accepts real image data (including labeled image data and unlabeled image data) and outputs the category of the real image (assuming that there are K categories in the real image).

[0036] Through adversarial training of the generator and the discriminator, the fake images generated by the generator will become closer and closer to the real images. In order to more accurately distinguish between real images and fake images generated by the generator, the discriminator will also more accurately learn the distribution of real image data and update the distribution information in its weight parameters, thereby enabling the classifier to make more accurate predictions about the authenticity and category of the image by sharing weight parameters with the discriminator.

[0037] However, for the above-mentioned training process, for example, the semi-supervised learning scheme based on the generative adversarial network, due to the original structural characteristics of the generative adversarial network, the free space for the structural design of the classification network is limited, resulting in the model training scheme being highly targeted. If the structure or even the type of the model changes, the model training scheme will be difficult to apply to the training process of other models.

[0038] Based on the above-mentioned defects, a possible implementation method provided by an embodiment of the present application is: using a first training image set containing multiple unlabeled images and a second training image set containing multiple labeled images as inputs of the deep learning model, and using the first loss function and the second loss function corresponding to the first training image set and the second training image set to update the model parameters of the deep learning model.

[0039] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0040] See also Figure 2 , Figure 2 A schematic structural block diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 can be used as a device for training a deep learning model to implement the deep learning model training method provided in an embodiment of the present application, such as a mobile phone, a personal computer (PC), a tablet computer, a laptop computer, etc.

[0041] The electronic device 100 includes a memory 101, a processor 102, and a communication interface 103. The memory 101, the processor 102, and the communication interface 103 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0042] The memory 101 can be used to store software programs and modules, such as the program instructions / modules corresponding to the deep learning model training device 300 provided in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0043] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0044] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0045] Understandably, Figure 2 The structure shown is for illustration only. The electronic device 100 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0046] Below Figure 2 Taking the electronic device 100 provided in the figure as an illustrative execution entity as an example, the deep learning model training method provided in the embodiment of the present application is further explained.

[0047] See also Figure 3 , Figure 3 A schematic flowchart of a deep learning model training method provided in an embodiment of the present application includes the following steps:

[0048] S201, obtaining a first training image set and a second training image set.

[0049] Among them, the first training image set contains multiple unlabeled images, and the multiple unlabeled images contained in the first training image set are used for unsupervised learning of the deep learning model; the second training image set contains multiple labeled images, and the multiple labeled images contained in the second training image set are used for supervised learning of the deep learning model.

[0050] S203, using the first training image set and the second training image set as inputs of the deep learning model respectively, to obtain a first loss function and a second loss function.

[0051] Among them, the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model.

[0052] S205: Update the model parameters of the deep learning model according to the first loss function and the second loss function.

[0053] In an embodiment of the present application, two training image sets, a first training image set and a second training image set, are used to train a deep learning model.

[0054] When training the deep learning model, the obtained first training image set and the second training image set are respectively used as inputs of the deep learning model to obtain a first loss function and a second loss function.

[0055] For example, see Figure 4 , Figure 4 A schematic training scenario diagram of a deep learning model provided in an embodiment of the present application, wherein the deep learning model may be composed of a multi-layer convolutional model, and may simultaneously accept inputs of two types of training images, namely a first training image set and a second training image set; wherein, since the first training image set is a set of multiple unlabeled images, when the first training image set is used as the input of the deep learning model, the deep learning model performs unsupervised learning, and accordingly, the first loss function is the loss function of the deep learning model under unsupervised learning; on the other hand, since the second training image set is a set of multiple labeled images, when the second training image set is used as the input of the deep learning model, the deep learning model performs supervised learning, and accordingly, the second loss function is the loss function of the deep learning model under supervised learning.

[0056] It is worth noting that in some possible application scenarios of the embodiments of the present application, the first training image set and the second training image set can be used as inputs of the deep learning model at the same time, and then the first loss function and the second loss function are obtained after the losses are calculated respectively; in addition, in some other possible application scenarios of the embodiments of the present application, the first training image set can be used as input of the deep learning model first, and then the first loss function is obtained after the losses are calculated, and then the second training image set is used as input of the deep learning model, and then the second loss function is obtained after the losses are calculated; the embodiments of the present application do not limit this, and the input order of the first training image set and the second training image set and the calculation order of the first loss function and the second loss function depend on the specific application scenario, as long as the first training image set and the second training image set are used as inputs of the deep learning model respectively, and the first loss function and the second loss function can be obtained after the losses are calculated respectively.

[0057] Therefore, the first loss function and the second loss function obtained by respectively training the deep learning model according to the first training image set and the second training image set are used to update the model parameters of the deep learning model, so that when the deep learning model is trained, the model parameters are updated by combining unsupervised learning and supervised learning until the trained deep learning model converges.

[0058] Based on the above design, the deep learning model training method provided in the embodiment of the present application adopts a first training image set including multiple unlabeled images and a second training image set including multiple labeled images as inputs of the deep learning model, and uses the first loss function and the second loss function corresponding to the first training image set and the second training image set to update the model parameters of the deep learning model. Compared with the prior art, when the deep learning model is trained, there is no need to consider the model structure characteristics, but the model parameters are updated by combining unsupervised learning and supervised learning, so that the model training method is no longer limited to a specific model structure, thereby improving the generalization of model training.

[0059] It is worth mentioning that, generally speaking, labeled image data relies on manual annotation of massive unlabeled images, which requires a lot of manpower, material resources and time.

[0060] Therefore, optionally, as a possible implementation method, the number of unlabeled images included in the first training image set is greater than the number of labeled images included in the second training image set. That is, in the embodiment of the present application, unsupervised training of a large number of unlabeled images is adopted, so that the deep learning model can perform feature extraction on the image data, so as to facilitate the deep learning model to learn image recognition tasks, and reduce the use of labeled image data, thereby reducing the manpower and physical resources required to train the deep learning model, making the deep learning model easier to train when using a semi-supervised learning scheme.

[0061] In addition, to achieve the purpose of obtaining the first loss function in S203, optionally, refer to Figure 5 , Figure 5 for Figure 3 A schematic flow chart of the sub-steps of S203 is shown in FIG. 1 . As a possible implementation, S203 includes the following sub-steps:

[0062] S203-1, each picture in the first training image set has missing image blocks, to obtain a third training image set.

[0063] S203-2, using the third training image set as input of the deep learning model to obtain a restored training image set.

[0064] Among them, each image in the restored training image set is restored by the deep learning model based on the image of the corresponding missing image block.

[0065] S203-3, calculating the loss according to the restored training image set and the first training image set to obtain a first loss function.

[0066] When the first training image set is used to perform unsupervised learning on the deep learning model, for each image in the first training image set, a missing image block is used, and then all images after the missing image block are combined to obtain a third training image set.

[0067] It is worth noting that, generally, the first training image set and the third training image set have the same number of pictures, and the pictures in the first training image set and the third training image set can correspond to each other one-to-one. The only difference is that the pictures included in the third training image set are missing image blocks compared to the corresponding pictures in the first training image set.

[0068] Therefore, the third training image set is used as the input of the deep learning model, and then the deep learning model restores each picture in the third training image set, and then all the restored pictures are combined to obtain a restored training image set.

[0069] It is worth noting that, generally speaking, the restored training image set and the first training image set have the same number of pictures, and the pictures in the restored training image set and the first training image set can correspond to each other one-to-one. The only difference is that the pictures included in the first training image set are original pictures, and the corresponding pictures in the restored training image set are restored through the pictures after the corresponding missing image blocks.

[0070] Obviously, it is understandable that since the corresponding pictures in the restored training image set are restored through the pictures after the corresponding missing image blocks, the pictures contained in the restored training image set may be different from the original pictures corresponding to the first training image set. The difference between the restored pictures and the original pictures can enable the deep learning model to learn to extract the features of the pictures.

[0071] Therefore, the loss is calculated based on the obtained restored training image set and the first training image set, and then the first loss function of the deep learning model under unsupervised learning is obtained.

[0072] Based on the above design, the deep learning model training method provided in the embodiment of the present application, after the image blocks in the training image set are missing, the deep learning model restores the image after the missing image blocks, and then calculates the loss function between the restored image and the original image before the missing image blocks. When the deep learning model performs self-supervised learning, the features of the image are extracted for learning by using the difference between the restored image and the original image before the missing image blocks, without the need to learn the distribution information of the images in the training image set, thereby avoiding the deterioration of model performance due to inaccurate image data.

[0073] Optionally, as a possible implementation manner, when implementing S203-1, the third training image set may be obtained by:

[0074] The third training image set is obtained by randomly missing an image block of a set size from each picture in the first training image set and combining all pictures after the missing image blocks.

[0075] It can be understood that the above-mentioned method of implementing S203-1 is only one possible implementation method, which adopts the method of randomly missing image blocks according to the set size; in some other possible application scenarios of the embodiments of the present application, the image set can also be missing in other ways to obtain the third training image set, such as obtaining the third training image set by missing fixed image blocks at set coordinate points. The present application does not limit the method of missing image blocks, which depends on the specific application scenario.

[0076] On the other hand, to achieve the purpose of obtaining the second loss function in S203, optionally, refer to Figure 6 , Figure 6 for Figure 3 Another schematic flow chart of the sub-steps of S203 in FIG. 1 , as another possible implementation, S203 includes the following sub-steps:

[0077] S203-6, using the second training image set as input of the deep learning model to obtain a training label corresponding to each image in the second training image set.

[0078] S203-7, calculating the loss according to the training label and the actual label corresponding to each image in the second training image set to obtain a second loss function.

[0079] When the second training image set is used to perform supervised learning on the deep learning model, the second training image set is used as the input of the deep learning model, and the deep learning model predicts the label of each image in the second training image set, thereby obtaining the training label corresponding to each image in the second training image set.

[0080] In an embodiment of the present application, the training labels predicted by the deep learning model for each image may be different from the actual labels corresponding to each image. Therefore, in an embodiment of the present application, the loss is calculated based on the training labels predicted by the deep learning model for each image in the second training image set and the actual labels corresponding to each image, thereby obtaining the second loss function of the deep learning model under supervised learning.

[0081] To implement the above S205, optionally, refer to Figure 7 , Figure 7 for Figure 3 A schematic flow chart of the sub-steps of S205 is shown in FIG. 1 . As a possible implementation, S205 includes the following sub-steps:

[0082] S205-1, performing weighted summation on the first loss function and the second loss function with corresponding scaling ratios to obtain a total training loss.

[0083] S205-2, updating the model parameters of the deep learning model according to the gradient of the model parameters of the deep learning model obtained by the sum of the training losses.

[0084] When the model parameters of the deep learning model are updated by using the first loss function and the second loss function, the first loss function and the second loss function are weightedly summed with corresponding scaling ratios to obtain the total training loss.

[0085] Therefore, according to the sum of the training losses obtained by weighted summation, the gradient of the model parameters of the deep learning model is obtained, and then according to the obtained gradient, the model parameters of the deep learning model are updated.

[0086] Among them, since the sum of training losses includes both the part of the first loss function and the part of the second loss function, the sum of training losses not only includes the learning information of the deep learning model under unsupervised learning, but also includes the learning information of the deep learning model under supervised learning. When the model parameters of the deep learning model are updated and iterated, unsupervised learning and supervised learning simultaneously optimize the deep learning model, and the two learning methods promote each other. The deep learning model learns to extract the features of the image through the first training image set of unlabeled images, thereby helping the model to recognize the image data and improve the generalization of the model.

[0087] Optionally, as a possible implementation manner, the scaling ratios corresponding to the first loss function and the second loss function are both 1.

[0088] It is worth noting that in some other possible implementation methods of the embodiments of the present application, values ​​can also be used as the scaling ratios corresponding to the first loss function and the second loss function, for example, the scaling ratio corresponding to the first loss function is 0.5, and the scaling ratio corresponding to the second loss function is 0.8. This depends on the specific application scenario. For example, in some other possible application scenarios of the embodiments of the present application, the scaling ratio corresponding to the first loss function can be set to 0.6, and the scaling ratio corresponding to the second loss function can be set to 0.9.

[0089] In addition, in some possible implementations of the embodiments of the present application, the scaling ratios corresponding to the first loss function and the second loss function may be pre-stored in the training device before the deep learning model is trained. In some other possible implementations of the embodiments of the present application, the scaling ratios may be obtained by receiving user input during the training of the deep learning model.

[0090] Based on the same inventive concept as the deep learning model training method provided in the above embodiment of the present application, please refer to Figure 8 , Figure 8 A schematic structural diagram of a deep learning model training device 300 provided in an embodiment of the present application, wherein the deep learning model training device 300 includes a processing model 301 and an update module 302.

[0091] The processing module 301 is used to obtain a first training image set and a second training image set, wherein the first training image set includes a plurality of unlabeled images, and the second training image set includes a plurality of labeled images;

[0092] The processing module 301 is also used to use the first training image set and the second training image set as inputs of the deep learning model, respectively, to obtain a first loss function and a second loss function, wherein the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model;

[0093] The updating module 302 is used to update the model parameters of the deep learning model according to the first loss function and the second loss function.

[0094] Optionally, as a possible implementation manner, when the processing module 301 uses the first training image set as the input of the deep learning model and obtains the first loss function, it is specifically used to:

[0095] Each image in the first training image set is missing an image block to obtain a third training image set;

[0096] Using the third training image set as an input of the deep learning model to obtain a restored training image set, wherein each image in the restored training image set is restored by the deep learning model based on an image corresponding to a missing image block;

[0097] The loss is calculated according to the restored training image set and the first training image set to obtain a first loss function.

[0098] Optionally, as a possible implementation, when the processing module 301 obtains the third training image set by removing the image blocks from each picture in the first training image set, the processing module 301 is specifically used to:

[0099] An image block of a set size is randomly deleted from each picture in the first training image set, and all pictures after the missing image blocks are combined to obtain a third training image set.

[0100] Optionally, as a possible implementation manner, the number of unlabeled images included in the first training image set is greater than the number of labeled images included in the second training image set.

[0101] Optionally, as a possible implementation manner, when the processing module 301 uses the second training image set as the input of the deep learning model and obtains the second loss function, it is specifically used to:

[0102] Using the second training image set as input of the deep learning model, and obtaining training labels corresponding to each image in the second training image set;

[0103] The loss is calculated according to the training labels and actual labels corresponding to each image in the second training image set to obtain a second loss function.

[0104] Optionally, as a possible implementation manner, when updating the model parameters of the deep learning model according to the first loss function and the second loss function, the updating module 301 is specifically used to:

[0105] The first loss function and the second loss function are weighted and summed with corresponding scaling ratios to obtain the total training loss;

[0106] Update the model parameters of the deep learning model according to the gradient of the sum of training losses with respect to the model parameters of the deep learning model.

[0107] Optionally, as a possible implementation manner, the scaling ratios corresponding to the first loss function and the second loss function are both 1.

[0108] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the module, program segment or a part of a code includes one or more executable instructions for implementing a specified logical function.

[0109] It should also be noted that in some alternative implementations, the functions marked in the blocks may also occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved.

[0110] It should also be noted that each box in the block diagram and / or flowchart, and combinations of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0111] In addition, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0112] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0113] In summary, the embodiments of the present application provide a deep learning model training method, device, electronic device and storage medium, which use a first training image set including multiple unlabeled images and a second training image set including multiple labeled images as inputs of the deep learning model, and use the first loss function and the second loss function corresponding to the first training image set and the second training image set to update the model parameters of the deep learning model. Compared with the prior art, when the deep learning model is trained, there is no need to consider the model structure characteristics, but the model parameters are updated by combining unsupervised learning and supervised learning, so that the model training method is no longer limited to a specific model structure, thereby improving the generalization of model training.

[0114] In addition, after the image blocks in the pictures in the training image set are missing, the deep learning model restores the picture after the missing image blocks, and then calculates the loss function between the restored picture and the original picture before the missing image blocks. When the deep learning model performs self-supervised learning, the features of the picture are extracted for learning through the difference between the restored picture and the original picture before the missing image blocks, without learning the distribution information of the images in the training image set, thereby avoiding the deterioration of model performance due to inaccurate image data.

[0115] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0116] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A deep learning model training method, characterized in that: The method comprises: Obtaining a first training image set and a second training image set, wherein the first training image set includes a plurality of unlabeled images, and the second training image set includes a plurality of labeled images; Respectively using the first training image set and the second training image set as inputs of the deep learning model to obtain a first loss function and a second loss function, wherein the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model; Update the model parameters of the deep learning model according to the first loss function and the second loss function; The step of using the first training image set as the input of the deep learning model to obtain a first loss function includes: Each picture in the first training image set is missing an image block to obtain a third training image set; Using the third training image set as input of the deep learning model to obtain a restored training image set, wherein each image in the restored training image set is restored by the deep learning model based on an image corresponding to a missing image block; Calculate the loss according to the restored training image set and the first training image set to obtain the first loss function; The step of using the second training image set as the input of the deep learning model to obtain a second loss function comprises: Using the second training image set as input of the deep learning model, and obtaining training labels corresponding to each image in the second training image set; The loss is calculated according to the training label and the actual label corresponding to each picture in the second training image set to obtain the second loss function.

2. The method according to claim 1, characterized in that The step of obtaining a third training image set by removing image blocks from each image in the first training image set comprises: An image block of a set size is randomly deleted from each picture in the first training image set, and all pictures after the missing image blocks are combined to obtain the third training image set.

3. The method according to claim 1, characterized in that The number of unlabeled images included in the first training image set is greater than the number of labeled images included in the second training image set.

4. The method according to any one of claims 1 to 3, characterized in that The step of updating the model parameters of the deep learning model according to the first loss function and the second loss function comprises: Performing weighted summation on the first loss function and the second loss function at corresponding scaling ratios to obtain a total training loss; Update the model parameters of the deep learning model according to the gradient of the model parameters of the deep learning model obtained by the sum of the training losses.

5. The method according to claim 4, characterized in that The scaling ratios corresponding to the first loss function and the second loss function are both 1.

6. A deep learning model training device using the deep learning model training method according to claim 1, characterized in that: The device comprises: A processing module, configured to obtain a first training image set and a second training image set, wherein the first training image set includes a plurality of unlabeled images, and the second training image set includes a plurality of labeled images; The processing module is also used to respectively use the first training image set and the second training image set as inputs of the deep learning model to obtain a first loss function and a second loss function, wherein the first loss function is the loss function when the first training image set is used as input to train the deep learning model, and the second loss function is the loss function when the second training image set is used as input to train the deep learning model; An updating module is used to update the model parameters of the deep learning model according to the first loss function and the second loss function.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images