Deep learning model training method and device and electronic equipment

Abstract

The invention discloses a deep learning model training method and device and electronic equipment, and belongs to the technical field of image processing and the field of industrial detection. The method comprises the following steps: training an initial deep learning model based on a training set of a current stage; under the condition that the number of stages ending the current stage is determined to be smaller than the target number of stages and the trained deep learning model of the current stage is determined to not meet the target requirement based on the test set of the current stage, performing augmentation processing on error detection test samples in the test set of the current stage to obtain augmented samples; according to the training set, the test set and the augmented sample of the current stage, obtaining a training set and a test set of the next stage; and when the number of stages ending the current stage is determined to be not less than the target number of stages, or the trained deep learning model of the current stage is determined to meet the target requirement based on the test set of the current stage, the trained deep learning model of the current stage is output, the training sample does not need to be manually adjusted, and the manpower resource consumption and the time consumption are reduced.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology and industrial detection, and in particular, relates to a deep learning model training, device and electronic equipment. Background Art

[0002] In recent years, the field of deep learning has continued to develop, and deep learning models have also begun to be applied in the field of industrial inspection. In industrial production lines, from proofing to mass production, there are multiple stages, and the process and form of the product will be adjusted at each stage. At each stage, due to the consistency differences of the upstream process materials of the product, the sample image will change. After the sample image changes, it is easy to cause over-detection by deep learning model detection. Related technologies adjust the sample image through artificial high experience, which will bring high human resource consumption and time cost. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a training, device and electronic device for a deep learning model, which automatically improves the richness of the training set and test set in the next stage without manually adjusting the training samples, while reducing human resource consumption and time consumption.

[0004] In a first aspect, the present application provides a method for training a deep learning model, the method comprising:

[0005] The initial deep learning model is trained based on the training set of this stage to obtain the trained deep learning model of this stage;

[0006] When it is determined that the number of stages ending in this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, augmenting the falsely detected test samples in the test set of this stage to obtain augmented samples;

[0007] According to the training set, test set and augmented samples of this stage, the training set and test set of the next stage are obtained;

[0008] When it is determined that the number of stages up to this stage is not less than the target number of stages, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output.

[0009] According to the training method of the deep learning model of the present application, the initial deep learning model is trained based on the training set of this stage to obtain the trained deep learning model of this stage; when it is determined that the number of stages up to this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, the false detection test samples in the test set of this stage are augmented to obtain augmented samples; according to the training set, test set and augmented samples of this stage, the training set and test set of the next stage are obtained; when it is determined that the number of stages up to this stage is not less than the target number of stages, or it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output; by augmenting the false detection test samples, there is no need to manually adjust the training samples, which improves the richness of the training set and test set of the next stage while reducing the consumption of human resources and time. In addition, it can quickly adapt to changes in samples in the production process, can reduce the probability of false detection after the deep learning model is put online, and reduce the difficulty of training and iterating new models for on-site personnel.

[0010] According to an embodiment of the present application, the false detection test samples include missed detection samples and over-detected samples; determining that the trained deep learning model in this stage does not meet the target requirements based on the test set in this stage includes:

[0011] Obtain the number of missed samples and the number of passed samples in the test set of this stage;

[0012] When it is determined that the number of missed detections in this stage is not less than the target missed detection number, or the number of passed inspections in this stage is not less than the target passed inspection number, it is determined that the trained deep learning model in this stage does not meet the target requirements.

[0013] According to an embodiment of the present application, augmentation processing is performed on the falsely detected test samples in the test set of this stage to obtain augmented samples, including:

[0014] Based on the preset offline augmentation strategy and / or the trained artificial intelligence generated content model, the false detection test samples in the test set of this stage are augmented to obtain augmented samples;

[0015] The offline augmentation strategy includes at least one of modifying the illumination parameters and background color of the falsely detected test sample and deforming the falsely detected test sample.

[0016] According to one embodiment of the present application, training the initial deep learning model based on the training set of this stage includes:

[0017] The initial deep learning model is trained based on the early stopping strategy and the training set of this stage until the training stop condition of this stage is met; the training stop condition of this stage includes any of the following:

[0018] The objective loss function of the initial deep learning model in this stage converges;

[0019] The training rounds of forward and backward propagation training of the initial deep learning model in this stage have reached the first target round in this stage.

[0020] According to one embodiment of the present application, when the training stop condition of the current stage is that the training rounds of the forward and backward propagation training of the initial deep learning model of the current stage reach the first target round of the current stage, after determining that the trained deep learning model of the current stage does not meet the target requirement based on the test set of the current stage, the method includes:

[0021] When it is determined that the training round of this stage reaches the first target round and the target loss function does not converge, the first target round of this stage is updated to obtain the first target round of the next stage training;

[0022] The first target round of training in the next stage is not less than the first target round of the current stage, but not greater than the second target round.

[0023] According to one embodiment of the present application, before training the initial deep learning model based on the training set of this stage, the method further includes:

[0024] Obtain the initial sample set and training parameters input by the user; the initial sample set is the sample set of the first stage; the training parameters include the augmentation parameters corresponding to the offline augmentation strategy and the training stop conditions of each stage;

[0025] Configure the batch size of the initial deep learning model based on the maximum capacity of the current graphics card resources;

[0026] The network width and height of the initial deep learning model are configured according to the width and height of the maximum image of the sample images in the initial sample set.

[0027] According to one embodiment of the present application, obtaining a training set and a test set for the next stage according to the training set, the test set, and the augmented samples of the current stage includes:

[0028] The training set, test set and augmented samples of this stage are used as the sample sets of the next stage;

[0029] The sample set of the next stage is divided into the training set and the test set of the next stage based on the target ratio.

[0030] In a second aspect, the present application provides a training device for a deep learning model, the device comprising:

[0031] The first processing module is used to train the initial deep learning model based on the training set of this stage to obtain the trained deep learning model of this stage;

[0032] The second processing module is used to perform augmentation processing on the false detection test samples in the test set of this stage to obtain augmented samples when it is determined that the stage number ending in this stage is less than the target stage number and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements;

[0033] The third processing module is used to obtain the training set and test set of the next stage according to the training set, test set and augmented samples of this stage;

[0034] The fourth processing module is used to output the trained deep learning model of this stage when it is determined that the number of stages up to this stage is not less than the target number of stages, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements.

[0035] According to the training device of the deep learning model of the present application, the initial deep learning model is trained based on the training set of this stage to obtain the trained deep learning model of this stage; when it is determined that the number of stages up to this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, the false detection test samples in the test set of this stage are augmented to obtain augmented samples; according to the training set, test set and augmented samples of this stage, the training set and test set of the next stage are obtained; when it is determined that the number of stages up to this stage is not less than the target number of stages, or it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output; by augmenting the false detection test samples, there is no need to manually adjust the training samples, which improves the richness of the training set and test set of the next stage while reducing the consumption of human resources and time. In addition, it can quickly adapt to changes in samples in the production process, can reduce the probability of false detection after the deep learning model is put online, and reduce the difficulty of training and iterating new models for on-site personnel.

[0036] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the deep learning model provided in the first aspect above is implemented.

[0037] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the deep learning model provided in the first aspect above.

[0038] In a fifth aspect, the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the training method of the deep learning model provided in the first aspect.

[0039] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the deep learning model provided in the first aspect above.

[0040] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0041] The trained deep learning model of this stage is obtained by training the initial deep learning model based on the training set of this stage; when it is determined that the number of stages up to this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, the false detection test samples in the test set of this stage are augmented to obtain augmented samples; according to the training set, test set and augmented samples of this stage, the training set and test set of the next stage are obtained, and when it is determined that the number of stages up to this stage is not less than the target number of stages, or it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output. By augmenting the false detection test samples, there is no need to manually adjust the training samples, which improves the richness of the training set and test set of the next stage while reducing the consumption of human resources and time. In addition, it can quickly adapt to changes in samples in the production process, reduce the probability of false detection after the deep learning model is put online, and reduce the difficulty of training and iterating new models for on-site personnel.

[0042] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 This is one of the flowcharts of the training method of the deep learning model provided in the embodiment of the present application;

[0045] Figure 2 This is the second flow chart of the training method of the deep learning model provided in the embodiment of the present application;

[0046] Figure 3 It is a structural schematic diagram of a training device for a deep learning model provided in an embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] In the following, in combination with the accompanying drawings, the deep learning model training method, deep learning model training device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0051] Among them, the training method of the deep learning model can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0052] The training method for a deep learning model provided in an embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the training method for the deep learning model. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The training method for a deep learning model provided in an embodiment of the present application is described below using an electronic device as an example of an execution subject.

[0053] like Figure 1 As shown, the training method of the deep learning model includes: step 110, step 120, step 130 and step 140.

[0054] Step 110: Train the initial deep learning model based on the training set of this stage to obtain the trained deep learning model of this stage. Execute step 120 or step 140.

[0055] The initial deep learning model in the embodiment of the present application refers to a deep learning model to be trained and not converged. The deep learning model can be a deep learning model of any function or task, such as classification, segmentation, detection, instance segmentation, OCR recognition, AI positioning, etc.

[0056] The training samples and training labels of the initial deep learning model can be set according to actual conditions. For example, when the initial deep learning model is an image classification model, the training samples are sample images, and the training labels are image categories (such as people, animals), etc.; for example, when the initial deep learning model is an image segmentation model, the training samples are sample images, and the training labels include the image's true label mask image, the target object's circumscribed rectangular frame, and the target object's category, etc.

[0057] The training process of the initial deep learning model in the embodiment of the present application is an intelligent training process. Before the intelligent training process, the user inputs the initial training set and performs a series of configurations. After the configuration is completed, the configuration is submitted to trigger "one-click intelligent training". During the intelligent training process, the initial deep learning model will be trained for at least one stage. Each stage of training is multiple rounds of forward and reverse propagation training of the initial deep learning model, that is, the training of each stage includes multiple rounds of forward and reverse propagation training.

[0058] A round of forward and backward propagation training (referred to as a training epoch) refers to the process in which the entire training data set undergoes a complete forward propagation and backward propagation through the neural network.

[0059] In the embodiment of the present application, each stage of training has a corresponding sample set. The sample set of the first stage is an initial sample set input by the user. The initial sample set can be sample images collected by the user from various stages of the production line.

[0060] The sample sets at each stage are divided into training sets and test sets, which can be divided based on a certain target ratio, such as 7:3, 8:2, and 9:1, etc., or other ratios. Both the training set and the test set include multiple samples, and the number of samples in the training set is usually greater than the number of samples in the test set. The training set is used to train the initial deep learning model, and the samples in the training set are training samples; the test set is used to evaluate the performance of the trained deep learning model, which can be evaluated by testing indicators such as accuracy, false detection (false detection probability, number of false detections), and the samples in the test set are test samples.

[0061] In this stage, the initial deep learning model can be trained based on the training set of this stage until the training stop condition of this stage is met.

[0062] The training of deep learning models is not endless. You can set the training stop condition for this stage. The training stop condition includes any of the following:

[0063] The target loss function of the initial deep learning model in this stage converges, and the training can be stopped early through the early stopping strategy (EarlyStopping).

[0064] The training rounds of the forward and reverse propagation training of the initial deep learning model in this stage reach the first target round of this stage. The first target round is the maximum round of this stage and belongs to the preset threshold, such as 100 rounds.

[0065] After this stage stops, the trained deep learning model of this stage is obtained.

[0066] Step 120: When it is determined that the number of stages ending in this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, augment the false positive test samples in the test set of this stage to obtain augmented samples. Execute step 130.

[0067] After obtaining the trained deep learning model of this stage, the embodiment of the present application needs to detect the performance of the trained deep learning model of this stage to determine whether the trained deep learning model is a deep learning model that meets the target requirements. The false detection situation of the test set can be used as an indicator for evaluating the performance of the trained deep learning model of this stage.

[0068] False detection situations include but are not limited to the number of false detections and the probability of false detection of false detection test samples. False detection test samples refer to test samples in the test set that identify "non-target objects" as "target objects", or identify "target objects" as "non-target objects". The target object can be any person, object, etc. For example, in the field of image processing defect detection, the target object can refer to "defects". The probability of false detection refers to the ratio of the number of false detections of false detection test samples to the number of test samples in the test set.

[0069] When the deep learning model is an image classification model, when it is determined that the true classification label and the predicted classification label are different, it can be determined that the test sample is a falsely detected test sample; when it is determined that the true classification label and the predicted classification label are the same, it can be determined that the test sample is not a falsely detected test sample but an error-free sample.

[0070] When the deep learning model is an image segmentation model, it can be determined whether the test sample is a falsely detected test sample based on the actual label mask image and the predicted label mask image of the test sample. For a certain test sample, if it is determined that the actual label mask image and the predicted label mask image of the test sample are inconsistent, it can be determined that the test sample is a falsely detected test sample; if it is determined that the actual label mask image and the predicted label mask image of the test sample are consistent, it can be determined that the test sample is not a falsely detected test sample, but a correct sample.

[0071] In an embodiment of the present application, each test sample in the test set of this stage can be input into a trained deep learning model to obtain the output result of the trained deep learning model for each test sample, and each output result can be tested to obtain the test result of each test sample, where the test result is a correct sample or a falsely detected test sample.

[0072] The false detection situation of the embodiment of the present application can be characterized by at least one of the false detection number and false detection probability of the false detection test sample mentioned above. Specifically, assuming that the false detection situation is characterized by the false detection number, then when it is determined that the false detection number of the trained deep learning model at this stage is not less than the target false detection number, it can be determined that the trained deep learning model at this stage does not meet the target requirements. When it is determined that the false detection number of the trained deep learning model at this stage is less than the target false detection number, it can be determined that the trained deep learning model at this stage meets the target requirements.

[0073] In the embodiment of the present application, the training of each stage of the initial deep learning model is not endless. After each stage of training, the number of stages up to the current stage will be counted. At the same time, it is also necessary to determine whether the trained deep learning model of this stage meets the target requirements. When it is determined that the number of stages up to the current stage is less than the target number of stages (for example, the target number of stages is 3, 4, etc., and the target number of stages is usually a small positive integer), and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, the false detection test samples in the test set of this stage are augmented to obtain augmented samples. The augmented samples can be obtained by rotating, scaling, cropping, flipping, color changing, and lighting the false detection test samples.

[0074] In some embodiments, one or more augmented samples corresponding to each falsely detected test sample may be generated, and the falsely detected test samples in the test set of this stage may be augmented based on a preset offline augmentation strategy and / or a trained artificial intelligence generated content model (Artificial Intelligence Generated Content, AIGC) to obtain augmented samples. The detailed process is described in subsequent content.

[0075] Step 130: Obtain the training set and test set of the next stage according to the training set, test set and augmented samples of this stage.

[0076] After obtaining the augmented samples of this stage, the present application uses the training set, test set and augmented samples of this stage as the sample set of the next stage, enriching the richness of the sample set of the next stage. The sample set of the next stage can be divided into the training set and test set of the next stage based on the target ratio, and the target ratio can be 7:3, 8:2, 9:1, etc., or other ratios.

[0077] After obtaining the training set and test set of the next stage, the embodiment of the present application can perform the next round of training, that is, return to the above step 110 to start execution.

[0078] Step 140: When it is determined that the number of stages up to this stage is not less than the target number of stages, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output.

[0079] The above-mentioned embodiments have explained that the training of each stage of the initial deep learning model in the embodiments of the present application is not endless. After each stage of training, the number of stages up to the current stage will be counted. At the same time, it is also necessary to determine whether the trained deep learning model of this stage meets the target requirements. When it is determined that the number of stages up to the current stage is not less than the target number of stages, the training of each stage is stopped. Or, when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, it can also be determined that the training of each stage of the initial deep learning model is stopped, and the trained deep learning model of this stage can be output.

[0080] In addition, the trained deep learning model of this stage can be saved and deployed online, so that the images to be processed in the online stage can be processed after going online.

[0081] The embodiment of the present application trains the initial deep learning model based on the training set of this stage to obtain the trained deep learning model of this stage; when it is determined that the number of stages up to this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, the false detection test samples in the test set of this stage are augmented to obtain augmented samples; according to the training set, test set and augmented samples of this stage, the training set and test set of the next stage are obtained; when it is determined that the number of stages up to this stage is not less than the target number of stages, or it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output; by augmenting the false detection test samples, there is no need to manually adjust the training samples, which improves the richness of the training set and test set of the next stage while reducing the consumption of human resources and time. In addition, it can quickly adapt to changes in samples in the production process, can reduce the probability of false detection after the deep learning model is put online, and reduce the difficulty of training and iterating new models for on-site personnel.

[0082] In some embodiments, the false positive test samples include missed positive samples and over-positive samples; determining that the trained deep learning model at this stage does not meet the target requirements based on the test set at this stage includes:

[0083] Obtain the number of missed samples and the number of passed samples in the test set of this stage;

[0084] When it is determined that the number of missed detections in this stage is not less than the target missed detection number, or the number of passed inspections in this stage is not less than the target passed inspection number, it is determined that the trained deep learning model in this stage does not meet the target requirements.

[0085] The false detection test samples in the embodiment of the present application include missed detection samples and over-detected samples. The missed detection samples refer to the test samples that identify the "target object" as a "non-target object", and the false detection samples refer to the test samples that identify the "non-target object" as a "target object" in the test set.

[0086] The number of missed detections and the number of passed detections in the test set of this stage, as well as the target number of missed detections and the target number of passed detections used to evaluate the performance of the trained deep learning model can be obtained. The target number of missed detections and the target number of passed detections both belong to the preset thresholds, for example, the target number of missed detections is 1 and the target number of passed detections is 5.

[0087] If it is determined that the number of missed detections in this stage is not less than the target missed detection number, or the number of passed inspections in this stage is not less than the target passed inspection number, it is determined that the trained deep learning model in this stage does not meet the target requirements. Continuing with the above example, for example, if the number of missed detections in this stage is 2 (greater than the target missed detection number) and the number of passed inspections in this stage is 4, it can be determined that the trained deep learning model in this stage does not meet the target requirements.

[0088] When it is determined that the number of missed detections in this stage is less than the target missed detection number, and the number of passed inspections in this stage is less than the target passed inspection number, it is determined that the trained deep learning model in this stage meets the target requirements.

[0089] In addition, in some embodiments, the missed detection probability of missed samples and the over-detection probability of over-detected samples in the test set of this stage can be obtained; when it is determined that the missed detection probability of this stage is not less than the target missed detection probability, or the over-detection probability of this stage is not less than the target over-detection probability, it is determined that the trained deep learning model of this stage does not meet the target requirements.

[0090] In some embodiments, augmenting the falsely detected test samples in the test set of this stage to obtain augmented samples includes:

[0091] Based on the preset offline augmentation strategy and / or the trained artificial intelligence generated content model, the false detection test samples in the test set of this stage are augmented to obtain augmented samples;

[0092] Among them, the offline augmentation strategy includes modifying the lighting parameters and background color of the falsely detected test samples and deforming the falsely detected test samples. The deformation of the falsely detected test samples may include rotation, scaling, cropping, flipping, etc. These changes help to improve the generalization ability and robustness of the model.

[0093] The AIGC model is a type of model that uses artificial intelligence technology to generate various types of content, which can include text, images, audio, video, etc., covering multiple fields such as creation, design, entertainment, news, and marketing. The AIGC model is usually based on technologies such as deep learning and natural language processing (NLP), and is trained using a large amount of data to generate content that meets certain rules or requirements. In an embodiment of the present application, the AIGC model can be used to inject target objects (such as defects) into the false detection test samples in the test set of this stage.

[0094] The augmentation at each stage can adopt different combinations of offline augmentation techniques in a random and strategic manner to improve the generalization and enhancement of missed images and quickly reach the end of training requirements.

[0095] In some embodiments, the initial deep learning model is trained based on the training set of this stage, including:

[0096] The initial deep learning model is trained based on the early stopping strategy and the training set of this stage until the training stop condition of this stage is met; the training stop condition of this stage includes any of the following:

[0097] The objective loss function of the initial deep learning model in this stage converges;

[0098] The training rounds of forward and backward propagation training of the initial deep learning model in this stage have reached the first target round in this stage.

[0099] Early Stopping is a technique to prevent overfitting of neural network models during training. It monitors the performance of the validation set during training and stops training in advance to avoid the situation where the model performs too well on the training set but poorly on the validation set, thereby improving the generalization ability of the model. The early stopping strategy can be used to determine whether the target loss function has converged, and then automatically stop the training of this stage in advance.

[0100] In each stage, multiple rounds of forward and backward propagation training are performed on the initial deep learning model. When the target loss function of the initial deep learning model in this stage converges, and / or the training rounds of the forward and backward propagation training of the initial deep learning model in this stage reach the first target round of this stage, it is determined that the training of this stage is stopped. The first target round can be set by the user according to the actual situation. It can be a larger value, such as 100 rounds, or a smaller value, such as 30 rounds.

[0101] In some embodiments, when the training stop condition of the current stage is that the training rounds of the forward and backward propagation training of the initial deep learning model of the current stage reach the first target round of the current stage, after determining that the false detection situation of the trained deep learning model of the current stage does not meet the target situation based on the test set of the current stage, the method includes:

[0102] When it is determined that the training round of this stage reaches the first target round and the target loss function does not converge, the first target round of this stage is updated to obtain the first target round of the next stage training;

[0103] The first target round of training in the next stage is not less than the first target round of the current stage, but not greater than the second target round.

[0104] The training stop condition in this stage is that the training rounds of the forward and backward propagation training of the initial deep learning model in this stage reach the first target round of this stage. The first target round can be a smaller value, such as 30 times, so that the training can be completed quickly. However, it is also possible that the training rounds in this stage reach the first target round, but the target loss function does not converge. In this case, the first target round of this stage can be updated to obtain the first target round of training in the next stage. The target round of training in the next stage is not less than the first target round of this stage, but less than the second target round. The first target round and the second target round are both preset round thresholds. The first target round is the round at which training is stopped in this stage, and the second target round is the maximum value of the first target rounds of each stage.

[0105] For example, if the first target round of this stage is 30 times and the second target round is 100 times, then the first target round of the next stage can be any positive integer between 30 and 100, such as 35.

[0106] In some embodiments, before training the initial deep learning model based on the training set of this stage, the method further includes:

[0107] Obtain the initial sample set and training parameters input by the user; the initial sample set is the sample set of the first stage; the training parameters include the augmentation parameters corresponding to the offline augmentation strategy and the training stop conditions of each stage of training;

[0108] Configure the batch size of the initial deep learning model based on the maximum capacity of the current graphics card resources;

[0109] The network width and height of the initial deep learning model are configured according to the width and height of the maximum image of the sample images in the initial sample set.

[0110] To achieve "one-click intelligent training", users are required to make a series of preparations and configurations. Specifically, users are required to input the initial sample set and training parameters. The initial sample set is the sample set of the first stage and will be divided into the training set and test set of the first stage.

[0111] The training parameters include the augmentation parameters corresponding to the offline augmentation strategy and the training stop conditions at each stage (first target round, second target round, etc.). These can avoid manually setting these parameters during the training process and lower the threshold for use.

[0112] In addition, the batch size of training is automatically configured according to the maximum capacity of the current graphics card resources, and the width and height of the trained network will use the largest image of the current initial sample set (limited to the range of 512*512). Based on these default rules, no manual configuration is required, which greatly improves the efficiency and convenience of deep learning models in the industrial quality inspection industry and reduces the investment in human resources.

[0113] In some embodiments, obtaining the training set and test set of the next stage according to the training set, test set and augmented samples of the current stage includes:

[0114] The training set, test set and augmented samples of this stage are used as the sample sets of the next stage;

[0115] The sample set of the next stage is divided into the training set and the test set of the next stage based on the target ratio.

[0116] This has been explained in the above-mentioned embodiment and will not be described again here.

[0117] like Figure 2 As shown, the present application embodiment provides a flowchart of a second method for training a deep learning model, comprising the following steps:

[0118] Step 201, obtaining the initial sample set and training parameters input by the user; the initial sample set is the sample set of the first stage; the training parameters include the augmentation parameters corresponding to the offline augmentation strategy and the training stop conditions of each stage; execute step 202;

[0119] Step 202, configuring the batch size of the initial deep learning model according to the maximum capacity of the current graphics card resources; executing step 203;

[0120] Step 203, configuring the network width and height of the initial deep learning model according to the width and height of the maximum image of the sample images in the initial sample set; executing step 204;

[0121] Step 204, training the initial deep learning model based on the training set of this stage; the training set and test set of the first stage are obtained by dividing the initial sample set; execute step 205;

[0122] Step 205, determine whether the training stop condition of this stage is met; if so, execute step 206; if not, execute step 204;

[0123] Step 206, obtaining the trained deep learning model of this stage; executing step 207;

[0124] Step 207, determining whether the stage number ending at the current stage is less than the target stage number, if so, executing step 208; if not, executing step 213;

[0125] Step 208, obtaining the number of missed detection samples and the number of passed detection samples in the test set of this stage; executing step 209;

[0126] Step 209, determining whether the number of missed detections in this phase is less than the target missed detection number; if so, executing step 213; if not, executing step 210;

[0127] Step 210, determining whether the number of passed inspections in this stage is less than the target number of passed inspections; if so, executing step 213; if not, executing step 211;

[0128] Step 211, based on the preset offline augmentation strategy and / or the trained artificial intelligence generated content model, the false detection test samples in the test set of this stage are augmented to obtain augmented samples; and step 212 is executed;

[0129] Step 212, obtaining the training set and test set of the next stage according to the training set, test set and augmented samples of this stage; executing step 204;

[0130] Step 213, output the trained deep learning model of this stage.

[0131] The detailed implementation process of the above steps 201 to 213 is shown in the above embodiment and is not limited thereto.

[0132] The embodiment of the present application can realize intelligent training based on the above process, without the need for manual adjustment of training samples, and without the need for manual evaluation of whether the deep learning model is qualified after each training. This greatly improves the efficiency and convenience of the application of deep learning models in the industrial quality inspection industry, and also reduces the investment in human resources.

[0133] The training method of the deep learning model provided in the embodiment of the present application can be executed by a training device for the deep learning model. In the embodiment of the present application, the training method of the deep learning model performed by the training device for the deep learning model is taken as an example to illustrate the training device for the deep learning model provided in the embodiment of the present application.

[0134] An embodiment of the present application also provides a training device for a deep learning model.

[0135] like Figure 3 As shown, the training device of the deep learning model includes: a first processing module 310, a second processing module 320, a third processing module 330 and a fourth processing module 340.

[0136] A first processing module 310 is used to train the initial deep learning model based on the training set of this stage to obtain a trained deep learning model of this stage;

[0137] The second processing module 320 is used to perform augmentation processing on the falsely detected test samples in the test set of this stage to obtain augmented samples when it is determined that the stage number ending in this stage is less than the target stage number and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements;

[0138] The third processing module 330 is used to obtain the training set and test set of the next stage according to the training set, test set and augmented samples of the current stage;

[0139] The fourth processing module 340 is used to output the trained deep learning model of this stage when it is determined that the stage number ending in this stage is not less than the target stage number, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements.

[0140] According to the training device of the deep learning model provided in the embodiment of the present application, the trained deep learning model of the present stage is obtained by training the initial deep learning model based on the training set of the present stage; when it is determined that the number of stages up to the present stage is less than the target number of stages, and it is determined based on the test set of the present stage that the trained deep learning model of the present stage does not meet the target requirements, the false detection test samples in the test set of the present stage are augmented to obtain augmented samples; according to the training set, test set and augmented samples of the present stage, the training set and test set of the next stage are obtained; when it is determined that the number of stages up to the present stage is not less than the target number of stages, or it is determined based on the test set of the present stage that the trained deep learning model of the present stage meets the target requirements, the trained deep learning model of the present stage is output; by augmenting the false detection test samples, there is no need to manually adjust the training samples, which improves the richness of the training set and test set of the next stage while reducing the consumption of human resources and time. In addition, it is possible to quickly adapt to changes in samples during the production process, reduce the probability of false detection after the deep learning model is put online, and reduce the difficulty of training and iterating new models for on-site personnel.

[0141] In some embodiments, the false detection test samples include missed detection samples and over-detected samples; the second processing module 320 includes:

[0142] The detection submodule is used to obtain the number of missed detection samples and the number of passed detection samples in the test set of this stage;

[0143] When it is determined that the number of missed detections in this stage is not less than the target missed detection number, or the number of passed inspections in this stage is not less than the target passed inspection number, it is determined that the trained deep learning model in this stage does not meet the target requirements.

[0144] In some embodiments, the second processing module 320 is specifically used to:

[0145] Based on the preset offline augmentation strategy and / or the trained artificial intelligence generated content model, the false detection test samples in the test set of this stage are augmented to obtain augmented samples;

[0146] The offline augmentation strategy includes at least one of modifying the illumination parameters and background color of the falsely detected test sample and deforming the falsely detected test sample.

[0147] In some embodiments, the first processing module 310 is specifically used to train the initial deep learning model based on the early stopping strategy and the training set of this stage until the training stop condition of this stage is met; the training stop condition of this stage includes any one of the following:

[0148] The objective loss function of the initial deep learning model in this stage converges;

[0149] The training rounds of forward and backward propagation training of the initial deep learning model in this stage have reached the first target round in this stage.

[0150] In some embodiments, the training device of the deep learning model further includes a fifth processing module, which is used to update the first target round of the current stage when the training stop condition of the current stage is that the training rounds of the forward and backward propagation training of the initial deep learning model of the current stage reach the first target round of the current stage, and when it is determined that the training rounds of the current stage reach the first target round and the target loss function does not converge, to obtain the first target round of the next stage of training;

[0151] The first target round of training in the next stage is not less than the first target round of the current stage, but not greater than the second target round.

[0152] In some embodiments, the deep learning model training device further includes:

[0153] The sixth processing module is used for:

[0154] Obtain the initial sample set and training parameters input by the user; the initial sample set is the sample set of the first stage; the training parameters include the augmentation parameters corresponding to the offline augmentation strategy and the training stop conditions of each stage;

[0155] Configure the batch size of the initial deep learning model based on the maximum capacity of the current graphics card resources;

[0156] The network width and height of the initial deep learning model are configured according to the width and height of the maximum image of the sample images in the initial sample set.

[0157] In some embodiments, the third processing module 330 is specifically used to:

[0158] The training set, test set and augmented samples of this stage are used as the sample sets of the next stage;

[0159] The sample set of the next stage is divided into the training set and the test set of the next stage based on the target ratio.

[0160] The training device of the deep learning model in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0161] The training device of the deep learning model in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0162] The training of the deep learning model provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0163] In some embodiments, Figure 4 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, each process of the training method embodiment of the above-mentioned deep learning model is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0164] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0165] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned deep learning model training method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0166] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0167] An embodiment of the present application also provides a computer program product, including a computer program, which implements the training method of the above-mentioned deep learning model when executed by a processor.

[0168] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0169] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned deep learning model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0170] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0171] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0173] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0174] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0175] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A training method for a deep learning model, characterized in that: include: The initial deep learning model is trained based on the training set of this stage to obtain the trained deep learning model of this stage; When it is determined that the number of stages ending in this stage is less than the target number of stages, and it is determined based on the test set of this stage that the trained deep learning model of this stage does not meet the target requirements, augmenting the falsely detected test samples in the test set of this stage to obtain augmented samples; According to the training set, test set and the augmented samples of this stage, the training set and test set of the next stage are obtained; When it is determined that the number of stages up to this stage is not less than the target number of stages, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements, the trained deep learning model of this stage is output.

2. The deep learning model training method according to claim 1, characterized in that: The false detection test samples include missed detection samples and over-detected samples; The determining, based on the test set of the current stage, that the trained deep learning model of the current stage does not meet the target requirements includes: Obtain the number of missed samples and the number of passed samples in the test set of this stage; When it is determined that the number of missed detections in this stage is not less than the target missed detection number, or the number of passed inspections in this stage is not less than the target passed inspection number, it is determined that the trained deep learning model in this stage does not meet the target requirements.

3. The deep learning model training method according to claim 1, characterized in that: The augmenting process is performed on the falsely detected test samples in the test set of the present stage to obtain augmented samples, including: Based on a preset offline augmentation strategy and / or a trained artificial intelligence generated content model, augmenting the falsely detected test samples in the test set of the present stage to obtain augmented samples; The offline augmentation strategy includes at least one of modifying the illumination parameters and background color of the falsely detected test sample and deforming the falsely detected test sample.

4. The deep learning model training method according to claim 1, characterized in that: The initial deep learning model is trained based on the training set of this stage, including: The initial deep learning model is trained based on the early stopping strategy and the training set of this stage until the training stop condition of this stage is met; the training stop condition of this stage includes any one of the following; The target loss function of the initial deep learning model in this stage converges; The training rounds of forward and backward propagation training of the initial deep learning model in this stage reach the first target round of this stage.

5. The deep learning model training method according to claim 4, characterized in that: When the training stop condition of this stage is that the training rounds of the forward and backward propagation training of the initial deep learning model of this stage reach the first target round of this stage, after determining that the trained deep learning model of this stage does not meet the target requirement based on the test set of this stage, the method includes: When it is determined that the training round of the current stage reaches the first target round and the target loss function does not converge, the first target round of the current stage is updated to obtain the first target round of the next stage of training; The first target round of the next stage of training is not less than the first target round of the current stage, but not greater than the second target round.

6. The deep learning model training method according to claim 3, characterized in that: Before training the initial deep learning model based on the training set of this stage, the method further includes: Obtaining an initial sample set and training parameters input by a user; the initial sample set is a sample set of the first stage; the training parameters include augmentation parameters corresponding to the offline augmentation strategy and training stop conditions of each stage; Configuring a batch size of the initial deep learning model according to the maximum capacity of the current graphics card resources; The network width and height of the initial deep learning model are configured according to the width and height of the maximum image of the sample images in the initial sample set.

7. The deep learning model training method according to any one of claims 1 to 6, characterized in that: The step of obtaining the training set and the test set of the next stage according to the training set, the test set and the augmented samples of the current stage includes: The training set, test set and augmented samples of this stage are used as sample sets of the next stage; The sample set of the next stage is divided into a training set and a test set of the next stage based on the target ratio.

8. A training device for a deep learning model, characterized in that: include: The first processing module is used to train the initial deep learning model based on the training set of this stage to obtain the trained deep learning model of this stage; A second processing module is used to perform augmentation processing on the falsely detected test samples in the test set of the current stage to obtain augmented samples when it is determined that the stage number ending in the current stage is less than the target stage number and it is determined based on the test set of the current stage that the trained deep learning model of the current stage does not meet the target requirements; A third processing module is used to obtain a training set and a test set for the next stage according to the training set, the test set and the augmented samples of the current stage; The fourth processing module is used to output the trained deep learning model of this stage when it is determined that the number of stages up to this stage is not less than the target number of stages, or when it is determined based on the test set of this stage that the trained deep learning model of this stage meets the target requirements.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the training method of the deep learning model as described in any one of claims 1-7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the training method of the deep learning model as described in any one of claims 1 to 7.

Images