Image detection model training method, computer device and storage medium

Abstract

The present application provides an image detection model training method, a computer device and a storage medium, wherein the method includes: obtaining an initial training sample; obtaining a labeled initial training sample; training a neural network to obtain an initial model; performing at least one iterative update on the initial model until an image detection model that meets preset requirements is obtained, wherein any one of the at least one iterative update includes: automatically marking new training samples with the model after the last iterative update; training a neural network to obtain a currently updated model; performing a quantitative evaluation on the currently updated model; determining whether the currently updated model meets the preset requirements; if the currently updated model has not yet met the preset requirements, performing the next iterative update; or if the currently updated model meets the preset requirements, using the currently updated model that meets the preset requirements as the image detection model. The present application can improve the efficiency of sample labeling, thereby improving the efficiency of model training.

Description

Technical Field

[0001] The present application relates to the field of image detection, and in particular to an image detection model training method, a computer device and a storage medium. Background Art

[0002] In the process of training deep learning models for object detection, it is usually necessary to perform multiple rounds of labeling and training on the objects in the images to be detected in order to achieve a relatively good model verification result. Moreover, the same object needs to be labeled repeatedly during the training process, which greatly increases the workload of labeling. In addition, the model verification results are also judged manually, and there is no quantitative detection standard that can be automatically identified. Summary of the invention

[0003] In view of the above content, it is necessary to provide an image detection model training method, a computer device and a storage medium, use an iterative method to train and generate a model, and then use the currently updated model to automatically mark the newly added training samples, thereby improving the efficiency of sample marking and thereby improving the efficiency of model training.

[0004] The image detection model training method includes: obtaining initial training samples, wherein the initial training samples include a preset number of initial images; obtaining labeled initial training samples; dividing the labeled initial training samples into an initial training set and an initial verification set; using the initial training set to train a neural network to generate an initial model; performing at least one iterative update on the initial model until an image detection model that meets preset requirements is obtained, wherein any one of the at least one iterative update includes: obtaining newly added training samples, wherein the newly added training samples include a preset number of newly added images; using the model updated after the previous iterative update to automatically label the newly added training samples to obtain labeled The invention relates to a method for detecting an image of an image in a neural network. The method comprises the following steps: adding a marked new training sample after the mark; dividing the marked new training sample into a new training set and a new verification set; training the neural network using the training set after the last iterative update and the new training set to obtain a currently updated model; performing a quantitative evaluation on the updated model based on the verification set after the last iterative update and the new verification set; determining whether the currently updated model meets the preset requirements based on the evaluation of the quantitative results of the currently updated model; if the currently updated model does not meet the preset requirements, performing the next iterative update on the currently updated model; or if the currently updated model meets the preset requirements, using the currently updated model as an image detection model.

[0005] Optionally, obtaining the labeled initial training sample includes: identifying a detection object in the initial image; and marking the detection object to obtain the label of the detection object.

[0006] Optionally, before acquiring the newly added training samples, the method further includes: before acquiring the newly added training samples, the method further includes: quantitatively evaluating the initial model based on the initial verification set, including: counting the mark A1 of the detection object of the initial image in the initial verification set, and obtaining the number of inspections a1 of the mark A1; using the initial model to identify the detection object of the initial image in the initial verification set, and using the initial model to mark the detection object of the initial image in the initial verification set, obtaining the mark A2 of the detection object of the initial image in the initial verification set, and obtaining the number of actual inspections a2 of the mark A2; evaluating the initial model The initial model is quantitatively evaluated and an evaluation result is obtained, including one or more evaluation methods: calculating the ratio of the actual inspection times a2 to the inspection times a1, calculating the average intersection over union (IoU) overlap of the mark A1 and the mark A2, and calculating the IoU variance of the mark A1 and the mark A2; determining whether the initial model meets the preset requirements according to the result of the quantitative evaluation; if the initial model meets the preset requirements, using the initial model as the image detection model; or if the initial model does not meet the preset requirements, performing at least one iterative update on the initial model until an image detection model that meets the preset requirements is obtained.

[0007] Optionally, the method further includes: automatically identifying the detection object in the newly added image using the model updated in the previous iteration; automatically marking the detection object in the newly added image using the model updated in the previous iteration to obtain the mark of the detection object in the newly added image.

[0008] Optionally, the method further includes: when there is any newly added image in the newly added training samples whose detection object is not marked, marking the detection object of the any newly added image to obtain the mark of the detection object of the any newly added image.

[0009] Optionally, the method further includes: counting the mark Z1 of the detection object in the image in the verification set updated in the last iteration to obtain the number of times z1 of the mark Z1 should be checked; counting the mark B1 of the detection object in the newly added image in the newly added verification set to obtain the number of times b1 of the mark B1 should be checked; generating a set C1 based on the mark Z1 and the mark B1; adding the number of times b1 of the mark B1 should be checked to the number of times z1 of the mark Z1 should be checked to obtain the number of times c1 should be checked.

[0010] Optionally, the quantitative evaluation of the currently updated model includes: using the currently updated model to identify the detection objects in the verification set updated in the last iteration and the newly added verification set; using the currently updated model to mark the detection objects in the verification set updated in the last iteration and the newly added verification set, to obtain the mark C2 of the detection objects in the verification set updated in the last iteration and the newly added verification set, and to count the actual inspection times c2 of the mark C2; calculating the ratio of the actual inspection times c2 to the expected inspection times c1; calculating the average IoU overlap of the marks in the set C1 and the marks C2; and calculating the IoU variance of the marks in the set C1 and the marks C2.

[0011] Optionally, the preset requirements include one or more of the following: the ratio of the actual inspection times c2 to the expected inspection times c1 is greater than a preset percentage threshold; the average IoU overlap is greater than a preset overlap threshold; and the IoU variance is less than a preset variance threshold.

[0012] The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, the image detection model training method is implemented.

[0013] The computer device includes a memory and at least one processor, wherein the memory stores at least one instruction, and when the at least one instruction is executed by the at least one processor, the image detection model training method is implemented.

[0014] Compared with the existing technology, the image detection model training method, computer device and storage medium use an iterative method to train and generate a model, and then use the currently updated model to automatically mark new training samples, thereby improving the efficiency of sample marking and thereby improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0016] Figure 1 It is a flowchart of the image detection model training method of the preferred embodiment of the present application.

[0017] Figure 2 It is a functional module diagram of the model training system of the preferred embodiment of the present application.

[0018] Figure 3It is an architectural diagram of a computer device of a preferred embodiment of the present application.

[0019] Main component symbols

[0020] Computer device 3 processor 32 Memory 31 Model training system 30 Get Module 301 Execution Module 302 Camera 33 monitor 34

[0021] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0022] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0025] See also Figure 1 As shown, it is a flowchart of the image detection model training method of the preferred embodiment of the present application.

[0026] In this embodiment, the image detection model training method can be applied to a computer device (eg Figure 2 In the computer device 3) shown in the figure, for the computer device that needs to perform image detection model training, the function for image detection model training provided by the method of the present application can be directly integrated on the computer device, or run on the computer device in the form of a software development kit (SDK).

[0027] like Figure 1 As shown, the image detection model training method specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed and some steps can be omitted.

[0028] Step S1: A computer device obtains initial training samples, wherein the initial training samples include a preset number of initial images.

[0029] In one embodiment, the initial image can be an image on a workbench obtained by a computer device using a camera. For example, when a worker assembles a computer host on a workbench, the camera captures a picture of the worker's hands and various computer host accessories, and the computer host accessories may include a battery, a Universal Serial Bus (USB) interface, a printed circuit board (PCB), etc. For example, the initial image can be an image of the worker installing a PCB board captured by the camera. In order to facilitate subsequent steps, the preset number of initial images can be a smaller number of initial images (for example, 200). In this embodiment, the initial image is a rectangular image.

[0030] In other embodiments, the computer device 3 may also use images obtained from other channels as initial training samples. For example, the computer device 3 may obtain the initial training samples by searching through an image search engine. In one embodiment, each initial image may include one or more objects, such as images containing pedestrians, trees, houses, and cars at the same time.

[0031] Step S2: The computer device obtains the labeled initial training samples.

[0032] In one embodiment, the user may mark the initial training sample, and the computer device obtains the marked initial training sample by: identifying the detection object in the initial image; and marking the detection object to obtain the label of the detection object.

[0033] In one embodiment, labeling the training samples means: using an image labeling tool (e.g., LabelImg) to identify each of the detection objects in each of the initial images in the initial training samples, and labeling each of the detection objects identified on the initial images using the image labeling tool, thereby obtaining a label for each of the detection objects. It should be noted that, since the number of the initial images in the initial training samples is small, each of the detection objects in each of the initial images in the initial training samples can be identified and labeled without omission.

[0034] In one embodiment, the marking includes: a marking graphic of a detection object obtained in the image, position information of the marking graphic in the image, and an identifier of the marking graphic.

[0035] In one embodiment, the marking graphic is a graphic used to mark the detection object, for example, it can be a rectangle.

[0036] In one embodiment, the position information of the marking graphic in the image can be represented by coordinates. For example, coordinate axes (e.g., x-axis and y-axis) are defined on the initial image to obtain the coordinates of the marking graphic in the initial image. In other embodiments, the position information can also be determined based on pixel information of the marking graphic in the initial image, which will not be described in detail here.

[0037] In one embodiment, the identifier of the marking graphic may be the name of the graphic. The identifier of the graphic is unique, so as to distinguish the detection objects of the same category. For example, if there are two PCB boards in the same initial image, the first PCB board may be identified as PCB_1 and the second PCB board may be identified as PCB_2 according to a preset order (for example, from top to bottom, from left to right, etc.). It should be noted that, for the convenience of example, the first PCB board is referred to as PCB1 below, and the corresponding identifier is PCB_1.

[0038] In one embodiment, the initial image containing the mark of the detection object is composed of the labeled initial training sample, each detection object in the initial training sample has a corresponding mark graphic, and each mark graphic has a corresponding logo.

[0039] Step S3: The computer device divides the labeled initial training samples into an initial training set and an initial verification set.

[0040] In one embodiment, the labeled initial training samples may be divided into the initial training set and the initial validation set in proportion. For example, 80% of the labeled initial training samples may be divided into the initial training set in a ratio of 8:2, and the remaining 20% ​​may be used as the initial validation set. For example, 160 of the 200 labeled initial training samples may be divided into the initial training set, and the remaining 40 may be used as the initial validation set.

[0041] Step S4: The computer device trains the neural network using the initial training set to generate an initial model.

[0042] In one embodiment, the neural network may be a convolutional neural network (CNN), and the computer device may train the CNN using the initial training set to generate the initial model. For example, the computer device may train the CNN using 160 labeled initial training samples in the initial training set to generate the initial model.

[0043] In one embodiment, the initial model can realize the function of detecting and marking the detection object in the initial image.

[0044] Step S5: The computer device performs quantitative evaluation on the initial model based on the initial validation set.

[0045] The following example describes the process of quantitatively evaluating the initial model. For the sake of clarity, please refer to the names of the terms and their corresponding definitions or descriptions shown in Table 1.

[0046]

[0047] Table 1

[0048] In one embodiment, the quantitative evaluation of the initial model includes: counting the mark A1 of the detection object of the initial image in the initial verification set, and obtaining the number of times a1 of the mark A1 should be inspected; using the initial model to identify the detection object of the initial image in the initial verification set, and using the initial model to mark the detection object of the initial image in the initial verification set, obtaining the mark A2 of the detection object of the initial image in the initial verification set, and obtaining the number of actual inspections a2 of the mark A2; quantitatively evaluating the initial model and obtaining an evaluation result, including one or more evaluation methods: calculating the ratio of the actual inspection number a2 to the number of inspections a1, calculating the average intersection over union (IoU) overlap of the mark A1 and the mark A2, and calculating the IoU variance of the mark A1 and the mark A2.

[0049] In one embodiment, the mark A1 of the detection object of the initial image in the initial verification set includes: the mark graphic mA1 of the detection object of the initial image in the initial verification set, the position information of the mark graphic mA1 in the initial image, and the identifier of the mark graphic mA1.

[0050] In one embodiment, the number of times a1 of the marker A1 to be checked is the number of the marker A1. Since the initial verification set is obtained by manual marking and the number is small, the markers A1 of the detection objects of the initial images in the initial verification set can be manually counted. For example, the marker graphic mA1 of PCB1 in the initial verification set is counted, and the number of occurrences of the marker graphic mA1 of PCB1 is obtained to be 25 times, that is, the number of times a1 of the marker A1 to be checked is 25.

[0051] In one embodiment, the mark A2 of the detection object of the initial image in the initial verification set includes: the mark graphic mA2 of the detection object obtained by using the initial model, the position information of the mark graphic mA2 in the initial image, and the identifier of the mark graphic mA2. It should be noted that for the same detection object on the same initial image in the initial verification set, the identifier of the mark graphic mA2 is consistent with the identifier of the mark graphic mA1. For example, in the same initial image in the initial verification set, the identifier of the mark graphic mA1 of PCB1 is PCB_1, and the identifier of the mark graphic mA2 of PCB1 is also PCB_1.

[0052] In one embodiment, the computer device uses the initial model to perform statistics on the mark A2 to obtain the number of actual inspections a2 of the mark A2. For example, the mark graphic mA2 of PCB1 in the initial verification set can be counted to obtain 20 occurrences of the mark graphic mA2 of PCB1, that is, the number of actual inspections a2 of the mark A2 is 20; the identifier PCB_1 of the mark A2 of PCB1 in the initial verification set can also be counted to obtain the number of actual inspections a2 of the mark A2 is 20. The ratio of the number of actual inspections 20 of PCB1 to the number of inspections 25 is 0.8, that is, 80%.

[0053] In one embodiment, the evaluation of the average IoU overlap of the marker A1 and the marker A2 includes: for the same initial image in the initial verification set, calculating the IoU overlap of the marker graphic mA1 and the corresponding marker graphic mA2 (the marker graphic mA1 and the marker graphic mA2 have the same identifier), and obtaining a2 IoU overlaps in total; calculating the average of the a2 IoU overlaps to obtain the average IoU overlap of the marker A1 and the marker A2. For example, the number of actual inspections of PCB1 is 20, and the IoU overlap of the marker graphic mA1 and the marker graphic mA2 of PCB1 with the same identifier PCB_1 on the same initial image in the initial verification set is calculated to obtain 20 IoU overlaps, and the average of the 20 IoU overlaps is calculated to obtain the average IoU overlap.

[0054] In one embodiment, for the same initial image in the initial verification set, calculating the IoU overlap of the marker graphic mA1 and the corresponding marker graphic mA2 includes: calculating the intersection I of the marker graphic mA1 and the corresponding marker graphic mA2; calculating the union U of the marker graphic mA1 and the corresponding marker graphic mA2; calculating the ratio of the intersection I to the union U to obtain the IoU overlap of the marker A1 and the corresponding marker A2. For example, the coordinates (x_11, y_11) of the lower left vertex and the coordinates (x_12, y_12) of the upper right vertex of the marker graphic mA1 in the corresponding initial image can be obtained; as well as the coordinates (x_21, y_21) of the lower left vertex and the coordinates (x_22, y_22) of the upper right vertex of the marker graphic mA2 in the corresponding initial image can be obtained. The IoU overlap of the marker graphic mA1 and the marker graphic mA2 is calculated based on the coordinate information using Python coding, that is, the IoU overlap of the marker A1 and the marker A2. The specific process will not be repeated here.

[0055] In one embodiment, calculating the IoU variance of the marker A1 and the marker A2 includes: calculating the variance of the a2 IoU overlap values.

[0056] Step S6, the computer device determines whether the initial model meets the preset requirements based on the quantitative evaluation results of the initial model; if the initial model meets the preset requirements, execute step S13; if the initial model does not meet the preset requirements, execute step S7.

[0057] In one embodiment, the preset requirements include one or more of the following: the ratio of the actual inspection times to the expected inspection times is greater than a preset percentage threshold; the average IoU overlap is greater than a preset overlap threshold; and the IoU variance is less than a preset variance threshold. It should be noted that when all the conditions of the preset requirements are met at the same time, the obtained model is the best model.

[0058] In one embodiment, the threshold value can be set according to the standard in the actual production requirements. For example, the preset percentage threshold value can be 80%, the preset overlap threshold value can be 0.6, and the preset variance threshold value can be 0.8. It should be noted that since the number of the labeled initial training samples in the initial training set in step S4 is small, the evaluation result obtained by quantitatively evaluating the initial model in step S5 generally does not meet the preset requirements, so it can be determined without judgment that the initial model does not meet the preset requirements; steps S5 and S6 are not necessary steps.

[0059] In one embodiment, the computer device determines whether the initial model meets the preset requirements based on the quantitative evaluation of the initial model, which may include one or more of the following: determining whether the ratio of the actual inspection times a2 to the expected inspection times a1 is greater than 80%; determining whether the average IoU overlap of the mark A1 and the mark A2 is greater than 0.6; determining whether the IoU variance of the mark A1 and the mark A2 is less than 0.8. When it is determined that the initial model does not meet the preset result, execute step S7.

[0060] Step S7: The computer device obtains new training samples, where the new training samples include a preset number of new images.

[0061] In one embodiment, the computer device may obtain the newly added images using the same method as that used to obtain the initial images; the preset number of newly added images may be 100 or more or less newly added images.

[0062] Step S8: The computer device automatically labels the newly added training samples using the model updated in the previous iteration to obtain labeled newly added training samples.

[0063] The following example describes how to use the model updated in the last iteration to automatically label the newly added training samples to obtain the labeled newly added training samples. For the sake of clarity, please refer to the names of the terms and their corresponding definitions or descriptions shown in Table 1 above and Table 2 below.

[0064] Term Name Definition or description Mark B1 Added markers for detection objects in newly added images in the validation set Marking graphic mB1 Marking graphic for Mark B1 Number of inspections b1 Mark the number of B1 Mark Z1 Labels of detected objects in the images in the validation set after the last iteration update Number of inspections to be performed z1 The number of markers Z1 Set C1 The set consisting of marker B1 and marker Z1 Number of inspections to be performed c1 The number of tags in set C1, that is, the sum of the number of times b1 should be checked and the number of times z1 should be checked Mark the graphic mC1 Labeled graphics in set C1

[0065] Table 2

[0066] In one embodiment, the model updated in the last iteration refers to the model obtained in the most recent training. For example, in the first iteration, the model updated in the last iteration is the initial model.

[0067] In one embodiment, the automatic labeling of the newly added training samples to obtain the labeled newly added training samples includes: automatically identifying the detection objects in the newly added images using the model updated in the previous iteration; and automatically labeling the detection objects in the newly added images using the model updated in the previous iteration to obtain the labels of the detection objects in the newly added images. For example, in the first iteration, the model updated in the previous iteration is the initial model: automatically identifying the detection objects in the newly added images using the initial model; and automatically labeling the detection objects in the newly added images using the initial model to obtain the labels of the detection objects in the newly added images.

[0068] In one embodiment, the mark of the detection object in the newly added image includes: a mark graphic of the detection object in the newly added image, position information of the mark graphic in the newly added image, and an identifier of the mark graphic.

[0069] In one embodiment, the obtaining of the label of the detection object in the newly added image further includes: manually checking and supplementing the label of the newly added training samples, and when there is a detection object in any newly added image in the newly added training samples that is not marked, marking the detection object in the any newly added image, and obtaining the label of the detection object in the any newly added image. It should be noted that after the newly added training samples are manually checked and supplemented with labeling, there is no omission in the labeling of the detection object in the newly added training samples, that is, each of the detection objects has a corresponding labeling graphic, and each labeling graphic has a corresponding identifier.

[0070] In one embodiment, in order to facilitate the use of subsequent steps, when the newly added images in the newly added training samples are manually checked and supplemented with marks, the marking graphic mB1 of the marking B1 of the detection object of the newly added images in the newly added verification set may be counted to obtain the number of times b1 of the marking B1 that should be checked. For example, the marking graphic mB1 of PCB1 in the newly added verification set may be counted to obtain 15 occurrences of the marking graphic mB1 of PCB1, that is, the number of times B1 of the marking B1 that should be checked is 15.

[0071] In one embodiment, the mark Z1 of the detection object of the image in the verification set after the last iteration update is counted to obtain the number of times z1 of the mark Z1 should be checked; the mark B1 of the detection object of the newly added image in the newly added verification set is counted to obtain the number of times b1 of the mark B1 should be checked; based on the mark Z1 and the mark B1, a set C1 is generated; the number of times b1 of the mark B1 should be checked is added to the number of times z1 of the mark Z1 should be checked to obtain the number of times c1 should be checked. For example, in the first iteration, the verification set after the last iteration update is the initial verification set: the set C1 of the mark A1 in the initial verification set and the mark B1 in the newly added verification set can be obtained, and the mark graph mA1 and the mark graph mB1 in the set C1 can be uniformly recorded as the mark graph mC1. For example, the mark A1 of PCB1 in the initial verification set and the mark B1 of PCB1 in the newly added verification set can be obtained to obtain the set C1 of the mark A1 and the mark B1 of PCB1.

[0072] In one embodiment, the number of times b1 of the marker B1 in the newly added verification set should be checked can be added to the number of times z1 of the marker Z1 in the verification set updated in the last iteration by a computer device to obtain the number of times c1 should be checked. For example, in the first iteration, the number of times b1 (i.e., 15) of the marker graphic mB1 of PCB1 in the newly added verification set should be checked can be added to the number of times a1 (i.e., 25) of the marker graphic mA1 of PCB1 in the initial verification set to obtain the number of times c1 (i.e., 40) of PCB1 should be checked.

[0073] Step S9: The computer device divides the labeled newly added training samples into a newly added training set and a newly added verification set.

[0074] In one embodiment, the labeled newly added training samples can be divided into the newly added training set and the newly added verification set in proportion. For example, 80% of the labeled newly added training samples can be divided into the newly added training set in a ratio of 8:2, and the remaining 20% ​​can be used as the newly added verification set. For example, 80 of 100 labeled newly added training samples can be divided into the newly added training set, and the remaining 20 can be used as the newly added verification set.

[0075] Step S10: The computer device trains the neural network using the training set updated in the last iteration and the newly added training set to obtain a currently updated model.

[0076] In one embodiment, the computer device may train the CNN using the training set updated in the last iteration and the set consisting of the newly added training set to obtain the currently updated model. For example, in the first iteration, the training set updated in the last iteration is the initial training set, and the computer device may train the CNN using a set consisting of 160 labeled initial training samples in the initial training set and 80 labeled newly added training samples in the newly added training set, for a total of 240 labeled training samples, to obtain the currently updated model.

[0077] Step S11: The computer device performs quantitative evaluation on the currently updated model based on the validation set after the last iterative update and the newly added validation set.

[0078] The following example describes the process of quantitatively evaluating the currently updated model based on the validation set after the last iterative update and the newly added validation set. For the sake of clarity, please refer to the names of the terms and their corresponding definitions or descriptions shown in Tables 1 to 3.

[0079]

[0080] Table 3

[0081] In one embodiment, the quantitative evaluation of the currently updated model includes: using the currently updated model to identify the detection objects in the verification set updated in the last iteration and the newly added verification set; using the currently updated model to mark the detection objects in the verification set updated in the last iteration and the newly added verification set, to obtain the mark C2 of the detection object in the verification set updated in the last iteration and the newly added verification set, and to count the actual inspection times c2 of the mark C2; calculating the ratio of the actual inspection times c2 to the expected inspection times c1; calculating the average IoU overlap of the marks in the set C1 and the marks C2; and calculating the IoU variance of the marks in the set C1 and the marks C2.

[0082] In one embodiment, the mark C2 of the detection object in the verification set after the last iteration update and the newly added verification set includes: the mark graphic mC2 of the detection object obtained by the computer device using the currently updated model, the position information of the mark graphic mC2 in the image in the verification set after the last iteration update or the newly added image, and the identifier of the mark graphic mC2. It should be noted that for the same detection object in the same image in the verification set after the last iteration update or the same detection object on the same newly added image in the newly added verification set, the identifier of the mark graphic mC2 is consistent with the identifier of the mark graphic mC1. For example, in the same newly added image in the newly added verification set, the identifier of the mark graphic mC1 of PCB1 is PCB_1, and the identifier of the mark graphic mC2 of PCB1 is also PCB_1.

[0083] In one embodiment, the computer device uses the currently updated model to count the mark C2 to obtain the number of actual inspections c2 of the mark C2. For example, in the first iteration, the verification set after the previous iteration update is the initial verification set, and the mark graph mC2 of PCB1 in the initial verification set and the newly added verification set can be counted to obtain the number of appearances of the mark graph mC2 of PCB1 30 times, that is, the number of actual inspections c2 of the mark C2 is 30. The ratio of the number of actual inspections of PCB 30 to the number of inspections 40 is 0.75, that is, 75%.

[0084] In one embodiment, the calculation of the average IoU overlap between the markers in the set C1 and the marker C2 includes: for the IoU overlap between the marker graphic mC1 and the corresponding marker graphic mC2 on the same image in the verification set updated in the last iteration (the marker graphic mC1 and the corresponding marker graphic mC2 have the same logo), calculating the IoU overlap between the marker graphic mC1 and the marker graphic mC2 with the same logo on the same newly added image in the newly added verification set, and obtaining a total of c2 IoU overlaps; calculating the average of the c2 IoU overlaps to obtain the average IoU overlap between the markers in the set C1 and the marker C2.

[0085] In one embodiment, the method for calculating the IoU overlap between the markers in the set C1 and the marker C2 is the same as the method for calculating the IoU overlap between the marker A1 and the marker A2 in step S5, and will not be repeated here.

[0086] In one embodiment, calculating the IoU variance of the markers in the set C1 and the marker C2 includes: calculating the variance of the c2 IoU overlaps.

[0087] Step S12, the computer device determines whether the currently updated model meets the preset requirements based on the quantitative evaluation results of the currently updated model. If the currently updated model meets the preset requirements, execute step S13; if the currently updated model does not meet the preset requirements, execute S7.

[0088] In one embodiment, the preset requirements may include one or more of the following: the ratio of the actual inspection times c2 to the inspection times c1 is greater than 80%; the average IoU overlap between the markers in the set C1 and the marker C2 is greater than 0.6; the IoU variance between the markers in the set C1 and the marker C2 is less than 0.8. When one or more of these conditions are met, the currently updated model reaches the preset result, and step S13 is executed. It should be noted that when all the conditions of the preset requirements are met at the same time, the obtained model is the best model.

[0089] It should be noted that when the currently updated model does not meet the preset requirements and the process steps are executed again to step S7, the computer device obtains other images as training samples and uses the other images obtained as new training samples.

[0090] In addition, it should be noted that when the currently updated model does not meet the preset requirements and the process steps are executed again to step S8, the model updated in the previous iteration refers to the currently updated model obtained in step S10 in the previous iteration. For example, in the second iteration, the model updated in the previous iteration in step S8 refers to the currently updated model obtained in step S10 in the first iteration.

[0091] Step S13: The computer device uses the currently updated model as an image detection model.

[0092] It should be noted that, when the initial model in step S6 meets the preset requirements, the currently updated model, ie, the initial model, will not be iterated again.

[0093] Above Figure 1 The image detection model training method of this application is introduced in detail. Figure 3 , the hardware device architecture for implementing the image detection model training method is introduced.

[0094] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0095] See also Figure 2 The figure shows a module diagram of a model training system provided in a preferred embodiment of the present application.

[0096] In some embodiments, the model training system 30 is run in a computer device 3. The model training system 30 may include a plurality of functional modules composed of program code segments. The program code of each program segment in the model training system 30 may be stored in a memory 31 of the computer device 3 and executed by at least one processor 32 to implement the function of training the model (see Figure 1 describe).

[0097] In this embodiment, the model training system 30 can be divided into multiple functional modules according to the functions it performs. The functional modules may include: an acquisition module 301 and an execution module 302. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module have been described in detail in the aforementioned embodiments.

[0098] Specifically, the acquisition module 301 can be used to obtain initial training samples and newly added training samples. The execution module 302 can be used to divide the marked initial training samples into an initial training set and an initial verification set; use the initial training set to train the neural network to generate an initial model; perform at least one iteration update on the initial model until an image detection model that meets the preset requirements is obtained, wherein any iteration update in the at least one iteration update includes: using the model updated in the last iteration to automatically mark the newly added training samples to obtain the marked newly added training samples; divide the marked newly added training samples into a newly added training set and a newly added verification set; use the training set updated in the last iteration and the newly added training set to train the neural network to obtain the currently updated model; perform quantitative evaluation on the updated model based on the verification set updated in the last iteration and the newly added verification set; determine whether the currently updated model meets the preset requirements based on the evaluation of the quantitative results of the currently updated model; if the currently updated model does not meet the preset requirements, perform the next iteration update on the currently updated model; or if the currently updated model meets the preset requirements, use the currently updated model as the image detection model.

[0099] See also Figure 3 As shown, it is a schematic diagram of the structure of a computer device provided in a preferred embodiment of the present application. In a preferred embodiment of the present application, the computer device 3 includes a memory 31, at least one processor 32, a camera 33, and a display 34. Those skilled in the art should understand that Figure 3 The structure of the computer device shown does not constitute a limitation of the embodiments of the present application, and can be either a bus structure or a star structure. The computer device 3 can also include more or less other hardware or software than shown in the figure, or a different component arrangement.

[0100] In some embodiments, the computer device 3 includes a terminal that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices.

[0101] It should be noted that the computer device 3 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

[0102] In some embodiments, the memory 31 is used to store program codes and various data. For example, the memory 31 can store the model training system 30 installed in the computer device 3, and can also be used to store pictures taken by the camera 33, and realize high-speed and automatic access to programs or data during the operation of the computer device 3. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable storage medium that can be used to carry or store data.

[0103] In some embodiments, the at least one processor 32 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The at least one processor 32 is the control core (Control Unit) of the computer device 3, and uses various interfaces and lines to connect various components of the entire computer device 3, and executes or executes programs or modules stored in the memory 31, and calls data stored in the memory 31 to execute various functions of the computer device 3 and process data, such as executing Figure 1 The image detection model training features shown.

[0104] Although not shown, the computer device 3 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 32 through a power management device, so that the power management device can manage charging, discharging, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 3 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0105] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0106] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a server, a personal computer, etc.) or a processor to execute a part of the method described in each embodiment of the present application.

[0107] In a further embodiment, in combination Figure 3 The at least one processor 32 can execute the operating device of the computer device 3 and various installed application programs (such as the model training system 30), program codes, etc., for example, the above-mentioned modules.

[0108] The memory 31 stores program codes, and the at least one processor 32 can call the program codes stored in the memory 31 to execute related functions. The program codes stored in the memory 31 can be executed by the at least one processor 32, thereby realizing the functions of the various modules to achieve Figure 1 The purpose of training the image detection model shown.

[0109] In one embodiment of the present application, the memory 31 stores one or more instructions (ie, at least one instruction), and the at least one instruction is executed by the at least one processor 32 to implement Figure 1 The purpose of training the image detection model shown.

[0110] In one embodiment of the present application, the camera 33 can be a charge coupled device (CCD) camera installed on the workbench as an image sensor, and the camera 33 only needs to be able to capture the entire workbench and the image thereon.

[0111] In one embodiment of the present application, the display 34 may be a touch-screen display device capable of presenting images, such as presenting images captured by the camera 33 and presenting an operation page of an image marking tool.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0113] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0115] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the above preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A method for training an image detection model, characterized in that: The method comprises: Obtaining an initial training sample, wherein the initial training sample includes a preset number of initial images; Obtain labeled initial training samples; Dividing the labeled initial training samples into an initial training set and an initial validation set; Using the initial training set to train a neural network to generate an initial model; The initial model is iteratively updated at least once until an image detection model that meets preset requirements is obtained, wherein any one of the at least one iterative updates includes: Acquire new training samples, where the new training samples include a preset number of new images; Using the model updated in the previous iteration, automatically label the newly added training samples to obtain labeled newly added training samples; Dividing the labeled newly added training samples into a newly added training set and a newly added verification set; Using the training set updated in the last iteration and the newly added training set to train the neural network to obtain a currently updated model; Performing a quantitative evaluation on the currently updated model based on the validation set updated in the last iteration and the newly added validation set; Determining whether the currently updated model meets preset requirements according to a quantitative evaluation result of the currently updated model; If the currently updated model has not yet met the preset requirement, performing the next iterative update on the currently updated model; or If the currently updated model meets the preset requirement, use the currently updated model as the image detection model; The method further comprises: Automatically identifying the detection object in the newly added image using the model updated in the previous iteration; Automatically marking the detection object in the newly added image by using the model updated in the previous iteration to obtain the mark of the detection object in the newly added image; When there is a detection object of any newly added image in the newly added training samples that is not marked, marking the detection object of the any newly added image to obtain a mark of the detection object of the any newly added image; Count the marks Z1 of the detection objects in the images in the verification set after the last iteration update, and obtain the number of times z1 of the marks Z1 that should be checked; Counting the marks B1 of the detection objects of the newly added images in the newly added verification set, and obtaining the number of times b1 of the marks B1 to be detected; Based on the mark Z1 and the mark B1, a set C1 is generated; The number of times b1 of the mark B1 to be checked is added to the number of times z1 of the mark Z1 to be checked, so as to obtain the number of times c1 to be checked.

2. The image detection model training method according to claim 1, characterized in that: The initial training samples after obtaining the label include: Identifying a detection object in the initial image; and The detection object is marked to obtain a mark of the detection object.

3. The image detection model training method according to claim 2, characterized in that: The method further comprises, before acquiring the newly added training sample: The initial model is quantitatively evaluated based on the initial validation set, including: Counting the marks A1 of the detection objects in the initial images in the initial verification set, and obtaining the number of times a1 of the marks A1 should be detected; Using the initial model to identify the detection object of the initial image in the initial verification set, and using the initial model to mark the detection object of the initial image in the initial verification set, to obtain a mark A2 of the detection object of the initial image in the initial verification set, and to obtain a number of actual inspections a2 of the mark A2; The initial model is quantitatively evaluated and an evaluation result is obtained, including one or more evaluation methods: calculating the ratio of the actual inspection number a2 to the inspection number a1, calculating the average intersection over union (IoU) overlap of the marker A1 and the marker A2, and calculating the IoU variance of the marker A1 and the marker A2; Determining whether the initial model meets the preset requirements according to the result of the quantitative evaluation; If the initial model meets the preset requirements, use the initial model as the image detection model; or If the initial model does not meet the preset requirements, perform at least one iterative update on the initial model until an image detection model that meets the preset requirements is obtained.

4. The image detection model training method according to claim 1, characterized in that: The quantitative evaluation of the currently updated model includes: Using the currently updated model to identify the detection objects in the verification set updated in the last iteration and the newly added verification set; Using the currently updated model, mark the detection objects in the verification set updated in the last iteration and the newly added verification set, obtain the mark C2 of the detection objects in the verification set updated in the last iteration and the newly added verification set, and count the number of actual inspections c2 of the mark C2; Calculate the ratio of the actual inspection number c2 to the required inspection number c1; Calculate the average IoU overlap between the markers in the set C1 and the marker C2; Calculate the IoU variance between the tags in the set C1 and the tags C2.

5. The image detection model training method according to claim 4, characterized in that: The preset requirements include one or more of the following: The ratio of the actual inspection times c2 to the required inspection times c1 is greater than a preset percentage threshold; The average IoU overlap is greater than a preset overlap threshold; and The IoU variance is less than a preset variance threshold.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, it implements the image detection model training method as described in any one of claims 1 to 5.

7. A computer device, characterized in that: The computer device includes a memory and at least one processor, wherein the memory stores at least one instruction, and when the at least one instruction is executed by the at least one processor, the image detection model training method as claimed in any one of claims 1 to 5 is implemented.

Images