Model enhancement training method and system, storage medium and program product

Abstract

The invention discloses a model enhancement training method and system, a storage medium and a program product, and the method comprises the steps: obtaining a to-be-segmented image containing a wrong detection target and a missing detection target in a training set and a verification set; using an inward cutting mode to cut the error detection target and the missing detection target in the to-be-segmented image, and adding labels to the error detection target and the missing detection target; taking an image in the training set as a background image; at least one error detection target or missing detection target is synthesized with the background image after being subjected to random rotation, zooming and color change processing; distributing the obtained composite image to a new training set and a new verification set according to a preset proportion; and training and verifying the target model by using the new training set and the new verification set to obtain a target model training result. The method is suitable for model enhanced training, that is, further training is carried out by using an enhanced data set on the basis of a formed model, so that the performance of the model on a specific task is improved.

Description

Technical Field

[0001] The present application relates to the photovoltaic field, and more specifically, to a method, system, storage medium and program product for model enhancement training. Background Art

[0002] In deep learning detection and segmentation tasks, we often encounter some targets that are difficult to detect accurately due to unclear features, severe background interference, or small target size. This situation can easily lead to missed detections, that is, the model fails to recognize the actual target, or false detections, that is, the model mistakenly recognizes the background or other non-target objects as targets. Missed detections and false detections not only reduce the accuracy of the model, but may also bring serious consequences due to incorrect identification in actual tasks. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a method, system, storage medium and program product for model enhancement training; applicable to the enhanced training of the model, that is, further training using enhanced data sets based on the formed model to improve its performance on specific tasks. Specifically, the technical solution of the present application is as follows:

[0004] In a first aspect, the present application discloses a method for model enhancement training, comprising:

[0005] In the training set and the validation set, images to be segmented containing misdetected targets and missed targets are obtained;

[0006] Using an inward cutting method to cut the misdetected target and the missed target in the image to be segmented, and adding labels to the misdetected target and the missed target;

[0007] Using the image in the training set as a background image; synthesizing at least one of the misdetected targets or the missed targets with the background image after randomly rotating, scaling, and changing the color;

[0008] The obtained synthetic images are distributed to a new training set and a new verification set according to a preset ratio; the target model is trained and verified using the new training set and the new verification set to obtain the target model training result.

[0009] In some implementations, the method of cutting the mis-detected target and the missed-detected target in the image to be segmented by using an inward cutting method comprises the following steps:

[0010] Identify a number of cutting points marked in the image to be segmented, and connect adjacent cutting points to form a region to be segmented;

[0011] The edge of the area to be cut is shrunk inwardly by a preset width as a whole, and the area is cut using an image cutting tool.

[0012] In some embodiments, the model enhancement training method further includes: saving the target obtained after cutting in a preset file format; extracting the coordinates of the cut area of ​​the target after shrinkage to represent the outer contour of the target cut.

[0013] In some embodiments, the method of using the image in the training set as a background image and synthesizing at least one of the misdetected targets or the missed targets with the background image after random rotation, scaling, and color change processing comprises the following steps:

[0014] Acquire a plurality of targets, including the misdetected targets and the missed targets;

[0015] Performing random rotation, scaling, and color change processing on the target;

[0016] Placing the processed targets in the background image in sequence, with the targets being placed at random positions;

[0017] Determine whether the target at the first placement position overlaps with the target at the previous placement position; if so, re-place the target; if not, synthesize the target with the background image at the first placement position.

[0018] In some implementations, before synthesizing the target with the background image at the first placement position, the method further includes: removing image pixels in an area originally belonging to the first placement position in the background image.

[0019] In some embodiments, after the target and the background image are synthesized, in the area where the edge of the synthesized image does not match or the color is inconsistent, the original pixels in the area are removed, and image restoration and completion techniques are used to make the area smoother and more natural.

[0020] In some implementations, after the target and the background image are synthesized, the natural changes of the target model are simulated on the basis of the synthesized image, and the synthesized image is enhanced again.

[0021] In some implementations, the method of using the new training set and the new validation set to train and validate the target model to obtain the target model training result comprises the following steps:

[0022] Using the new training set to train the target model multiple times to obtain multiple model training results;

[0023] Verify the multiple model training results using the new verification set; select at least one of the model training results whose training effect meets the preset conditions;

[0024] The synthetic image is not assigned to a new test set, and the new test set is used to test the at least one model training result; and a final model is determined based on the test result.

[0025] In some embodiments, the preset ratio is different from the allocation ratio of each data set when the target model is first trained; and the new test set does not include the synthetic image.

[0026] In a second aspect, the present application also discloses a system for model enhancement training, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the model enhancement training method described in any one of the above embodiments.

[0027] In a third aspect, the present application also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model enhancement training method described in any of the above-mentioned embodiments.

[0028] In a fourth aspect, the present application also discloses a computer program product including a computer program, which, when executed by a processor, implements the steps of the model enhancement training method described in any of the above embodiments.

[0029] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0030] 1. This application is applicable to the enhanced training of the model, that is, further training is performed using an enhanced data set based on the established model to improve its performance on specific tasks. The data enhancement method proposed in this application can maximize the efficiency of data set expansion, and then efficiently obtain more pictures to form a new data set. For example, for the detection and recognition task of photovoltaic panel connectors, the use of a new data set to enhance the training model can improve its performance in identifying misdetected targets and missed targets without the need to collect new data.

[0031] 2. The data enhancement method in the present application uses an inward cutting method to cut the misdetected targets and missed targets in the original data set, and then the cut targets are synthesized with the background image after random rotation, scaling, and color change. The resulting synthetic image contains at least one target; unlike the existing data enhancement methods, the prior art only performs random rotation, scaling, and other processing on the entire background image, which easily overfits specific training samples. The data enhancement method in the present application can not only effectively expand the data set and increase the number of samples, but also reduce the demand for new sample collection and the cost of labeling a large amount of data. The synthetic image is based on inward cutting and splicing technology, which improves the model recognition effect to be more natural, helps the model learn more robust features, and reduces the risk of overfitting.

[0032] 3. The training method proposed in this application distributes the obtained synthetic images to the new training set and the new validation set according to a preset ratio, wherein the preset ratio is different from the distribution ratio of the data set when the model is first trained. In particular, the new test set does not contain synthetic images. During training, multiple trainings are usually performed, as well as new validation set screening and new test set testing. Because there are no synthetic images in the new test set, the verification of the model training results avoids the problem of overfitting, and it is easy to obtain ideal model training results in the end. By using different distribution ratios of the data set and screening of the model training results, the final model will effectively avoid the problem of overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The preferred implementation scheme will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present application.

[0034] Figure 1 A flowchart of the steps of an embodiment of a method for model enhancement training of the present application;

[0035] Figure 2 This is a flowchart of step S200 in another embodiment of a method for model enhancement training of the present application;

[0036] Figure 3 This is a flowchart of step S300 in another embodiment of a method for model enhancement training of the present application;

[0037] Figure 4 This is a flowchart of step S400 in another embodiment of a model enhancement training method of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0040] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0041] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0042] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0043] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, tutoring machines, or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the terminal device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0044] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0046] In deep learning detection and segmentation tasks, we often encounter some targets that are difficult to detect accurately due to unclear features, severe background interference, or small size. This situation can easily lead to missed detections, that is, the model fails to recognize the actual target, or false detections, that is, the model mistakenly recognizes the background or other non-target objects as targets. Missed detections and false detections not only affect the performance of the model, but may also have serious consequences in practical applications.

[0047] For example, in the field of photovoltaic inspection, since the connectors of photovoltaic panels are key components in the photovoltaic system, their status directly affects the performance and safety of the entire system. Taking the connector inspection project as an example, when the connector is blocked or in an environment with insufficient light or a huge difference in light and dark, some feature information such as texture and shape information is missing, making it difficult for traditional feature extraction methods to effectively capture enough details. As a result, the detection system is often difficult to detect accurately, and it is easy to miss or misdetect. The inability to accurately detect the status of the connector may cause system failures or maintenance delays.

[0048] Solving these object detection problems usually requires considering improving the model or retraining the model using a more comprehensive dataset.

[0049] The technology of this application chooses to repeatedly train the existing model without improving the model, but in actual operation, repeated training will encounter the following problems: Rebuilding the data set requires collecting a large number of images to be segmented, which is time-consuming and labor-intensive, and the directly taken pictures also need to be manually labeled, which is inefficient and cannot effectively improve the detection effect. If the currently available data enhancement technology is used, the original data set is rotated, translated, scaled, and other deformations are performed to expand the training data set. The model trained in this way will have the problem of overfitting. The overfitting phenomenon is that the model performs well on the trained data, but performs poorly on new and unseen data.

[0050] In response to the above technical problems, the present application discloses a method for model enhancement training, which is different from the current data enhancement technology and can expand the data volume based on the original data set; and the model trained using the data set expanded by the data enhancement method of the present application will not have the problem of overfitting, which can effectively improve the efficiency of data set expansion.

[0051] Reference Manual Attached Figure 1 An embodiment of a model enhancement training method provided in the present application includes the following steps:

[0052] S100, obtaining an image to be segmented containing misdetected targets and missed targets in a training set and a validation set.

[0053] This application is applicable to the intensive training of the model, that is, further training is performed on the basis of the formed model using an enhanced data set to improve its performance on a specific task. Specifically, the first training of the model is the process of learning the model from scratch. At this stage, the model learns how to perform a specific task through a large amount of data. And intensive training usually occurs after the first training. At this stage, the trained model will be further trained on the new data set to improve its performance on a specific task. The main difference between the two is that the first training is the basic ability to build a model, while intensive training is to optimize and adjust on the existing basis. In this application, the data set used for the first training includes: training set, verification set, and test set. The images therein often encounter some targets that are difficult to be accurately detected due to their unclear features, serious background interference, or too small target size, resulting in missed targets or wrongly detected targets. It is necessary to manually judge the images in the training set and the verification set to determine whether there are wrongly detected and missed targets. Determine the picture that meets the requirements, that is, automatically save the image to be segmented through the tool.

[0054] S200, cutting the mis-detected targets and the missed-detected targets in the image to be segmented by using an inward cutting method, and adding labels to the mis-detected targets and the missed-detected targets.

[0055] Specifically, use a cutting tool such as labelme or anylabeling, set a labeling tool such as a rectangle tool or a polygon tool, and manually or automatically label the area where the target is located in the segmented image.

[0056] For example, use the polygon tool to cut out targets that are difficult to identify. When cutting, use the "inward cutting" method, that is, do not cut the outermost edge, but cut a little bit inward. This is because there is often a small amount of external light on the outermost edge due to the actual lighting conditions at the scene. At this time, "inward cutting" can remove the influence of these small amounts of external light, allowing the model to learn the real detection target instead of learning the external light characteristics, bringing better detection results.

[0057] Labels are added to the misdetected targets and missed targets. For example, some tangled wire harnesses are sometimes mistakenly identified as connectors. In this case, these wire harnesses can be cut out and a "nothing" misdetected label is added. The misdetected labels are used as negative samples for model training, so that the training results are no longer based on the misdetected targets, which can effectively reduce the learning and prediction of the model for the wrong targets during the training process and improve the accuracy and robustness of the model.

[0058] S300, using the image in the training set as a background image; and synthesizing at least one of the misdetected targets or the missed targets with the background image after randomly rotating, scaling, and changing the color.

[0059] Specifically, random rotation, scaling, color change, etc. of images are common image enhancement methods in this field, but the data enhancement method in this application is different in that this application only performs random rotation, scaling, color change, etc. on the cut-out targets, and the background image is fixed. After the target is enhanced, it is synthesized into the background image. One target or multiple targets can be synthesized in one background image, which improves the efficiency of data set expansion. Reduce the need for new sample collection, thereby reducing the cost of collecting a large amount of labeled data.

[0060] S400, allocating the obtained synthetic images to a new training set and a new verification set according to a preset ratio; using the new training set and the new verification set to train and verify the target model to obtain the training result of the target model.

[0061] Specifically, the synthetic images contain a large number of misdetected and missed targets. Therefore, when allocating the data set, the synthetic images are allocated to the training set and the validation set according to the set ratio. This step does not involve the test set. The images in the test set are unlabeled images that have not participated in model training, so they can best avoid overfitting problems. The model finally selected should be the model with the best performance on the test set.

[0062] Another embodiment of a method for model enhancement training of the present application is based on an embodiment of the above method. Step S200 is referred to in the appendix of the specification. Figure 2 As shown, the specific steps include:

[0063] S210, identifying a number of cutting points marked in the image to be segmented, and connecting adjacent cutting points to form a region to be segmented.

[0064] S220, shrinking the edge of the area to be cut inward as a whole by a preset width, and cutting it using an image cutting tool.

[0065] Specifically, conventional automatic cutting tools may not be accurate enough in identifying image edges, especially when the contrast between the edge color and the background color of the image is not high, resulting in the cutting line failing to closely follow the image content, thereby leaving a background edge.

[0066] Our method is to continue cutting inwards based on the current automatic segmentation, that is, to cut off the background border around the target, or even cut off part of the target. This is because, for conventional cutting, the object and the background are in a normal relationship and no extra operations are required. However, we are synthesizing images. If we cut them normally, the background of the original image will be introduced, that is, the background edge around the target will be introduced. This background edge is completely impossible to exist in reality for our synthetic images. For our model, this is equivalent to introducing an incorrect pattern, which leads to lower efficiency of model learning and worse model. Therefore, it must be cut off.

[0067] The size of the preset width is set by those skilled in the art. The purpose of shrinking the preset width is to subtract the background edge of the cut target. The specific set value is not specifically limited in this application. The technical effect of inward cutting in this application can also be achieved by using the existing algorithm for removing the image border after cutting.

[0068] Preferably, based on the present embodiment, the present application further discloses another implementation mode, wherein step S220 further includes step: S230, saving the target obtained after cutting in a preset file format.

[0069] S240, extracting the coordinates of the cut region of the target after shrinkage to represent the outer contour of the cut target.

[0070] Specifically, the cutting result is saved in a json file. This is convenient for calling in the next step of target synthesis. The cutting points of the target after contraction are the multiple vertices of the polygon after cutting. Multiple points are used to represent the outer contour of the cutting, which is convenient for calculating the position of the target after rotation and scaling.

[0071] Another embodiment of a method for model enhancement training of the present application is as shown in the attached specification. Figure 3 As shown, based on an embodiment of the above method, step S300 refers to the attached specification. Figure 3 As shown, the specific steps include:

[0072] S310, acquiring a plurality of targets, wherein the targets include the misdetected targets and the missed detected targets.

[0073] S320, performing random rotation, scaling, and color change processing on the target.

[0074] Specifically, you need to set the color HSV data enhancement range first. Set the maximum and minimum values ​​for scaling. Set the maximum and minimum angles for rotation. Put the target into the background image, randomly rotate, scale, and translate it.

[0075] In one implementation of this embodiment, the method specifically includes: 1. Rotate first. When rotating and scaling, the size of the output image remains unchanged. Use the center point of the inserted target as the rotation origin. 2. Scaling. 3. Translation. 4. Synthesize the enhanced target with the background. Finally, limit the size of the output image to the size of the background image. Preferably, set the parameters of the inserted target. These parameters are used to randomly place the image somewhere in the background image, as well as to randomly scale and randomly rotate.

[0076] In another implementation of this embodiment, a matrix transformation-based method can calculate the position of the target after random rotation and scaling.

[0077] Specifically, first rotate and set the rotation center. Since the outer contour of the model is known in step S200, the position coordinates of the target after rotation or translation can be calculated by matrix operation by obtaining the rotation angle or translation position; rotation transformation is to rotate the figure around a fixed point (rotation center) at a certain angle. In two-dimensional space, rotation transformation can be represented by the following matrix:

[0078]

[0079] Where θ is the rotation angle. If you want to move the rotation center to the center of the figure or another point, you need to translate it first, rotate it, and then translate it back.

[0080] Scaling transformation: Scaling transformation is to change the size of the graphics. In two-dimensional space, scaling transformation can be represented by the following matrix:

[0081] Where Sx and Sy are the scaling factors in the x-axis and y-axis directions respectively.

[0082] For the objects cut in this application, each object is an irregular shape, which is represented as a set of points or vertices. Then, the above transformation matrix is ​​applied to each point to calculate the new position after transformation.

[0083] S330, placing the processed targets in the background image in sequence, with the targets being placed at random positions.

[0084] S340, determining whether the target at the first placement position overlaps with the target at the previous placement position.

[0085] S341, if there is overlap, reposition the first target until there is no overlap at the placement position.

[0086] S342: If there is no overlap, synthesize the target and the background image at the first placement position.

[0087] Specifically, after the target is synthesized into the background image, if it is found to overlap with other targets, this will make it difficult for the model to learn, and the target needs to be moved to a suitable position. This position is called the Region of Interest (ROI).

[0088] The appropriate position means that the objects do not overlap and are not moved outside the background image area. When placing randomly, check whether the currently placed object overlaps with the previously placed object. If it is judged that the object is placed in an inappropriate position, it will be re-placed, that is, it will be randomly moved to a new position and then checked for overlap. If it still overlaps, it will continue to move to the new position.

[0089] Preferably, the time for image synthesis can be set. If the synthesis is not successful within the preset time, the current synthesis is stopped. For example, the task requires synthesizing 10 targets in a background image. The targets are synthesized in sequence. If only 9 targets are synthesized within 30 seconds, and the 10th target still cannot find a suitable position, the synthesis of the 10th target is stopped, and the result of synthesizing 9 targets is retained. The next image is adjusted.

[0090] Another embodiment of the method of the present application, based on any one of the above embodiments, before synthesizing the first target with the background image at the first placement position in the step S342, it also includes: in the background image, removing the image pixels in the area originally belonging to the first placement position.

[0091] Specifically, when the synthesis is actually performed, the image pixels in the area of ​​the background image that originally belong to the placement position must be removed first. That is, the region of interest mentioned in the previous embodiment is removed, and then the enhanced target and the background image are synthesized. For example, the cv.add tool of OpenCV can be used for synthesis.

[0092] In the process of image synthesis, the original pixels at the synthesis position are removed to ensure that the synthesized image can be naturally integrated. If the original pixels are not removed, the newly added image elements may be mixed with the background pixels, resulting in color and brightness mismatch, thus affecting the model training effect. In the synthesis process, removing the original pixels can improve the quality of the synthesized image and make the synthesized image more natural.

[0093] Preferably, in one embodiment of the present application, after the target is synthesized with the background image, in the area where there is mismatch or inconsistent color at the edge of the synthesized image, the original pixels are removed in the area, and image restoration and completion techniques are used to make the area smoother and more natural.

[0094] Specifically, when synthesizing images, some defects or inconsistencies may appear, such as mismatched edges or inconsistent colors in overlapping areas. Removing the original pixels and using image restoration and completion techniques can remove these defects and make the synthesized image smoother and more natural. In practice, various image processing software and tools can be used to achieve this process. Through image synthesis technology, high-quality synthetic images can be created. Avoid the problem of model recognition errors caused by the background edges of the target in the synthetic image.

[0095] Preferably, in other embodiments, after the target is synthesized with the background image, the natural changes of the target model are simulated on the basis of the synthesized image, and the synthesized image is enhanced again. For example, the natural changes of the training data, such as lighting, angle, scale, etc., can be simulated on the basis of the synthesized image. This helps the model maintain stable performance when facing the diversity in the real world. Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0096] Another embodiment of the method of the present application, based on any one of the above embodiments, the step S400, refer to the attached specification Figure 4 Specifically including:

[0097] S410, distributing the obtained synthetic images to a new training set and a new verification set according to a preset ratio.

[0098] S420: Use the new training set to train the target model multiple times to obtain multiple model training results.

[0099] S430: Use the new verification set to verify the multiple model training results, and select at least one model training result whose training effect meets a preset condition.

[0100] S440, using a new test set to test the at least one model training result, and determining a final model according to the test result. The synthetic image is not assigned to the new test set.

[0101] Specifically, in the general deep learning dataset allocation method, the ratio of dataset allocation is fixed. For example, the collected images are allocated to the training set, validation set, and test set in a ratio of 3:1:1. However, if we allocate the synthetic images in this way, it will inevitably cause the problem of overfitting. Overfitting is a common problem in machine learning and statistical modeling. It refers to the phenomenon that the model performs well on the training data but performs poorly on new, unseen data. The reasons for overfitting usually include: the dataset for model training is insufficient, so that it learns the features in the training data but does not capture the true pattern behind the data.

[0102] Therefore, we use a new allocation and screening method: the proportion of data set allocation is different during the first training and enhanced training.

[0103] Specifically, the data set used for the first training includes: a training set, a validation set, and a test set. The new data set used for the intensive training includes a new training set, a new validation set, and a new test set. During the intensive training of the present application, the obtained synthetic images are distributed to the new training set and the new validation set according to a preset ratio. The preset ratio is different from the distribution ratio of the data set when the target model is first trained. Preferably. The synthetic image is not included in the new test set.

[0104] Let's take an example of this allocation method: suppose our initial data set has 5,000 images. When training the model for the first time, set the training set, validation set, and test set to be allocated in a ratio of 3:1:1. After the model training is completed, 200 images are synthesized for false positives and missed positives. Then these 200 images are allocated to the new training set and the new validation set in a ratio of 3:2 for enhanced model training. If the new test set does not account for a certain proportion or the proportion is 0, the synthesized image will not be included in the test set. The images of the new test set can be re-shot target images or test set images from the first training.

[0105] This application also involves the screening of model training results. Specifically, the first screening of the new validation set can screen out models with good image recognition effects on the training set, and save multiple models. It is assumed here that x models are saved.

[0106] The second screening with the new test set can get the best result in the actual scene. Because the screening results of the new validation set above must contain overfitting, the new test set should be used again to test x models. The model finally selected should be the one with the best effect on the new test set. Because the new test set does not contain synthetic images, it can best avoid overfitting.

[0107] Based on the same technical concept, the present application also discloses a system for model enhancement training, including a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the model enhancement training method described in any one of the above embodiments.

[0108] The present application discloses an embodiment of a system for model enhancement training, specifically comprising:

[0109] An acquisition module is used to obtain images to be segmented that contain misdetected targets and missed targets in a training set and a validation set;

[0110] A cutting module, used for cutting the mis-detected targets and the missed targets in the image to be segmented by using an inward cutting method, and adding labels to the mis-detected targets and the missed targets;

[0111] An enhanced synthesis module is used to use the image in the training set as a background image; synthesize at least one of the misdetected targets or the missed targets with the background image after random rotation, scaling, and color change processing;

[0112] The data set allocation module is used to allocate the obtained synthetic images to a new training set and a new verification set according to a preset ratio; use the new training set to train the model to obtain the target model training result.

[0113] Preferably, in another embodiment of a system for model enhancement training disclosed in the present application, it also includes: a model training module, which is used to train the model using the new training set to obtain a target model training result.

[0114] Based on the same technical concept, the present application also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the model enhancement training method described in any of the above embodiments are implemented.

[0115] Based on the same technical concept, the present application also discloses a computer program product including a computer program, which, when executed by a processor, implements the steps of the model enhancement training method described in any of the above embodiments.

[0116] The model enhancement training method, system, storage medium and program product of the present application have the same technical concept. The technical details of the above four embodiments are applicable to each other. In order to reduce repetition, they will not be repeated here.

[0117] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned program modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a processing unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software program units. In addition, the specific names of the program modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0118] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for model enhancement training, characterized in that: The steps include: In the training set and the validation set, images to be segmented containing misdetected targets and missed targets are obtained; Using an inward cutting method to cut the misdetected target and the missed target in the image to be segmented, and adding labels to the misdetected target and the missed target; Using the image in the training set as a background image; synthesizing at least one of the misdetected targets or the missed targets with the background image after randomly rotating, scaling, and changing the color; The obtained synthetic images are distributed to the new training set and the new validation set according to the preset ratio; The target model is trained and verified using the new training set and the new verification set to obtain a training result of the target model.

2. A method for model enhancement training as claimed in claim 1, characterized in that: The method of using an inward cutting method to cut the mis-detected target and the missed-detected target in the image to be segmented comprises the following steps: Identify a number of cutting points marked in the image to be segmented, and connect adjacent cutting points to form a region to be segmented; The edge of the area to be cut is shrunk inwardly by a preset width as a whole, and the area is cut using an image cutting tool.

3. A method for model enhancement training as claimed in claim 2, characterized in that: Also includes: Save the target obtained after cutting in a preset file format; The coordinates of the cut region of the target after shrinkage are extracted to represent the outer contour of the cut target.

4. A method for model enhancement training as claimed in claim 1, characterized in that: The method of using the image in the training set as the background image and synthesizing at least one of the misdetected targets or the missed targets with the background image after random rotation, scaling, and color change processing comprises the following steps: Acquire a plurality of targets, wherein the targets include the misdetected targets and the missed targets; Performing random rotation, scaling, and color change processing on the target; Placing the processed targets in the background image in sequence, with the targets being placed at random positions; Determining whether the target at the first placement position overlaps with the target at the previous placement position; If there is overlap, repositioning the target; If there is no overlap, the target is synthesized with the background image at the first placement position.

5. A method for model enhancement training as claimed in claim 4, characterized in that: Before synthesizing the target with the background image at the first placement position, the method further includes: removing image pixels in an area originally belonging to the first placement position in the background image.

6. A method for model enhancement training as claimed in claim 4, characterized in that: After the target and the background image are synthesized, in the area where the edge of the synthesized image does not match or the color is inconsistent, the original pixels in the area are removed, and image restoration and completion techniques are used to make the area smoother and more natural.

7. A method for model enhancement training as claimed in claim 4, characterized in that: After the target and the background image are synthesized, the natural changes of the target model are simulated on the basis of the synthesized image, and the synthesized image is enhanced again.

8. A method for model enhancement training as claimed in claim 1, characterized in that: The method of using the new training set and the new verification set to train and verify the target model to obtain the training result of the target model comprises the following steps: Using the new training set to train the target model multiple times to obtain multiple model training results; Verify the multiple model training results using the new verification set; select at least one of the model training results whose training effect meets the preset conditions; The synthetic image is not assigned to a new test set, and the at least one model training result is tested using the new test set; The final model is determined based on the test results.

9. A method for model enhancement training as claimed in claim 8, characterized in that: The preset ratio is different from the allocation ratio of each data set when the target model is trained for the first time; the new test set does not include the synthetic image.

10. A system for model enhancement training, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for model enhancement training described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for model enhancement training described in any one of claims 1 to 7 are implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for model enhancement training described in any one of claims 1 to 7 are implemented.

Images