A sampling method and storage medium for industrial image target detection image training set

Abstract

The present invention discloses a method and storage medium for collecting an image training set for industrial image target detection. The method includes collecting targets of each image to be trained, annotating the image to be trained, annotating the image to be trained to form n rectangular image blocks, and forming a coordinate set Rect; obtaining the coordinates of a cut-off sub-image on the image to be trained; excluding the coordinates of the rectangular image blocks in the coordinate set Rect that are not within the coordinate range of the cut-off sub-image, and correcting the coordinate values ​​of the remaining rectangular image blocks in the coordinate set Rect to obtain a new coordinate set Rect'; obtaining a sampled image, and cutting the image to be trained with the coordinates of the cut-off sub-image to form a sampled image, wherein the coordinates of the rectangular image blocks in the sampled image are corrected according to the coordinate set Rect'. The original image set can be annotated only once, and segmentation position changes can be continuously generated, and the proportion of pure background sub-images and sub-images with targets can be automatically controlled through parameters.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a sampling method, device and storage medium for an industrial image target detection image training set. Background Art

[0002] In industrial production, the industrial images collected by computers are generally huge in size. When using deep learning target detection task models, it is often necessary to segment the collected industrial images into sub-images and detect the sub-images one by one. However, in the deep learning model training stage of the target detection task, it is not appropriate to directly use the fixed segmentation method to produce the training set. Effective targets on industrial images, such as appearance defects and product defects, have a low probability of occurrence and occupy a small area. During the segmentation process, most of the sub-images of a complete industrial image are pure background sub-image areas without targets, and the target sub-images containing targets account for a very small proportion, and these target sub-images are precisely the samples with the greatest information value for the training model.

[0003] There are usually two methods for sampling the existing industrial image target detection image training set: the first is to fix the industrial image segmentation first, then label it, and balance the ratio of pure background sub-images and target sub-images by manual selection. In this method, although each target is equivalent to being labeled only once, the target sub-image containing the target is fixed and unchanged, and the samples lack diversity in position space. The second method is to use random sampling segmentation and then labeling. In this method, although the diversity of the target sub-image containing the target is increased, the same target on the original image before segmentation must be labeled multiple times, which increases the cost of manual labeling. Summary of the invention

[0004] In order to overcome the above shortcomings, the purpose of the present invention is to provide a sampling method for industrial image target detection image training set, which can not only annotate the original image set only once, but also continuously produce segmentation position changes, and can automatically control the category ratio of pure background sub-image and sub-image with target through parameters.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: a method for collecting an image training set for industrial image target detection, characterized in that: it is used to collect targets for each image to be trained in the training set, and the target collection of each image to be trained includes the following steps:

[0006] S1. Annotate the training image to form n rectangular image blocks, record the coordinates of each rectangular image block, and form a coordinate set Rect, where n is an integer greater than or equal to 0.

[0007] S2, obtaining the coordinates of the intercepted sub-image on the image to be trained, randomly sampling on the image to be trained to form a rectangular intercepted sub-image, wherein the intercepted sub-image is a background sub-image without a target object or an object sub-image containing a target object, and the interception ratio of the background sub-image and the object sub-image is adjustable, and the coordinates of the intercepted sub-image on the image to be trained are within the range of the image to be trained;

[0008] S3, excluding the coordinates of the rectangular image blocks in the coordinate set Rect that are not within the coordinate range of the intercepted sub-image, and correcting the coordinate values ​​of the remaining rectangular image blocks in the coordinate set Rect to obtain a new coordinate set Rect';

[0009] S4, obtaining a sampling image, and using the coordinates of the intercepted sub-image to intercept the image to be trained to form a sampling image, wherein the coordinates of the rectangular image block in the sampling image are corrected according to the coordinate set Rect'.

[0010] Specifically, the coordinates of the rectangular image block are (x1, y1, x2, y2), where (x1, y1) and (x2, y2) are the coordinate values ​​of the first vertex and the second vertex of the diagonally opposite corners of the rectangular image block. The coordinate set Rect = {(x1, y1, x2, y2) 1 ,(x1,y1,x2,y2) 2 ,……(x1,y1,x2,y2) n}.

[0011] Further, the S2 specifically includes:

[0012] S21, randomly generate the coordinates (x3, y3) of the first vertex of the intercepted subgraph on the image to be trained;

[0013] S22, calculate the coordinates (x4, y4) of the second vertex of the intercepted sub-image on the image to be trained, where the first vertex and the second vertex are two vertices on the diagonal of the rectangle, x4=x3+ws, y4=y3+hs, where ws is the width of the sub-image to be intercepted, and hs is the height of the sub-image to be intercepted;

[0014] S23, determine whether the coordinates of the intercepted sub-image are within the image to be trained, if not, execute step S24, if yes, execute step S25;

[0015] S24, correcting (x3, y3) and (x4, y4) in S22 by a correction algorithm to ensure that the coordinates of the intercepted sub-image after correction are within the range of the image to be trained, and using the corrected (x3, y3, x4, y4) as the coordinates of the intercepted sub-image on the image to be trained;

[0016] S25. Use (x3, y3, x4, y4) in S22 as the coordinates of the intercepted sub-image on the image to be trained.

[0017] Further, the S21 specifically includes generating a random number a uniformly distributed on [0, 1] and comparing it with a preset threshold b. If the random number a > the threshold b, the random range of (x3, y3) is x3 ∈ [(x1 + x2) i / 2 - ws, (x1 + x2) i / 2], y3 ∈ [(y1 + y2) i / 2 - hs, (y1 + y2) i / 2], where (x1, y1, x2, y2) i are the coordinates of randomly selecting a rectangular image block from the coordinate set Rect; if the random number a ≤ the threshold b, the random range of (x3, y3) is x3 ∈ [-ws / 2, w - ws / 2], y3 ∈ [-hs / 2, h - hs / 2], where w is the width of the image to be trained and h is the height of the image to be trained. The size relationship between the random number a and the threshold b determines whether to intercept a background sub-image or a target sub-image in this round.

[0018] Further, the threshold b is a hyperparameter, representing the probability that the expected intercepted sub-image contains a target. The range is b ∈ [0, 1]. The smaller the value of the threshold b, the greater the probability that the intercepted sub-image is a background sub-image; the larger the value of the threshold b, the greater the probability that the intercepted sub-image contains a target.

[0019] Further, the S3 specifically includes

[0020] Traversing the elements in the coordinate set Rect, for the coordinate values (x1, y1, x2, y2) of a rectangular image block i , if (x1) i > x4, or (y1) i > y4, or (x2) i > x3, or (y2) i < y3, then remove the coordinate (x1, y1, x2, y2) of this rectangular image block from the coordinate set Rect i ;

[0021] Otherwise, correct the coordinate values of this rectangular image block, making (x1)' i = max(0, x1 - x3), (y1)' i = max(0, y1 - y3), (x2)' i = min(w, x2 - x3), (y2)' i = min(h, y2 - y3). The coordinate values (x1', y1', x2', y2') i are the corrected coordinate values of the rectangular image block, forming the coordinate set Rect'.

[0022] Furthermore, the correction algorithm in S24 specifically includes

[0023] If x3<0, then correct x3=0, x4=ws; if y3<0, then correct y3=0, y4=hs; if x4>w, then correct x3=w-ws, x4=w; if y4<0, then correct y3=h-hs, y4=h.

[0024] The beneficial effects of the present invention are: only one annotation is made on the image to be trained, and there is no need to mark the intercepted sub-images that are segmented, thereby reducing the cost of manual annotation. The present invention can control the sampling ratio of the background sub-image and the target sub-image by setting a threshold b, thereby avoiding the problem of a small number of target sub-images and a large number of background sub-images, which leads to an unbalanced ratio. At the same time, the diversity of the target sub-image is improved, providing a more diverse and balanced image training set for deep learning training.

[0025] The present invention also provides a computer-readable storage medium storing a program, wherein the computer program, when running, executes the method for collecting an industrial image target detection image training set as claimed in claim 1. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of an embodiment of the present invention;

[0027] Figure 2 is a flow chart of step S2 in an embodiment of the present invention;

[0028] Figure 3 This is a visualization effect diagram of the coordinate set Rect after the training image is annotated in one embodiment of the present invention;

[0029] Figure 4 A schematic diagram of the position of a sub-image captured on an image to be trained in one embodiment of the present invention;

[0030] Figure 5 This is a visualization effect diagram of the intercepted sub-image and the annotated corrected coordinate set Rect' in one embodiment of the present invention. DETAILED DESCRIPTION

[0031] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0032] Example

[0033] See attached Figure 1-2 As shown, a method for collecting an industrial image target detection image training set of the present invention is used to collect targets for each image to be trained in the training set. The target collection of each image to be trained includes the following steps:

[0034] S1. Annotate the image to be trained to form n rectangular image blocks, record the coordinates of each rectangular image block, and the coordinates of each rectangular image block form a coordinate set Rect, where n is an integer greater than or equal to 0.

[0035] The coordinates of the rectangular image block are (x1, y1, x2, y2), where (x1, y1) and (x2, y2) are the coordinate values ​​of the first vertex and the second vertex of the diagonally opposite corners of the rectangular image block. The coordinate set Rect = {(x1, y1, x2, y2) 1 ,(x1,y1,x2,y2) 2 ,……(x1,y1,x2,y2) n}. The first vertex is the upper left corner vertex of the rectangular image block, and the second vertex is the lower right corner vertex of the rectangular image block. The rectangular image block is manually selected. When n=0, the coordinate set Rect is an empty set.

[0036] S2, obtaining the coordinates of the intercepted sub-image on the image to be trained, randomly sampling on the image to be trained to form a rectangular intercepted sub-image, the intercepted sub-image is a background sub-image without the target object or an object sub-image containing the target object, and the coordinates of the intercepted sub-image on the image to be trained are within the range of the image to be trained. Specifically including:

[0037] S21, randomly generate the coordinates (x3, y3) of the first vertex of the intercepted sub-graph on the image to be trained, where the first vertex is the upper left corner vertex of the intercepted sub-graph.

[0038] S211. Generate a random number a uniformly distributed on [0, 1], and compare the random number a with a preset threshold value b; if the random number a>threshold value b, go to S214 to perform a trial extraction of the background sub-image; if the random number a≤threshold value b, go to step S212 to perform a trial extraction of the target sub-image.

[0039] The relationship between the random number a and the threshold b determines whether the background sub-image or the target sub-image is intercepted in this round. The threshold b is a hyperparameter, which indicates the probability that the intercepted sub-image contains the target, and its range is b∈[0,1]. The smaller the b value, the greater the probability that the intercepted sub-image is pure background without target; the larger the b value, the greater the probability that the intercepted sub-image contains target.

[0040] S212, randomly select the coordinates (x1, y1, x2, y2) of a rectangular image block from the coordinate set Rect i , get the coordinates of the upper left corner of the rectangular image block (x1, y1) i and the coordinates of the lower right corner (x2, y2) i . i is a positive integer less than or equal to n.

[0041] S213, randomly generate the coordinates (x3, y3) of the upper left corner of the intercepted sub-image on the image to be trained, and the random range of (x3, y3) is x3∈[(x1+x2) i / 2-ws, (x1+x2) i / 2],y3∈[(y1+y2) i / 2-hs, (y1+y2) i / 2], where ws is the width of the intercepted sub-image, and hs is the height of the intercepted sub-image. Jump to S22.

[0042] S214. Randomly generate the coordinates (x3, y3) of the upper left corner of the intercepted sub-image on the image to be trained. The random range of (x3, y3) is x3∈[-ws / 2, w-ws / 2], y3∈[-hs / 2, h-hs / 2], where w is the width of the image to be trained, and h is the height of the image to be trained.

[0043] S22, calculate the coordinates (x4, y4) of the second vertex of the intercepted sub-graph on the image to be trained, where the first vertex and the second vertex are two vertices on the diagonal of the rectangle, x4 = x3 + ws, y4 = y3 + hs. The second vertex of the intercepted sub-graph is the lower right corner vertex of the rectangle of the intercepted sub-graph.

[0044] S23, determine whether the coordinates of the intercepted sub-image are within the image to be trained. If not, that is, (x3, y3) or (x4, y4) exceeds the range of the width and height w, h of the original image, and execute step S24 to newly correct (x3, y3) and (x4, y4) to be within the range of the image to be trained. If yes, directly use (x3, y3, x4, y4) in S22 as the coordinates of the intercepted sub-image on the image to be trained, and continue to step S3.

[0045] S24. Correct (x3, y3) and (x4, y4) in S22 through a correction algorithm to ensure that the coordinates of the trimmed intercepted sub-image are within the range of the image to be trained, and use the corrected (x3, y3, x4, y4) as the coordinates of the intercepted sub-image on the image to be trained, and proceed to step S3.

[0046] The correction algorithm specifically includes

[0047] If x3<0, then correct x3=0, x4=ws;

[0048] If y3<0, then correct y3=0, y4=hs;

[0049] If x4>w, then correct x3=w-ws,x4=w;

[0050] If y4<0, then correct y3=h-hs, y4=h.

[0051] S3. Exclude the coordinates of the rectangular image blocks in the coordinate set Rect that are not within the coordinates of the intercepted sub-image, and correct the coordinate values of the remaining rectangular image blocks in the coordinate set Rect to obtain a new coordinate set Rect'. Specifically, it includes

[0052] Traverse the elements in the coordinate set Rect. For the coordinate values (x1, y1, x2, y2) of a rectangular image block i , if (x1) among them i > x4, or (y1) i > y4, or (x2) i > x3, or (y2) i < y3, then remove the coordinates (x1, y1, x2, y2) of this rectangular image block from the coordinate set Rect i .

[0053] Otherwise, correct the coordinate values of this rectangular image block, and let (x1)' i = max(0, x1 - x3), (y1)' i = max(0, y1 - y3), (x2)' i = min(w, x2 - x3), (y2)' i = min(h, y2 - y3). The coordinate values (x1', y1', x2', y2') i are the corrected coordinate values of the rectangular image block, forming the coordinate set Rect'. In the form of max(a, b), it represents the maximum value of a and b, and the form of min(a, b) represents the minimum value of a and b.

[0054] S4. Obtain a sampled image, intercept a sampled image on the to-be-trained image according to the coordinates of the intercepted sub-image, where the coordinates of the rectangular image blocks in the sampled image are corrected according to the coordinate set Rect'. Return to step S1 to perform target acquisition on the next to-be-trained image.

[0055] All decimals obtained in all calculations are rounded to integers, and all coordinate values are integers.

[0056] To illustrate this method in detail, select the to-be-trained image in the appendix Figure 3 for illustration.

[0057] Step 1, label the to-be-trained image as Figure 3 . The resolution of the to-be-trained image is 2400×1080, that is, w = 2400, h = 1080. Select two rectangular image blocks to form the coordinate set Rect = {(2267, 81, 2331, 184) 1 , (2288, 229, 2337, 307) 2}, set the width of the sub - figure to be intercepted as ws = 256 and the height as hs = 256. The annotation visualization is as Figure 3 .

[0058] Step 2: Set the threshold b to 0.5 and generate a random number a with a value equal to 0.22.

[0059] Step 3: Randomly select a rectangular image patch from the annotation coordinate set Rect as (2228, 229, 2337, 307) 2 , obtaining the upper - left corner coordinates (2228, 229) and the lower - right corner coordinates (2337, 307) of this rectangular image patch.

[0060] Step 4: The random range of (x3, y3) is x3 ∈ [(2288 + 2337) / 2 - 256, (2228 + 2337) / 2], y3 ∈ [(229 + 307) / 2 - 256, (229 + 307) / 2], that is, x3 ∈ [2057, 2283], y3 ∈ [12, 268]. The coordinates of the upper - left point of the randomly generated intercepted sub - figure on the original image are (2100, 100).

[0061] Step 5: Calculate the coordinates (x4, y4) of the lower - right point of the intercepted sub - figure on the original image. x4 = x3+ws = 2100 + 256 = 2356, y4 = y3+hs = 100 + 256 = 356. The coordinates (x4, y4) of the lower - right point of the intercepted sub - figure on the original image are (2356, 356); since 0 < 2100, 0 < 100, 2356 < w = 2400, 0 < 356 < h = 1080, no correction is required. The position of the intercepted sub - figure on the image to be trained is as shown in the appendix Figure 4 .

[0062] Step 6: Traverse the coordinate set Rect{(2267, 81, 2331, 184) 1 , (2288, 229, 2337, 307) 2}; among them, (2267, 81, 2331, 184) 1 and the intercepted sub - figure area (2100, 100)(2356, 356) have a common part, and make a correction: (2267, 81, 2331, 184) 1 =(max(0, 2267 - 2100), max(0, 81 - 100), min(256, 2331 - 2100), min(256, 184 - 100)), that is, correct it to (167, 0, 231, 84) 1 ; among them, (2288, 229, 2337, 307) 2The area with the sub-image (2100, 100) (2356, 356) has a common part, which is corrected to (max (0, 2288-2100), max (0, 229-100), min (256, 2337-2100), min (256, 307-100)), that is, (128, 129, 237, 207) 2 ; Get Rect' = {(167,0,231,84) 1 ,(188,129,237,207) 2};

[0063] Step 7: Take a sub-image from the image to be trained with the coordinates of the upper left corner (x3, y3) = (2100, 100) and the coordinates of the lower right corner (x4, y4) = (2356, 356) as shown in the attached figure. Figure 5 , output the intercepted sub-image and coordinate set Rect'={(167,0,231,84) 1 ,(188,129,237,207) 2}.

[0064] Figure 4 for Figure 5 Schematic diagram of the position of the intercepted sub-image on the image to be trained, the sub-image interception range (x3, y3, x4, y4) = (2100, 100, 2356, 356), whether the intercepted sub-image includes the target area is determined by the relationship between the random number a and the threshold b. The probability of the intercepted sub-image containing the target can be adjusted by adjusting the size of the threshold b. The value of the intercepted sub-image range (x3, y3, x4, y4) ensures that the rectangular image block is included while being random, forming a staggered sub-image interception range.

[0065] According to another aspect of the application, a storage medium is also provided, on which a computer program is stored, and when the computer program is run by a computer or a processor, the computer program is used to execute the relevant steps S1-S4 of the method for collecting an image training set for industrial image target detection in the embodiment of the present application. The storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory, a USB memory, or any combination of the above storage media.

[0066] The above implementation modes are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the content of the present invention and implement it, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for collecting image training sets for industrial image target detection. Features: It is used to collect targets for each image to be trained in the training set. The target collection of each image to be trained includes the following steps: S1. Annotate the training image to form n rectangular image blocks, record the coordinates of each rectangular image block, and form a coordinate set Rect, where n is an integer greater than or equal to 0. S2, obtaining the coordinates of the intercepted sub-image on the image to be trained, randomly sampling on the image to be trained to form a rectangular intercepted sub-image, wherein the intercepted sub-image is a background sub-image without a target object or an object sub-image containing a target object, and the interception ratio of the background sub-image and the object sub-image is adjustable, and the coordinates of the intercepted sub-image on the image to be trained are within the range of the image to be trained; S3, excluding the coordinates of the rectangular image blocks in the coordinate set Rect that are not within the coordinate range of the intercepted sub-image, and correcting the coordinate values ​​of the remaining rectangular image blocks in the coordinate set Rect to obtain a new coordinate set Rect'; S4, obtaining a sampling image, and using the coordinates of the intercepted sub-image to intercept the image to be trained to form a sampling image, wherein the coordinates of the rectangular image block in the sampling image are corrected according to the coordinate set Rect'.

2. The method for collecting an industrial image target detection image training set according to claim 1, Features: The coordinates of the rectangular image block are (x1, y1, x2, y2), where (x1, y1) and (x2, y2) are the coordinate values ​​of the first vertex and the second vertex of the diagonally opposite corners of the rectangular image block. The coordinate set Rect = {(x1, y1, x2, y2) 1 ,(x1,y1,x2,y2) 2 ,……(x1,y1,x2,y2) n }.

3. The method for collecting an industrial image target detection image training set according to claim 2, Features: The S2 specifically includes: S21, randomly generate the coordinates (x3, y3) of the first vertex of the intercepted subgraph on the image to be trained; S22, calculate the coordinates (x4, y4) of the second vertex of the intercepted sub-image on the image to be trained, where the first vertex and the second vertex are two vertices on the diagonal of the rectangle, x4=x3+ws, y4=y3+hs, where ws is the width of the intercepted sub-image, and hs is the height of the intercepted sub-image; S23, determine whether the coordinates of the intercepted sub-image on the image to be trained are within the image to be trained, if not, execute step S24, if yes, execute step S25; S24, correcting (x3, y3) and (x4, y4) in S22 by a correction algorithm to ensure that the coordinates of the intercepted sub-image after correction are within the range of the image to be trained, and using the corrected (x3, y3, x4, y4) as the coordinates of the intercepted sub-image on the image to be trained; S25. Use (x3, y3, x4, y4) in S22 as the coordinates of the intercepted sub-image on the image to be trained.

4. The method for collecting an industrial image target detection image training set according to claim 3, Features: The S21 specifically includes: Generate a random number a uniformly distributed on [0, 1] and compare it with the preset threshold b. If the random number a>threshold b, the random range of (x3, y3) is x3∈[(x1+x2) i / 2-ws, (x1+x2) i / 2],y3∈[(y1+y2) i / 2-hs, (y1+y2) i / 2], where (x1,y1,x2,y2) i The coordinates of a rectangular image block are randomly selected in the coordinate set Rect; if the random number a≤threshold b, the random range of (x3, y3) is x3∈[-ws / 2, w-ws / 2], y3∈[-hs / 2, h-hs / 2], where w is the width of the image to be trained and h is the height of the image to be trained.

5. The method for collecting an industrial image target detection image training set according to claim 4, Features: The threshold b is a hyperparameter, which indicates the probability that the target is contained in the expected intercepted sub-image. The range b∈[0,1]. The smaller the threshold b value is, the greater the probability that the intercepted sub-image is a background sub-image is; the larger the threshold b value is, the greater the probability that the target is contained in the intercepted sub-image is.

6. The method for collecting an industrial image target detection image training set according to claim 3, Features: The S3 specifically includes: Traverse the elements in the coordinate set Rect for the coordinate values (x1, y1, x2, y2) of a rectangular image block i , if among them (x1) i > x4, or (y1) i > y4, or (x2) i > x3, or (y2) i < y3, then remove the coordinates (x1, y1, x2, y2) of this rectangular image block from the coordinate set Rect i ; Otherwise, correct the coordinate value of this rectangular image block, let (x1)' i =max(0,x1-x3),(y1)' i =max(0,y1-y3),(x2)' i =min(w,x2-x3),(y2)' i =min(h,y2-y3), coordinate value (x1',y1',x2',y2') i That is, the corrected coordinate value of the rectangular image block forms a coordinate set Rect'.

7. The method for collecting an industrial image target detection image training set according to claim 4, Features: The correction algorithm in S24 specifically includes: If x3<0, then correct x3=0, x4=ws; If y3<0, then correct y3=0, y4=hs; if x4>w, then correct x3=w-ws, x4=w; if y4<0, then correct y3=h-hs, y4=h.

8. A computer-readable storage medium storing a program, Features: The computer program executes the method for collecting an industrial image target detection image training set as described in any one of claims 1 to 7 when running.

Images