Sample data marking method of image detection model

Abstract

The invention discloses a sample data marking method for an image detection model. The method comprises the following steps: S1, determining a non-abnormal image as a basic image; s2, an image to be detected is obtained and compared with the basic image, and the image to be detected and the basic image are subjected to inverse color superposition or inverse color duplicate removal to screen out abnormal images in the image to be detected; s3, independently extracting and splitting each abnormal position in the abnormal image through contour detection to form a plurality of independent graphs; s4, determining a minimum bounding rectangular frame according to the edge pixel position of each independent graph; and S5, digitizing the determined minimum bounding rectangular frame, and taking the digitized minimum bounding rectangular frame as sample data for image detection model training.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition data labeling, and in particular to a sample data labeling method for an image detection model. Background Art

[0002] In the field of semiconductor processing and production, the showerhead is used in the etching equipment of semiconductor integrated circuit manufacturing. In the wafer manufacturing etching process, it serves as an electrode for adding voltage and a passage for etching gas to enter the cavity. It is a core component necessary for the wafer manufacturing etching process. The core factor affecting the production quality of the etching process is the detection of patterned wafers, which specifically refers to the identification and analysis of defects or errors that may occur on the wafer surface during pattern lithography, such as defects, misalignment, and feature loss on the pattern surface of the wafer. Therefore, technicians in this field generally use visual inspection technology to inspect wafer quality and assist in identifying defects in the manufacturing process to improve the yield rate.

[0003] The recognition software that traditional visual inspection relies on is limited by the limited data set, and it takes a long time to manually mark abnormal images. However, this type of abnormal images has the following two conditions: 1) It is difficult to determine the scope of the abnormality: because there is a problem of repeated color intersections at the boundaries between the good area and the bad area, manual marking of abnormal data may result in inaccurate marking; 2) Marking is cumbersome and time-consuming: because each data point in the abnormal area needs to be marked one by one, the marking process is cumbersome and complicated, and relatively time-consuming when processing large amounts of data. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a sample data labeling method for an image detection model with high efficiency, accuracy and stability.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a sample data labeling method for an image detection model, comprising the following steps: S1. Establish an image without abnormalities as a basic image; S2. Acquire the image to be tested and compare it with the base image, wherein the image to be tested and the base image are superimposed or deduplicated by inverting the colors to screen out abnormal images in the image to be tested; S3. The abnormal positions in the abnormal image are independently extracted and split into multiple independent graphics by contour detection; S4. Determine the minimum bounding rectangle according to the edge pixel position of each independent graphic; S5. The established minimum bounding rectangle is converted into data and used as sample data for training the image detection model.

[0006] Furthermore, the color inversion and deduplication includes the following steps: first, the base image is inverted, then, based on the RGB matrix difference between the inverted base image and the image to be tested, finally, the area corresponding to the RGB matrix difference is used as the abnormal image in the image to be tested.

[0007] Furthermore, the color inversion overlay includes the following steps: first, a contour range image of the image to be tested is obtained through contour detection, then, the area enclosed by the contour range is filled with black, and then, the filled contour range image is superimposed on the basic image to form an overlay image, and finally, the overlay image is inverted and used as an abnormal image in the image to be tested.

[0008] Furthermore, each abnormal position in the abnormal image can be manually marked, and the manually marked pixel position can be digitized as sample data for image detection model training.

[0009] Further, in step S4, calculating the minimum bounding rectangle includes the following steps: first, based on a given point set of an independent figure obtained by contour detection; then, finding the minimum and maximum values ​​in the X-axis and Y-axis directions from the point set respectively; next, based on the minimum and maximum values, respectively, calculating and confirming the upper left endpoint coordinates and the rectangular size of the independent figure, wherein the upper left endpoint coordinates are the minimum value of the X-axis and the maximum value of the Y-axis, the width of the rectangular size is the difference between the maximum value and the minimum value of the X-axis, and the width of the rectangular size is the difference between the maximum value and the minimum value of the Y-axis; finally, establishing the minimum bounding rectangle based on the upper left endpoint coordinates and the rectangular size.

[0010] The present invention adopts the above-mentioned scheme, and its beneficial effects are: 1. Improving detection efficiency: By establishing a basic image and performing image comparison, the abnormal part in the image to be tested can be quickly identified, thereby improving the overall detection efficiency. 2. Accurately identifying anomalies: Using inverted color overlay or inverted color deduplication technology, the abnormal area in the image can be more accurately identified, reducing misjudgment and missed judgment. 3. Automated processing: Through contour detection and calculation of the minimum circumscribed rectangular frame, the automatic extraction and marking of abnormal areas can be achieved, reducing manual intervention and improving the level of automation. 4. Digitized samples: The minimum circumscribed rectangular frame of the abnormal image is digitized to provide standardized training samples for the image detection model, which helps to improve the accuracy and generalization ability of the model. 5. Easy to implement: The entire method has clear steps, is easy to program and implement, and is easy to integrate into existing image processing and detection systems. 6. Improving model performance: By providing high-quality training samples, it helps to improve the performance of the image detection model, making it more reliable and effective in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1Schematic diagram of the flow of the sample data labeling method.

[0012] Figure 2 The figure is a flowchart of color inversion and de-duplication.

[0013] Figure 3 This is a flowchart of color inversion overlay.

[0014] Figure 4 Schematic diagram of the contour detection process. DETAILED DESCRIPTION

[0015] In order to facilitate the understanding of the present invention, the present invention is described more fully below with reference to the accompanying drawings. The accompanying drawings provide preferred embodiments of the present invention. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. The purpose of providing these embodiments is to make the disclosure of the present invention more thoroughly and comprehensively understood.

[0016] See attached Figure 1-4 As shown, in this embodiment, a sample data labeling method of an image detection model includes the following steps: Step S1. Establish an image without abnormalities as the basic image. Figure 2 Taking the black and white image as an example, the image without abnormality is fixed, with a complete white circle in the middle and a hollow area with black edges.

[0017] Step S2. Acquire the image to be tested and compare it with the base image, wherein the image to be tested and the base image are superimposed or deduplicated by inverting the colors to screen out abnormal images in the image to be tested; In this embodiment, the abnormal images in the image to be tested can be screened out by using two methods: inverted color superposition or inverted color deduplication, as follows: See attached Figure 3As shown in FIG, the inversion and deduplication process includes the following steps: first, the base image is inverted (i.e., the original white area becomes black, and the original black area becomes white). Then, based on the RGB matrix difference between the inverted base image and the image to be tested (i.e., the RGB matrix of the inverted base image is: A=[RA, GA, BA], the RGB matrix between the images to be tested is =[RB, GB, BB], and the RGB matrix difference C=[|RA− RB|,|GA−GB|,|BA−BB|]), finally, the area corresponding to the RGB matrix difference is used as the abnormal image in the image to be tested. Specifically, the white area in the inverted base image and the image to be tested is R=255, G=255, B=255, and the black area is R=0, G=0, B=0. The RGB matrix difference between the two is used to screen out the black area with the RGB matrix difference of R=0, G=0, B=0 as the abnormal position, and the RGB matrix difference calculated for the remaining areas is R=255, G=255, B=255 or R=-255, G=-255, B=-255, so that the absolute value is taken to convert the negative RGB value into a positive RGB value, so that the remaining areas can all be converted into white areas. Based on this, the abnormal image in the image to be tested can be quickly screened out.

[0018] See attached Figure 4 As shown, the color inversion overlay includes the following steps: first, the contour range image of the image to be tested is obtained through contour detection (that is, the contour image of the white area is obtained through contour detection), then, the area enclosed by the contour range is filled with black, and then, the filled contour range image is superimposed on the basic image to form an overlay image (that is, the white area remaining after the black contour range image is superimposed on the basic image is the abnormal position), and finally, the overlay image is inverted as the abnormal image in the image to be tested (by inverting the overlay image to invert the remaining white area into black, the inverted black area is used as the abnormal image in the image to be tested).

[0019] In summary, the above-mentioned two methods of inverted color overlay or inverted color deduplication can quickly and accurately screen out abnormal images in the image to be tested, which is more convenient for subsequent processing and effectively solves the problem that the binary image is not conducive to marking because the local black abnormal position near the edge will be intersected and mixed with the black part of the normal position.

[0020] Furthermore, each abnormal position in the abnormal image screened out by the above-mentioned inversion and overlay or inversion and deduplication methods can be manually marked, and the manually marked pixel positions can be digitized as sample data for image detection model training.

[0021] Step S3. Extract and split each abnormal position in the abnormal image into multiple independent graphics by contour detection. Figure 4 As shown, each location (black area) in the abnormal image is independently extracted and split into multiple independent graphics by contour detection.

[0022] Specifically, contour detection includes the following steps: 1) Boundary tracking: First, scan from the upper left corner of the abnormal image to determine the first foreground pixel (contour starting point), then use the foreground pixel as the starting point to scan and track along the boundary of the abnormal position until returning to the foreground pixel again, thus completing the scanning and identification of a contour; the above-mentioned boundary tracking uses the eight-neighborhood connection method to confirm the next foreground pixel and the next contour, and finally extract and split each abnormal position independently. 2) Contour direction and hierarchy: While performing boundary tracking, record the direction of the contour, which is conducive to confirming the hierarchical relationship between multiple contours (a contour can contain other contours, or be contained by other contours); 3) Recursive search for sub-contours: However, after completing the boundary tracking of a contour, continue to explore the area inside the contour to find possible sub-contours, and repeat boundary tracking and hierarchy establishment for the sub-contours found. 4) End condition: When all foreground pixels in the abnormal image have been visited and all contours and their sub-contours have been identified, the contour detection ends.

[0023] Step S4. Confirm the minimum bounding rectangle according to the edge pixel position of each independent figure. Specifically, in step S4, calculating the minimum bounding rectangle includes the following steps: first, based on the point set given by the independent figure obtained by contour detection; then, find the minimum and maximum values ​​in the X-axis and Y-axis directions from the point set respectively; then, based on the minimum and maximum values, respectively calculate and confirm the upper left endpoint coordinates and the rectangular size of the independent figure, wherein the upper left endpoint coordinates are the minimum value of the X-axis and the maximum value of the Y-axis (that is, the upper left endpoint is (Xmin, Ymax)), the width of the rectangular size is the difference between the maximum value and the minimum value of the X-axis (that is, the width S=Xmax-Xmin) and the width of the rectangular size is the difference between the maximum value and the minimum value on the Y-axis (length L=Ymax-Ymin); finally, establish the minimum bounding rectangle based on the upper left endpoint coordinates and the rectangular size.

[0024] S5. The established minimum bounding rectangle is digitized (saved in an XML file) and used as sample data for image detection model training. This effectively solves the problem of only abnormal points being manually marked in traditional images, automates the marking process, and saves time for data marking.

[0025] In summary, the present invention provides an abnormal image detection method based on image comparison technology, which quickly screens out abnormal images in the image to be tested by two methods: inverted color superposition and inverted color deduplication, and realizes accurate marking and digitization of abnormal positions through contour detection and calculation of the minimum circumscribed rectangular frame, thereby providing efficient and accurate sample data for the training of image detection models.

[0026] In addition, the method of the present invention can also be combined with deep learning technology to further improve the accuracy and robustness of abnormal image detection. By training the deep learning model, the features in the image can be automatically learned and extracted, thereby achieving more intelligent and automated abnormal image detection.

[0027] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Any technician familiar with the art who, without departing from the scope of the technical solution of the present invention, makes more possible changes and modifications to the technical solution of the present invention using the technical content disclosed above, or modifications are all equivalent embodiments of the present invention. Therefore, any equivalent and equivalent changes made according to the ideas of the present invention without departing from the content of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A sample data labeling method for an image detection model, characterized in that: The following steps are involved: S1. Establish an image without abnormalities as a basic image; S2. Acquire the image to be tested and compare it with the base image, wherein the image to be tested and the base image are superimposed or deduplicated by inverting the colors to screen out abnormal images in the image to be tested; S3. The abnormal positions in the abnormal image are independently extracted and split into multiple independent graphics by contour detection; S4. Determine the minimum bounding rectangle according to the edge pixel position of each independent graphic; S5. The established minimum bounding rectangle is converted into data and used as sample data for training the image detection model.

2. The sample data labeling method of an image detection model according to claim 1, characterized in that: The color inversion and deduplication process comprises the following steps: first, the base image is inverted, then, based on the RGB matrix difference between the inverted base image and the image to be tested, finally, the area corresponding to the RGB matrix difference is used as the abnormal image in the image to be tested.

3. The sample data labeling method of an image detection model according to claim 1, characterized in that: The color inversion overlay comprises the following steps: first, obtaining a contour range image of the image to be tested by contour detection, then filling the area enclosed by the contour range with black, then overlaying the filled contour range image on the basic image to form an overlay image, and finally inverting the overlay image as an abnormal image in the image to be tested.

4. The sample data labeling method of an image detection model according to claim 2 or 3, characterized in that: Each abnormal position in the abnormal image can be manually marked, and the manually marked pixel position can be digitized as sample data for image detection model training.

5. The sample data labeling method of an image detection model according to claim 1, characterized in that: In step S4, calculating the minimum bounding rectangle includes the following steps: First, the point set given by the independent figure obtained by contour detection; Then, the minimum and maximum values ​​in the X-axis and Y-axis directions are found from the point set respectively; next, the upper left endpoint coordinates and the rectangular size of the independent figure are calculated and confirmed based on the minimum and maximum values, wherein the upper left endpoint coordinates are the minimum value of the X-axis and the maximum value of the Y-axis, the width of the rectangular size is the difference between the maximum value and the minimum value of the X-axis, and the width of the rectangular size is the difference between the maximum value and the minimum value of the Y-axis; finally, the minimum circumscribed rectangular frame is established based on the upper left endpoint coordinates and the rectangular size.

Images