Image classification and annotation method, device, electronic device and storage medium

Abstract

The present invention provides an image classification and annotation method, device, electronic device, and storage medium. The method includes: acquiring an image to be tested; comparing the image to be tested with a reference image to generate an image mask, wherein the image mask includes a plurality of connected domains; performing defect detection on the image to be tested; when at least one defect exists in the image to be tested, acquiring the defect coordinates corresponding to the at least one defect; determining whether the corresponding connected domain is a defect connected domain or a normal connected domain based on the center coordinates of the connected domain and the defect coordinates; generating a corresponding first image mask and a second image mask for the defect connected domain and the normal connected domain, respectively; processing the first image mask and the second image mask with the image to be tested, respectively, to obtain a defect component image corresponding to the defect connected domain and a normal component image corresponding to the normal connected domain, respectively. The method provided by the present invention effectively reduces the labor cost of data annotation.

Description

Technical Field

[0001] The present invention relates to the field of data annotation, and in particular to an image classification and annotation method, device, electronic device and storage medium. Background Art

[0002] The surface mounting process for printed circuit boards (PCBs) is delicate and complex, making various defects prone to occur during production. Current researchers are attempting to use neural network training to achieve automated PCB defect detection. However, neural network training requires a large amount of image data of PCB defects as negative samples and image data of normal PCBs as positive samples. Therefore, manual labeling of this vast array of PCB defect data would be a waste of manpower, time, and cost. Summary of the Invention

[0003] In view of the above, it is necessary to provide an image classification and annotation method, device, electronic device and storage medium to solve the above problems.

[0004] A method for image classification and annotation, comprising:

[0005] (a) Acquire an image to be tested;

[0006] (b) comparing the image to be tested with a reference image to generate a corresponding image mask, wherein the image mask includes a plurality of connected domains;

[0007] (c) performing defect detection on the image to be tested;

[0008] (d) when there is at least one defect in the image to be measured, obtaining defect coordinates corresponding to the at least one defect;

[0009] (e) determining, based on the center coordinates of the connected domain and the defect coordinates, whether the corresponding connected domain is a defective connected domain or a normal connected domain;

[0010] (f) generating a corresponding first image mask and a corresponding second image mask for the defect connected domain and the normal connected domain respectively;

[0011] (g) Processing the first image mask and the second image mask with the image to be tested, respectively, to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain, respectively.

[0012] Furthermore, step (b) includes the following sub-steps:

[0013] Step S21: performing grayscale processing on the reference image and the image to be measured respectively to obtain a corresponding first image and second image;

[0014] Step S22: Compare the first image and the second image to obtain a third image;

[0015] Step S23: performing binarization processing on the third image to obtain a fourth image;

[0016] Step S24: marking connected domains of the fourth image to obtain the image mask, where the image mask includes a plurality of connected domains.

[0017] Furthermore, the third image is obtained by calculating a mean square error between the first image and the second image or calculating a structural similarity index between the first image and the second image.

[0018] Furthermore, step (e) includes the following sub-steps:

[0019] Step S51: Calculate the center coordinates of a plurality of connected regions to obtain a plurality of center coordinates;

[0020] Step S52: Calculating the Euclidean distance between each defect coordinate and a plurality of center coordinates to obtain a plurality of sets of Euclidean distances;

[0021] Step S53: selecting the defect connected domain according to a preset rule, and recording the other connected domains on the image mask as the normal connected domains.

[0022] Furthermore, the center coordinates are the centroid coordinates of the connected domain.

[0023] Furthermore, the preset rule is: respectively selecting the minimum values ​​of the plurality of groups of Euclidean distances to obtain a plurality of minimum values, and selecting a plurality of connected domains corresponding to the plurality of minimum values, and recording them as the defective connected domains.

[0024] Furthermore, step (f) includes the following sub-steps:

[0025] Step S61: setting the pixel values ​​of the pixels in the defect connected domain in the image mask to 0 to obtain the first image mask;

[0026] Step S62: repeat the above steps to set the pixel values ​​of the pixels in the normal connected domain in the image mask to 0, thereby obtaining the second image mask.

[0027] Furthermore, step (g) includes the following sub-steps:

[0028] Step S71: multiplying the first image mask and the image to be tested to obtain the defective component image, and marking the defect coordinates on the defective component image;

[0029] Step S72: multiplying the second image mask by the image to be tested to obtain the normal component image.

[0030] Furthermore, when the detection result of step (c) is that the image to be tested is flawless, that is, when the image mask is the second image mask, step (g) is directly performed.

[0031] An image classification and labeling device, comprising:

[0032] An acquisition module, used for acquiring an image to be tested;

[0033] a comparison module, configured to compare the image to be tested with a reference image to generate a corresponding image mask, wherein the image mask includes a plurality of connected domains;

[0034] A defect detection module, used to perform defect detection on the image to be tested;

[0035] a coordinate acquisition module, configured to acquire defect coordinates corresponding to the at least one defect when the defect detection module detects that the image to be tested has at least one defect;

[0036] a determination module, configured to determine, based on the center coordinates of the connected domain and the defect coordinates, whether the corresponding connected domain is a defective connected domain or a normal connected domain;

[0037] a mask generation module, configured to generate a corresponding first image mask and a corresponding second image mask for the defect connected domain and the normal connected domain respectively;

[0038] The processing module is used to process the first image mask and the second image mask with the image to be tested respectively, so as to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain respectively.

[0039] An electronic device, comprising:

[0040] a memory storing at least one instruction; and

[0041] A processor executes instructions stored in the memory to implement the image classification and labeling method.

[0042] A storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the image classification and labeling method.

[0043] The image classification and labeling method provided by the present invention, combined with image masking and defect detection algorithms, can automatically obtain and label normal component images and defective component images, effectively reducing the labor cost of data labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of one embodiment of the image classification and labeling method of the present invention.

[0045] Figure 2 for Figure 1 Sub-steps of step S2 are shown.

[0046] Figure 3 for Figure 1 Sub-steps of step S5 are shown.

[0047] Figure 4 for Figure 1 Sub-steps of step S6 are shown.

[0048] Figure 5 for Figure 1 Sub-steps of step S7 are shown.

[0049] Figure 6 for Figure 1 Schematic diagram of an application scenario of the image classification and labeling method shown.

[0050] Figure 7 It is a functional module diagram of one embodiment of the image classification and labeling device of the present invention.

[0051] Figure 8 It is a structural diagram of an electronic device for implementing one embodiment of the image classification and labeling method of the present invention.

[0052] Description of main component symbols

[0053] Image classification and annotation device-100 Acquisition module-101 Comparison module-102 Defect detection module-103

[0054] Coordinate acquisition module-104 Determination module-105 Mask generation module-106 Processing module-107

[0055] Electronic device-200 Memory-201 Processor-202 Computer program-203

[0056] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1 FIG. 1 is a flowchart of an embodiment of the image classification and labeling method of the present invention. The order of the steps in the flowchart can be changed according to different requirements, and some steps can be omitted.

[0059] Step S1: Acquire an image to be tested.

[0060] Step S2: Compare the image to be tested with a reference image to generate a corresponding image mask, wherein the image mask includes a plurality of connected domains.

[0061] When executing step S2, please refer to Figure 2 , step S2 includes the following sub-steps:

[0062] Step S21: performing grayscale processing on the image to be tested and the reference image respectively to obtain corresponding first and second images;

[0063] Step S22: Compare the first image and the second image to obtain a third image;

[0064] Step S23: performing binarization processing on the third image to obtain a fourth image;

[0065] Step S24: marking connected domains of the fourth image to obtain the image mask, where the image mask includes a plurality of connected domains.

[0066] It can be understood that in step S21, the grayscale processing refers to converting the reference image and the image to be measured from color images into grayscale images in which each pixel has only one pixel value.

[0067] It is understood that in step S22 , the third image may be obtained by calculating the mean-square error (MSE) between the first image and the second image or calculating the structural similarity (SSIM) index between the first image and the second image.

[0068] It can be understood that step S23 further includes the following sub-steps:

[0069] Get a threshold value;

[0070] The grayscale values ​​of pixels in the third image that are greater than or equal to the threshold are set as first pixel values, and the grayscale values ​​of pixels in the third image that are less than the threshold are set as second pixel values ​​to obtain the fourth image.

[0071] The present invention does not limit the threshold value. For example, the threshold value in step S23 may be a global threshold value or an adaptive threshold value. Those skilled in the art may select different threshold values ​​by selecting different binarization algorithms.

[0072] It can be understood that the connected domain in step S24 refers to the image region composed of pixels with the same pixel value and adjacent positions in the fourth image. The connected domain marking refers to finding each connected domain in the fourth image, marking it, and recording all coordinate values ​​corresponding to each marked value.

[0073] It can be understood that the pixel values ​​of the pixels within the connected domains on the image mask are the first pixel values, and the pixel values ​​of the pixels outside the connected domains on the image mask are the second pixel values.

[0074] In one embodiment, the first pixel value in step S23 is 1, and the second pixel value is 0.

[0075] It is understood that there are various connected component labeling algorithms in step S24, and the present invention does not limit such algorithms. For example, the connected component labeling algorithm may be a path-based connected component labeling algorithm or a contour-based connected component labeling algorithm. It is understood that the connected component labeling algorithm is a prior art, and the present invention does not further elaborate on the detailed steps of the connected component labeling algorithm.

[0076] In one embodiment, the image to be tested includes an image of a printed circuit board assembly (PCBA) to which electronic components have been installed, and the reference image includes an image of a printed circuit board (PCB) to which no electronic components have been installed.

[0077] It can be understood that each of the connected domains on the mask corresponds to the electronic component.

[0078] It can be understood that before executing step S21, the reference image and the image to be measured may be pre-processed.

[0079] In at least one embodiment of the present invention, the preprocessing includes:

[0080] translating, rotating or scaling the reference image and the image to be measured so that the reference image and the plurality of images to be measured have the same orientation;

[0081] The reference image and the image to be measured are adjusted to images of standard size.

[0082] It can be understood that the reference image, the image to be measured, the first image, the second image, the third image, the fourth image and the image mask have the same size.

[0083] Step S3: performing defect detection on the image to be tested.

[0084] In at least one embodiment of the present invention, defect detection can be performed on the image to be tested using a trained neural network model.

[0085] In the present invention, the image to be tested may also be subjected to defect detection by other methods, such as image corrosion and / or image expansion algorithms, and the present invention does not limit the defect detection method.

[0086] Step S4: when there is at least one defect in the image to be measured, obtaining defect coordinates corresponding to the at least one defect.

[0087] Step S5: determining whether the corresponding connected domain is a defective connected domain or a normal connected domain according to the center coordinates of the connected domain and the defect coordinates.

[0088] When executing step S5, please refer to Figure 3 , step S5 includes the following sub-steps:

[0089] Step S51: Calculate the center coordinates of a plurality of connected regions to obtain a plurality of center coordinates;

[0090] Step S52: Calculating the Euclidean distance between each defect coordinate and a plurality of center coordinates to obtain a plurality of sets of Euclidean distances;

[0091] Step S53: selecting the defect connected domain according to a preset rule, and recording the other connected domains on the image mask as normal connected domains.

[0092] In one embodiment, the center coordinates are centroid coordinates. Thus, in step S51, a plurality of centroid coordinates can be obtained according to the following centroid coordinate calculation formula (1).

[0093]

[0094] In formula (1), X and Y are the row coordinates and column coordinates of the centroid coordinates, respectively, S represents the connected domain, (i, j) represents the coordinates of each pixel in the connected domain, and N represents the number of pixels in the connected domain.

[0095] It can be understood that the Euclidean distance refers to the real distance between two points in a multidimensional space. In this embodiment, since the image mask and the image to be tested have the same size, the defect coordinates are located at the same position on the image to be tested and on the image mask. Thus, in step S52, the Euclidean distance between the defect coordinates and the center coordinates refers to the distance between the defect coordinates and the center coordinates on the image mask. Thus, the Euclidean distance between the defect coordinates and the center coordinates of each connected domain can be directly calculated using the following Euclidean distance calculation formula (2).

[0096]

[0097] In formula (2), ( , ) is the center coordinate of any connected domain, ( , ) are the defect coordinates. is the Euclidean distance between the defect coordinates and the center coordinates.

[0098] In step S53 , the preset rule is: respectively selecting the minimum values ​​of the plurality of groups of Euclidean distances to obtain a plurality of minimum values; and selecting a plurality of connected domains corresponding to the plurality of minimum values, and recording them as the defective connected domains.

[0099] Step S6: Generate a corresponding first image mask and a corresponding second image mask for the defective connected domain and the normal connected domain. Figure 4 , step S6 includes the following sub-steps:

[0100] Step S61: setting the pixel values ​​of the pixels in the defect connected domain in the image mask to 0 to obtain the first image mask;

[0101] Step S62: repeat the above steps to set the pixel values ​​of the pixels in the normal connected domain in the image mask to 0, thereby obtaining the second image mask.

[0102] Step S7, processing the first image mask and the second image mask with the image to be tested respectively, so as to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain. Figure 5 , step S7 includes the following sub-steps:

[0103] Step S71: multiplying the first image mask and the image to be tested to obtain the defective component image, and marking the defect coordinates on the defective component image;

[0104] Step S72: multiplying the second image mask by the image to be tested to obtain the normal component image.

[0105] It is understandable that in other embodiments, when the detection result of step S3 is that the image to be tested is flawless, that is, when the image mask is the second image mask, step S7 is directly executed to obtain a normal component image.

[0106] See also Figure 6 In the embodiment of the present application, the image classification and labeling method of this case is described in detail by taking the example of two electronic components being provided in the image to be tested.

[0107] First, an image to be tested and a corresponding reference image are obtained, wherein, as described above, the image to be tested includes two electronic components. The image to be tested and the reference image are compared to generate an image mask corresponding to the image to be tested, wherein the image mask includes connected domain 1 and connected domain 2.

[0108] Next, a defect detection is performed on the image to be tested, and if the detection result indicates that a defect (ie, a triangle in the figure) exists in the image to be tested, defect coordinates corresponding to the defect are obtained.

[0109] Then, according to the centroid coordinate formula, the centroid coordinates of the connected domain 1 and the connected domain 2 are calculated respectively, and are recorded as center coordinate a and center coordinate b respectively. Then, the Euclidean distances between the center coordinate a and the center coordinate b and the defect coordinates are calculated respectively, and are recorded as Euclidean distance (1) and Euclidean distance (2). By comparing the Euclidean distance (1) and the Euclidean distance (2), it is found that the Euclidean distance (1) is greater than the Euclidean distance (2), then the connected domain 1 is a normal connected domain, and the connected domain 2 is a defect connected domain corresponding to the defect.

[0110] Next, the pixel values ​​of the pixels in the defective connected domain in the image mask are set to 0 to obtain a first image mask; this step is repeated to set the pixel values ​​of the pixels in the normal connected domain in the image mask to 0 to obtain a second image mask.

[0111] Then, the first image mask is multiplied by the image to be tested to obtain a defective component image, and the defect is marked on the defective component image. The second image mask is multiplied by the image to be tested to obtain a normal component image.

[0112] The present invention obtains an image mask including several connected domains by comparing a reference image with a corresponding image to be tested. When there is a defect in the image to be tested, the defect connected domain corresponding to the defect is found, and the connected domains on the image mask other than the defect connected domain are marked as normal connected domains. The pixel values ​​of the pixels of the defect connected domain in the image mask are set to 0 to obtain a first image mask; the pixel values ​​of the pixels of the normal connected domain in the image mask are set to 0 to obtain a second image mask. The first image mask is multiplied by the image to be tested to obtain a defective component image; the second image mask is multiplied by the image to be tested to obtain a normal component image. When the image to be tested is defect-free, the image mask is directly multiplied by the image to be tested to obtain a normal component image. The image classification and annotation method provided by the present invention effectively reduces the labor cost during data annotation.

[0113] See also Figure 7 Another embodiment of the present invention further provides an image classification and labeling device 100. The image classification and labeling device 100 includes an acquisition module 101, a comparison module 102, a defect detection module 103, a coordinate acquisition module 104, a determination module 105, a mask generation module 106, and a processing module 107.

[0114] The acquisition module 101 is used to acquire an image to be tested.

[0115] The comparison module 102 is configured to compare the image to be tested with a reference image to generate a corresponding image mask. The image mask includes a plurality of connected domains.

[0116] The defect detection module 103 is used to perform defect detection on the image to be tested.

[0117] The coordinate acquisition module 104 is configured to acquire defect coordinates corresponding to the at least one defect when the defect detection module 103 detects that at least one defect exists in the image to be detected.

[0118] The determining module 105 is configured to determine whether a corresponding connected domain is a defective connected domain or a normal connected domain according to the center coordinates of the connected domain and the defect coordinates.

[0119] The mask generation module 106 is configured to generate a corresponding first image mask and a corresponding second image mask for the defect connected region and the normal connected region, respectively.

[0120] The processing module 107 is configured to process the first image mask and the second image mask with the image to be tested, respectively, to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain, respectively.

[0121] It can be understood that when the detection result of the defect detection module 103 is that the image to be tested has no defects, that is, when the mask is the second image mask, the processing module 107 is directly used to process the mask and the image to be tested to obtain the normal component image.

[0122] It can be understood that the acquisition module 101, comparison module 102, defect detection module 103, coordinate acquisition module 104, determination module 105, mask generation module 106 and processing module 107 are used to jointly implement steps S1 to S7 in the above-mentioned image classification and labeling method embodiment. The specific implementation process of each functional module will not be repeated here. Please refer to the above-mentioned steps S1 to S7 for details.

[0123] Understandable, please refer to Figure 8 Another embodiment of the present invention further provides an electronic device 200. The electronic device 200 includes a memory 201, a processor 202, and a computer program 203 stored in the memory 201 and executable on the processor 202.

[0124] The electronic device 200 may be any one of a smartphone, a tablet computer, a laptop computer, an embedded computer, a desktop computer, or a server, etc. Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 200 and does not limit the electronic device 200 , which may include more or fewer components than shown, or may combine certain components, or may include different components.

[0125] The processor 202 is configured to implement the steps of the above-described image classification and labeling method embodiments, such as steps S1-S7 shown in the first embodiment, when executing the computer program 203. Alternatively, the processor 202 implements the functions of the modules / units of the above-described image classification and labeling apparatus 100 embodiments, such as the acquisition module 101, comparison module 102, defect detection module 103, coordinate acquisition module 104, determination module 105, mask generation module 106, and processing module 107 in the second embodiment, when executing the computer program 203.

[0126] Exemplarily, the computer program 203 may be divided into one or more modules / units, which are stored in the memory 201 and executed by the processor 202 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 203 in the electronic device 200. For example, the computer program 203 may be divided into the acquisition module 101, comparison module 102, defect detection module 103, coordinate acquisition module 104, determination module 105, mask generation module 106, and processing module 107 in the second embodiment.

[0127] The processor 202 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 202 may be any conventional processor, etc. The processor 202 is the control center of the electronic device 200, and connects various parts of the entire electronic device 200 using various interfaces and lines.

[0128] The memory 201 can be used to store the computer program 203 and / or modules / units. The processor 202 implements the various functions of the electronic device 200 by running or executing the computer program and / or modules / units stored in the memory 201 and accessing data stored in the memory 201. The memory 201 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback). The data storage area may store data generated during the use of the electronic device 200 (such as video data, audio data, a phone book, etc.). Furthermore, the memory 201 may include high-speed random access memory (RAM) and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0129] In one embodiment of the present invention, the electronic device 200 is an automated optical inspection (AOI) instrument. It can be understood that an AOI instrument is a device that detects common defects encountered in welding production based on optical principles.

[0130] If the modules / units integrated in the electronic device 200 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in the form of source code, an executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical signal, and a software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electrical carrier signals or telecommunications signals.

[0131] In the several embodiments provided by the present invention, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the electronic device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0132] In addition, the functional modules in various embodiments of the present invention may be integrated into the same processing module, each module may exist physically separately, or two or more modules may be integrated into the same module. The above-mentioned integrated modules may be implemented in the form of hardware or hardware plus software functional modules.

[0133] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims be included in the present invention. Any figure marks in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices stated in the electronic device claim may also be implemented by the same module or electronic device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image classification and annotation method, characterized in that: The image classification and annotation method comprises: (a) Acquire an image to be tested; (b) comparing the image to be tested with a reference image to generate a corresponding image mask, wherein the image mask includes a plurality of connected domains; (c) performing defect detection on the image to be tested; (d) when there is at least one defect in the image to be measured, obtaining defect coordinates corresponding to the at least one defect; (e) determining, based on the center coordinates of the connected domain and the defect coordinates, whether the corresponding connected domain is a defective connected domain or a normal connected domain; (f) generating a corresponding first image mask and a corresponding second image mask for the defect connected domain and the normal connected domain respectively; (g) Processing the first image mask and the second image mask with the image to be tested, respectively, to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain, respectively.

2. The image classification and labeling method according to claim 1, wherein: Step (b) includes the following sub-steps: Step S21: performing grayscale processing on the reference image and the image to be measured respectively to obtain a corresponding first image and second image; Step S22: Compare the first image and the second image to obtain a third image; Step S23: performing binarization processing on the third image to obtain a fourth image; Step S24: marking connected domains of the fourth image to obtain the image mask, where the image mask includes a plurality of connected domains.

3. The image classification and labeling method according to claim 2, wherein: The third image is obtained by calculating a mean square error between the first image and the second image or calculating a structural similarity index between the first image and the second image.

4. The image classification and labeling method according to claim 1, wherein: Step (e) includes the following sub-steps: Step S51: Calculate the center coordinates of a plurality of connected regions to obtain a plurality of center coordinates; Step S52: Calculating the Euclidean distance between each defect coordinate and a plurality of center coordinates to obtain a plurality of sets of Euclidean distances; Step S53: selecting the defect connected domain according to a preset rule, and recording the other connected domains on the image mask as the normal connected domains.

5. The image classification and labeling method according to claim 4, wherein: The center coordinates are the centroid coordinates of the connected domain.

6. The image classification and labeling method according to claim 4, wherein: The preset rule is: respectively selecting the minimum values ​​of the plurality of groups of Euclidean distances to obtain a plurality of minimum values, and selecting a plurality of connected domains corresponding to the plurality of minimum values, and recording them as the defective connected domains.

7. The image classification and labeling method according to claim 1, wherein: Step (f) includes the following sub-steps: Step S61: setting the pixel values ​​of the pixels in the defect connected domain in the image mask to 0 to obtain the first image mask; Step S62: repeat the above steps to set the pixel values ​​of the pixels in the normal connected domain in the image mask to 0, thereby obtaining the second image mask.

8. The image classification and labeling method according to claim 1, wherein: Step (g) includes the following sub-steps: Step S71: multiplying the first image mask and the image to be tested to obtain the defective component image, and marking the defect coordinates on the defective component image; Step S72: multiplying the second image mask by the image to be tested to obtain the normal component image.

9. The image classification and labeling method according to claim 1, wherein: When the detection result of step (c) is that the image to be tested is flawless, that is, when the image mask is the second image mask, step (g) is directly performed.

10. An image classification and labeling device, characterized in that: The image classification and labeling device comprises: An acquisition module, used for acquiring an image to be tested; a comparison module, configured to compare the image to be tested with a reference image to generate a corresponding image mask, wherein the image mask includes a plurality of connected domains; A defect detection module, used to perform defect detection on the image to be tested; a coordinate acquisition module, configured to acquire defect coordinates corresponding to the at least one defect when the defect detection module detects that the image to be tested has at least one defect; a determination module, configured to determine, based on the center coordinates of the connected domain and the defect coordinates, whether the corresponding connected domain is a defective connected domain or a normal connected domain; a mask generation module, configured to generate a corresponding first image mask and a corresponding second image mask for the defect connected domain and the normal connected domain respectively; The processing module is used to process the first image mask and the second image mask with the image to be tested respectively, so as to obtain a defective component image corresponding to the defective connected domain and a normal component image corresponding to the normal connected domain respectively.

11. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the image classification and labeling method according to any one of claims 1 to 9.

12. A storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is executed by a processor in the electronic device to implement the image classification and labeling method according to any one of claims 1 to 9.

Images