Image processing method, region detection method, detection device and storage medium

Abstract

The present application discloses an image processing method, a region detection method, a detection device and a storage medium. The image processing method comprises: obtaining an image to be processed, and obtaining image texture features of the image to be processed; dividing the image to be processed into several regions based on the gradient change of the image texture features; determining the region that meets the image gradient conditions in the several regions as the circuit board region, and determining the remaining regions as the non-circuit board region; segmenting the image to be processed according to the circuit board region to obtain a circuit board image. Through the above implementation, the change of the image texture features of the image to be processed is obtained, the image to be processed is divided into the circuit board region and the non-circuit board region, and the region is segmented to obtain a circuit board image, so as to facilitate the alignment operation of the circuit board image with the original circuit board design image, and facilitate defect detection of the circuit board image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, a region detection method, a detection device and a storage medium. Background Art

[0002] Automated Optical Inspection (AOI) equipment in the printed circuit board (PCB) industry has very important practical significance. AOI refers to the detection and location of defects on PCBs on PCB production lines by collecting images with optical cameras and using image processing, machine learning and other methods.

[0003] There are inevitably a lot of defects in the PCB production process, and the circuits on the PCB are very small and dense. It is very time-consuming and laborious to check the defects manually, and it is easy to make mistakes. In addition, if the defects are not detected and the defective PCB boards are directly put into the next process for manufacturing, the subsequent repair costs will become higher and higher, and they are more likely to be scrapped, resulting in a lot of waste costs.

[0004] However, during the inspection process, the original PCB design and the image captured by the camera must first be aligned before they can be compared and then defect detection can be performed. In order to ensure that the image captured by the camera does not miss the PCB board, the camera usually captures the area outside the PCB board area when capturing the PCB board image. However, this part of the image is not processed. Directly aligning the original PCB design and the image captured by the camera may result in alignment failures and a large number of false point defect reports. Summary of the invention

[0005] In order to solve the above-mentioned technical problems existing in the prior art, the present application provides an image processing method, a region detection method, an automatic optical detection device and a storage medium.

[0006] To solve the above problems, the present application provides an image processing method, which includes: obtaining an image to be processed and obtaining image texture features of the image to be processed; dividing the image to be processed into several regions based on the gradient changes of the image texture features; determining the region that meets the image gradient conditions among the several regions as a circuit board region, and determining the remaining regions as non-circuit board regions; and segmenting the image to be processed according to the circuit board region to obtain a circuit board image.

[0007] To solve the above problems, the present application provides an abnormal area detection method, which includes: obtaining a circuit board image, wherein the circuit board image is obtained by the above-mentioned image processing method; obtaining a repeated unit area; using the repeated unit area to traverse and match all areas including the circuit board image to obtain the similarity between the repeated unit area and each area; marking the area in the circuit board image whose similarity is less than a first similarity threshold as an abnormal area.

[0008] To solve the above problems, the present application provides an automatic optical inspection device, which includes: a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the computer program to implement the above method.

[0009] To solve the above problems, the present application provides a computer-readable storage medium on which program instructions are stored, and the above method is implemented when the program instructions are executed by a processor.

[0010] Compared with the prior art, the image processing method of the present application includes: obtaining an image to be processed, and obtaining image texture features of the image to be processed; dividing the image to be processed into several regions based on the gradient change of the image texture features; determining the region that meets the image gradient conditions among the several regions as the circuit board region, and determining the remaining regions as the non-circuit board region; segmenting the image to be processed according to the circuit board region to obtain a circuit board image. Through the above implementation, the change of the image texture features of the image to be processed is obtained, the image to be processed is divided into the circuit board region and the non-circuit board region, and the region is segmented to obtain a circuit board image, so as to facilitate the alignment operation of the circuit board image with the original circuit board design image, and facilitate defect detection of the circuit board image.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 It is a schematic diagram of an embodiment of the image processing method provided by the present application;

[0014] Figure 2 is a flowchart of another embodiment of the image processing method provided by the present application;

[0015] Figure 3 is a schematic diagram of an embodiment of an image to be processed provided by the present application;

[0016] Figure 4 yes Figure 2 A schematic diagram of a flow chart of an embodiment before step S206 is shown;

[0017] Figure 5 yes Figure 1 A schematic diagram of a flow chart of an embodiment before step S104 is shown;

[0018] Figure 6 It is a schematic diagram of a flow chart of an embodiment of determining a circuit board area provided by the present application;

[0019] Figure 7 It is a schematic diagram of a flow chart of an embodiment of determining a circuit board area provided by the present application;

[0020] Figure 8 It is a schematic diagram of a flow chart of an embodiment of the abnormal area detection method provided by the present application;

[0021] Fig. 9 yes Figure 8 A schematic flow chart of an embodiment of step S802;

[0022] Fig.10 is a schematic diagram of an embodiment of the first repeating unit region and the first adjacent region in the present application;

[0023] Fig.11 yes Fig. 9 A flowchart of an embodiment of the steps after step S903;

[0024] Fig.12 This is a schematic diagram of an embodiment of the second repeating unit region and the second adjacent region in the present application;

[0025] Fig.13 yes Fig. 9 A flowchart of another embodiment of the steps after step S903;

[0026] Fig.14 yes Figure 8 A flowchart of an embodiment of the steps before step S104;

[0027] Fig.15 is a schematic diagram of an embodiment of a similarity histogram provided by the present application;

[0028] Fig.16 yes Figure 8 A flowchart of an embodiment of the steps after step S804;

[0029] Fig.17 It is a structural schematic diagram of an embodiment of the automatic optical inspection device of the present application;

[0030] Fig.18 It is a structural diagram of an embodiment of the computer storage medium of the present application. DETAILED DESCRIPTION

[0031] The present application is further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present application, but are not intended to limit the scope of the present application. Similarly, the following examples are only some embodiments of the present application rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present application.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] In the description of this application, it is necessary to explain that, unless otherwise clearly specified and limited, the terms "install", "set", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or connected through an intermediate medium. For ordinary technicians in this field, the above-mentioned specific meanings belonging to this application can be connected according to specific circumstances.

[0034] This application provides an image processing method, see Figure 1 , Figure 1 1 is a flow chart of an embodiment of the image processing method provided by the present application. Specifically, it may include the following steps S101 to S104.

[0035] Step S101: obtaining an image to be processed, and obtaining image texture features of the image to be processed.

[0036] The image to be processed includes a circuit board image. The image to be processed can be obtained by an automatic optical inspection device, or by transmitting the image to be processed obtained by other means to the automatic optical inspection device for subsequent processing. For example, when the image to be processed is obtained by the automatic optical inspection device, the motion control module can be used to control the movement of the mechanical table and the motor, and the circuit board can be transferred to the image acquisition area, and the image of the circuit board can be acquired by the image acquisition module. In the process of image acquisition, the image acquisition area can be supplemented with light, such as by controlling the switch and brightness of the light source through the light source control module.

[0037] During the image acquisition process, in order to ensure that the circuit board image is not missed in the acquired image to be processed, the image acquisition module usually enlarges the acquisition area, that is, the area other than the circuit board area is acquired at the same time, and finally the circuit board area and the area other than the circuit board area together constitute the image to be processed. That is, the image to be processed includes both the circuit board area and the non-circuit board area. Among them, due to the many lines in the circuit board, the images of the circuit board area acquired are mostly dense patterns, and their image gradients are large; while the objects outside the circuit board are mostly blank areas, and the non-circuit board areas acquired usually have no dense patterns, and their image gradients are relatively small relative to the circuit board area image gradients. By acquiring the image texture features of the image to be processed, the image gradients of different areas in the image to be processed can be obtained.

[0038] Step S102: Divide the image to be processed into several regions based on the gradient changes of the image texture features.

[0039] After obtaining the texture features of the image to be processed, the gradient change of the texture features on the image to be processed can be obtained. Based on the gradient change of the image texture features, the image to be processed is divided into several regions, that is, the region with larger image texture features and the region with smaller image texture features can be divided to obtain several regions.

[0040] Step S103: determining the area that meets the image gradient condition among the plurality of areas as the circuit board area, and determining the remaining areas as the non-circuit board area.

[0041] After the image to be processed is divided into several regions, the image portion with larger image texture and the image portion with smaller image texture are respectively divided into different regions. Therefore, the region with larger image texture can be determined as the circuit board region; the region with smaller image texture can be determined as the non-circuit board region.

[0042] Step S104: segment the image to be processed according to the circuit board area to obtain a circuit board image.

[0043] After distinguishing the circuit board area and the non-circuit board area in the image to be processed, the image to be processed can be segmented to obtain a circuit board image that only includes the circuit board area. Thus, by obtaining the change of the image texture features of the image to be processed, the image to be processed is divided into the circuit board area and the non-circuit board area, and the area is segmented to obtain the circuit board image, so as to facilitate the alignment operation of the circuit board image with the original circuit board design image and facilitate defect detection of the circuit board image.

[0044] In one embodiment, the step of dividing the image to be processed into a plurality of regions based on the gradient change of the image texture feature (step S102) includes: calculating the image gradient of the image to be processed based on the image texture feature; dividing the image to be processed into a plurality of regions according to the image gradient of the image to be processed and a preset gradient threshold.

[0045] After obtaining the texture features of the image to be processed, the gradients of the images of different parts in the image to be processed can be calculated. The image gradient of the circuit board area in the image to be processed is larger, and the image gradient of the non-circuit board area is smaller. There will be a large image gradient change from the circuit board area to the non-circuit board area, or from the non-circuit board area to the circuit board area. The image to be processed can be divided into several areas according to the image gradient and the preset gradient threshold.

[0046] In one embodiment, the step of determining an area among several areas that meets the image gradient condition as a circuit board area and determining the remaining areas as non-circuit board areas (step S103) includes: determining an area with an image gradient greater than or equal to a preset gradient threshold as a circuit board area; and determining an area with an image gradient less than a preset gradient threshold as a non-circuit board area.

[0047] The image gradient of the circuit board area in the image to be processed is relatively large, and the image gradient of the non-circuit board area is relatively small. When the image gradient of a certain area is greater than the preset gradient threshold, it can be determined that the image gradient of the area is relatively large, that is, the area with the image gradient greater than the preset gradient threshold is determined as the circuit board area; when the image gradient of a certain area is less than the preset gradient threshold, it can be determined that the image gradient of the area is relatively small, that is, the area with the image gradient less than the preset gradient threshold is determined as the non-circuit board area.

[0048] In one embodiment, the preset gradient threshold includes a first preset gradient threshold and a second preset gradient threshold, wherein the first preset gradient threshold is greater than the second preset gradient threshold. Figure 2 , Figure 2 It is a schematic flow chart of another embodiment of the image processing method provided by the present application. Specifically, it may include the following steps S201 to S206.

[0049] Step S201: obtaining an image to be processed, and obtaining image texture features of the image to be processed.

[0050] Step S202: Divide the image to be processed into several regions based on the gradient changes of the image texture features.

[0051] The implementation methods of step S201 and step S202 are respectively the same as those of step S101 and step S102, and are not described in detail herein.

[0052] Step S203: determining a region where the image gradient is greater than or equal to a first preset gradient threshold as a circuit board region.

[0053] Combination Figure 3 , Figure 3 is a schematic diagram of an embodiment of an image to be processed provided by the present application. Figure 3 As shown in the figure, the entire image is the image to be processed. According to the color, the entire image to be processed can be divided into four areas, namely, a white area, a gray area and two black areas. The black part is the circuit board area, the white part is the non-circuit board area, and the gray part is the board edge area of ​​the circuit board. The image gradient of the board area is the largest, the image gradient of the board edge area is the second largest, and the image gradient of the non-circuit board area is the smallest.

[0054] By acquiring the texture features of the image to be processed, the gradients of the images of different parts in the image to be processed can be calculated. The region where the image gradient is detected to be greater than the first preset gradient threshold is determined to be the circuit board region. Figure 3 In the image to be processed, the image gradient of the black area portion of the image is greater than the first preset gradient threshold, that is, the image of the black area portion is detected as the circuit board area.

[0055] Step S204: determining a region where the image gradient is less than a first preset gradient threshold and greater than or equal to a second preset gradient threshold as a board edge region.

[0056] The region where the detected image gradient is less than the first preset gradient threshold and greater than or equal to the second preset gradient threshold is determined as a non-board edge region. Figure 3 In the image to be processed, the image gradient of the gray area portion of the image is less than the first preset gradient threshold and greater than or equal to the second preset gradient threshold, that is, the image of the gray area portion is detected as the edge plate area.

[0057] Step S205: determining the area where the image gradient is less than the second preset gradient threshold as a non-circuit board area.

[0058] The region where the image gradient is detected to be less than the second preset gradient threshold is determined as a non-circuit board region. Figure 3In the image to be processed, the image gradient of the white area portion of the image is less than the second preset gradient threshold, that is, the image of the white area portion is detected as a non-circuit board area.

[0059] Step S206: segmenting the image to be processed according to the non-circuit board area, the circuit board area and the board edge area to obtain the circuit board image.

[0060] After distinguishing the circuit board area, the non-circuit board area, and the board edge area in the image to be processed, the image to be processed can be segmented to obtain a circuit board image that only includes the circuit board area.

[0061] See also Figure 4 , Figure 4 yes Figure 2 The flowchart of an embodiment before step S206 is shown. Specifically, it may include the following steps S401 to S404.

[0062] Step S401: Acquire a first region image of a board edge region.

[0063] The first region image may be an image of the board edge region; or may be a partial image of the board edge region including the first edge pixel point set.

[0064] Step S402: extracting a first edge pixel point set of the first region image.

[0065] The first edge pixel point set is a set of pixel points in the edge area of ​​the first area image. In this embodiment, the first edge pixel point set is a pixel point set that distinguishes the first area image from the circuit board area image.

[0066] Step S403: performing an image processing flow of opening and / or closing operations on the first edge pixel point set to update the first edge pixel point set.

[0067] Both the opening operation and the closing operation use the erosion operation and the dilation operation. The erosion operation includes calculating the local minimum through the convolution operation, and then corroding the foreground image so that the edge of the image shrinks inward. The dilation operation is opposite to the erosion operation, and the local maximum is calculated through the convolution operation, and the foreground image is expanded so that the edge of the image expands outward. In this embodiment, the first edge pixel point is image processed by the opening operation and / or the closing operation, so as to update the first edge pixel point set. The opening operation and the closing operation can be used together. For example, the first edge pixel point can be image processed by the opening operation, the closing operation, and the opening operation again, so as to update the first edge pixel point set.

[0068] Step S404: updating the board edge region according to the updated first edge pixel point set.

[0069] The board edge region is updated using the updated first edge pixel point set, that is, the first edge pixel point set between the board edge region and the circuit board region is updated, so as to facilitate segmentation processing of the board edge region and the circuit board region.

[0070] See also Figure 5 , Figure 5 yes Figure 1 The flowchart of an embodiment before step S104 is shown. Specifically, it may include the following steps S501 to S504.

[0071] Step S501: Acquire a second area image of a circuit board area.

[0072] The second region image may be an image of the circuit board region; or may be a partial image of the circuit board region including the second edge pixel point set.

[0073] Step S502: extracting a second edge pixel point set of the second region image.

[0074] The second edge pixel set is a set of pixels in the edge region of the second region image. In this embodiment, the second edge pixel set is a set of pixels that distinguish the second region image from the non-circuit board region image.

[0075] Step S503: performing an image processing flow of opening and / or closing operations on the second edge pixel point set to update the second edge pixel point set.

[0076] The second edge pixel points are image processed by an opening operation and / or a closing operation, thereby updating the second edge pixel point set. The opening operation and the closing operation can be used together. For example, the second edge pixel points can be image processed by an opening operation, a closing operation, and then an opening operation, thereby updating the second edge pixel point set.

[0077] Step S504: updating the circuit board area according to the updated second edge pixel point set.

[0078] The circuit board area is updated using the updated second edge pixel point set, that is, the second edge pixel point set between the circuit board area and the non-circuit board area is updated, so as to facilitate segmentation processing of the circuit board area and the non-circuit board area.

[0079] See also Figure 6 , Figure 6 This is a schematic diagram of a process flow of an embodiment of determining a circuit board area provided by the present application. Specifically, it may include the following steps S601 to S604.

[0080] Step S601: Determine a region among a plurality of regions that meets the image gradient condition as a first circuit board region.

[0081] After the image to be processed is divided into several regions, a portion of the image with larger image texture is determined as the first circuit board region. That is, there may be multiple circuit board regions in the image to be processed, and the multiple circuit board regions are all determined as the first circuit board region.

[0082] Step S602: when the number of first circuit board regions in the image to be processed is greater than or equal to two, obtaining distances between the first circuit board regions.

[0083] When there are multiple first circuit board areas in the image to be processed, there is a certain distance between two adjacent first circuit board areas. The distance can be the distance between the two nearest points of the two adjacent first circuit board areas, or the distance between the midpoints of the two side lines of the two adjacent first circuit board areas. For example, Figure 3 In the figure, the two black parts are two first circuit board areas respectively, and the distance between the midpoints of two adjacent side lines of the two black parts can be used as the distance between the two first circuit board areas.

[0084] Step S603: merging two or more first circuit board regions whose distance is less than or equal to a preset distance threshold into a second circuit board region.

[0085] When it is detected that the distance between two or more first circuit board areas is less than or equal to a preset distance threshold, the first circuit board area can be identified as one circuit board area, and the two or more first circuit boards with a distance less than the preset distance threshold are merged into a second circuit board area. In other words, the multiple first circuit board areas are regarded as a whole as the second circuit board area.

[0086] Step S604: Determine the merged second circuit board area and the remaining first circuit board area as the circuit board area.

[0087] After at least part of the first circuit board area is merged, the image to be processed may contain the first circuit board area, the second circuit board area, or both the first circuit board area and the second circuit board area. The first circuit board area and / or the second circuit board area of ​​the image to be processed are simultaneously determined as the circuit board area, so as to facilitate segmentation of the circuit board area and the non-circuit board area.

[0088] See also Figure 7 , Figure 7 This is a schematic diagram of a flow chart of an embodiment of determining a circuit board area provided by the present application. Specifically, it may include the following steps S701 to S704.

[0089] Step S701: Determine an area among a plurality of areas that meets the image gradient condition as a third circuit board area.

[0090] After the image to be processed is divided into several regions, the image with larger image texture is determined as the third circuit board region. The image texture of the third circuit board region meets the image gradient condition.

[0091] Step S702: Detect whether the area size of the third circuit board area is greater than or equal to a preset size threshold.

[0092] The third circuit board area includes the circuit board area. In some possible cases, there may be an area in the non-circuit board area that meets the image gradient condition, but if it is determined that the area is a circuit board area, the final segmented image will still include the non-circuit board area image. In order to avoid the above situation, it can be determined whether the area size of each third circuit board area is greater than or equal to a preset size threshold, so as to determine whether it is an image of the non-circuit board area by the size of the area of ​​the third circuit board.

[0093] Step S703: If not, delete the third circuit board area whose size is smaller than the preset size threshold.

[0094] In some embodiments, the third circuit board area may include an interference area in the circuit board area and the non-circuit board area, the interference area meets the image gradient condition, but it is not the circuit board area, and the size of the interference area is smaller than the size of the circuit board area. When it is detected that the area size of the third circuit board is smaller than the preset size threshold, the area smaller than the preset size threshold is deleted.

[0095] Step S704: determining the remaining third circuit board area after the size detection as the circuit board area.

[0096] After deleting the third circuit board area whose size is smaller than the preset size threshold, the remaining third circuit board area in the image to be processed is the circuit board area, so as to facilitate segmentation processing of the circuit board area and the non-circuit board area.

[0097] See also Figure 8 , Figure 8 This is a flow chart of an embodiment of the abnormal region detection method provided by the present application. Specifically, it may include the following steps S801 to S804.

[0098] Step S801: Acquire a circuit board image, wherein the circuit board image is obtained by an image processing method. The circuit board image can be obtained by the image processing method of any of the above embodiments.

[0099] Step S802: Acquire the repeating unit region.

[0100] The repeating unit region may be an image framed by a rectangular frame or other shape, and the similarity condition is the similarity between an image framed by a repeating unit region and an image framed by another repeating unit region adjacent thereto. When the similarity condition is satisfied, the repeating unit region may be identified as a repeating unit region that meets the similarity condition. Exemplarily, in the image to be detected, the first repeating unit region includes the first image, and the second repeating unit region adjacent to the first repeating unit region is selected in the image to be detected using the same size as the first repeating unit region, and the similarity between the first repeating unit region and the second repeating unit region is compared. When the similarity meets the condition, the repeating unit region is a repeating unit region that meets the similarity condition.

[0101] Step S803: using the repeated unit region to perform traversal matching on all regions including the circuit board image, and obtaining the similarity between the repeated unit region and each region.

[0102] By using repeated unit areas to traverse and match all areas of the image to be inspected including the circuit board, a size frame corresponding to a number of non-overlapping repeated unit areas can be obtained, and a grid image to be inspected is divided. The size frame of each repeated unit area selects a regional image of the image to be inspected to obtain the similarity between the repeated unit area and each area selected by the size frame of the repeated unit area.

[0103] Step S804: marking the region in the circuit board image whose similarity is less than the first similarity threshold as an abnormal region.

[0104] When the similarity between the repeating unit region and the region selected by the size frame of the repeating unit region is less than the first similarity threshold, the partial region is an abnormal region, otherwise it is a non-abnormal region. Exemplarily, when the first similarity threshold is 0.9, the region selected by the size frame of the repeating unit region includes four regions, namely the first region, the second region, the third region and the fourth region, the similarities between the first region, the second region and the third region and the repeating unit are 0.91, 0.92 and 0.93 respectively, and the similarities between the fourth region and the repeating unit are 0.71 respectively, so the fourth region can be identified as an abnormal region.

[0105] Through the above implementation, the obtained repetitive unit area is matched with the image to be detected, and the matched repetitive unit is compared with the first similarity to find the abnormal area. This method can use the repetitive unit area to find the abnormal area in the image to be detected with a high degree of repetition, thereby simplifying the complexity of the algorithm; at the same time, it can accurately find the abnormal area from the image to be detected with a high degree of repetition, so as to facilitate the subsequent image processing using the found abnormal area.

[0106] See also Fig. 9 , Fig. 9 yes Figure 8 Specifically, the process may include the following steps S901 to S904.

[0107] Step S901: Acquire the first starting point in the image to be detected.

[0108] The first starting point can be obtained manually, for example, the first starting point is determined manually in the image to be detected. The first starting point can also be obtained automatically, for example, the center point of the image to be detected is the first starting point, or the first detection image is divided into nine equal areas, and the intersection between the areas is the first starting point. The first starting point can be a corner point of the repeating unit area.

[0109] Step S902: Generate a first repeating unit region based on a first starting point and in accordance with a first preset size.

[0110] When the repeating unit region is a rectangular frame, the first preset size may include the length and width of the repeating unit, and after determining the length, width and first starting point of the repeating unit region, the first repeating unit region may be generated.

[0111] Step S903: using the first repeating unit region to obtain a first adjacent region having the same size as the first repeating unit region, and obtaining a first similarity between the first repeating unit region and the first adjacent region.

[0112] After determining the size of the first repeating unit area, a first adjacent area with the same size as the first repeating unit area can be obtained, the first adjacent area is adjacent to the first repeating unit area, and a partial image area is framed from the image to be detected using the same size as the first repeating unit, and a first similarity between the first repeating unit area and the first adjacent area is obtained, that is, a first similarity between the image in the first repeating unit area and the image in the first adjacent area is obtained. For example, see Fig.10 , Fig.10 In the figure, a is the first repeating unit region, b is the first adjacent region, and the first repeating unit region and the first adjacent region have the same size, that is, the length and width of the first repeating unit region and the first adjacent region are approximately 100 and 70 respectively.

[0113] Step S904: when the first similarity is greater than or equal to the second similarity threshold, the first repeating unit region is regarded as a repeating unit region that meets the similarity condition.

[0114] When the first similarity is greater than or equal to the second similarity threshold, the first repeating unit region is determined to meet the similarity condition, and the first repeating unit region can be used to perform subsequent processing. Fig.10In the example, the first repeating unit region and the first adjacent region are relatively similar, and the first similarity may be about 0.99. When the second similarity threshold includes 0.95, it is determined that the first similarity is greater than the second similarity threshold, and the Fig.10 Figure a in the figure is identified as the first repeating unit region that meets the similarity conditions.

[0115] In one embodiment, after obtaining the first similarity between the first repeating unit region and the first adjacent region (step S903), the abnormal region detection method further includes: generating a second repeating unit region based on the first starting point according to a second preset size when the first similarity is less than or equal to a second similarity threshold; repeatedly executing: using the second repeating unit region to obtain a second adjacent region of the same size, obtaining a second similarity between the second repeating unit region and the second adjacent region, until a second repeating unit region having a second similarity greater than or equal to the second similarity threshold is obtained as a repeating unit region that meets the similarity condition.

[0116] When the first similarity is less than the second similarity threshold, it indicates that the first repeating unit region does not meet the similarity condition, and the first repeating unit region cannot be used as the repeating unit region for subsequent processing. At this time, the first starting point can be used to generate the second repeating unit region according to the second preset size, wherein the second preset size is different from the first preset size. When the second repeating unit region meets the similarity condition, the second repeating unit region can be used as the repeating unit region for subsequent processing. When the second repeating unit also does not meet the similarity condition, the first starting point can be used to generate the third repeating unit region according to the third preset size until a repeating unit region that meets the similarity condition is finally found.

[0117] In one embodiment, after obtaining the first similarity between the first repeating unit region and the first adjacent region (step S903), the image processing method further includes: when the first similarity is less than a third similarity threshold, selecting a second starting point to replace the first starting point in the image to be detected; wherein the third similarity threshold is less than the second similarity threshold.

[0118] When the first similarity is less than the third similarity threshold, it may be because the first starting point is located near the abnormal area, and the abnormal area is selected according to the first repeating unit generated by the first starting point. At this time, it is still difficult to generate a first repeating unit that meets the similarity condition with the first starting point. In order to avoid the above situation, the second starting point can be selected to replace the first starting point. The second starting point can be obtained in the same way as the first starting point.

[0119] See also Fig.11 , Fig.11 yes Fig. 9Specifically, the steps after step S903 may include the following steps S1101 to S1103.

[0120] Step S1101: Generate a second repeating unit region based on the first starting point and in accordance with a second preset size.

[0121] After obtaining the first similarity between the first repeating unit region and the first adjacent region, the first repeating unit is not directly used as the repeating unit region meeting the similarity condition, but a second repeating unit region is first generated based on the first starting point according to the second preset size.

[0122] Step S1102: using the second repeating unit region to obtain a second adjacent region of the same size, and obtaining a second similarity between the second repeating unit region and the second adjacent region.

[0123] After determining the size of the second repeating unit area, a second adjacent area with the same size as the second repeating unit area can be obtained, the second adjacent area is adjacent to the second repeating unit area, and a partial image area is framed from the image to be detected using the same size as the second repeating unit, and a second similarity between the second repeating unit area and the second adjacent area is obtained, that is, a second similarity between the image in the second repeating unit area and the image in the second adjacent area is obtained. For example, see Fig.12 , Fig.12 a is the second repeating unit region, b is the second adjacent region, and the second repeating unit region and the second adjacent region have the same size, that is, the length and width of the second repeating unit region and the second adjacent region are approximately 100 and 60 respectively.

[0124] Step S1103: taking the repeating unit region corresponding to the greatest similarity between the first similarity and the second similarity as the repeating unit region meeting the similarity condition.

[0125] Among them, at least one of the first repeating unit region corresponding to the first similarity and the repeating unit region corresponding to the second similarity meets the similarity condition. The first similarity and the second similarity are compared, and the repeating unit region corresponding to the larger one of the first similarity and the second similarity is taken as the repeating unit region that meets the similarity condition. For example, see Fig.10 and Fig.12 , Fig.10 Where a is the first repeating unit region, b is the first adjacent region, and Fig.10 The first similarity between the first repeating unit region and the first adjacent region in is high, exemplarily, the first similarity is 0.99; Fig.12 Where a is the second repeating unit region, b is the second adjacent region, and Fig.12The second similarity between the second repeating unit region and the adjacent region in is relatively low, and exemplarily, the second similarity is 0.89. In this case, the first similarity is greater than the second similarity, and Fig.10 The first repeating unit region in is taken as the repeating unit region that meets the similarity condition.

[0126] See also Fig.13 , Fig.13 yes Fig. 9 Another embodiment of the process flow diagram of the steps after step S903 in the embodiment of the present invention can include the following steps 1301 to S1303.

[0127] Step S1301: Determine whether the first similarity and the second similarity are both greater than or equal to a second similarity threshold.

[0128] At least one of the first similarity and the second similarity is greater than or equal to the second similarity threshold. Exemplarily, it includes: the first similarity is greater than or equal to the second similarity threshold, and the second similarity is less than the second similarity threshold; the first similarity is less than the second similarity threshold, and the second similarity is greater than or equal to the second similarity threshold; the first similarity and the second similarity are both greater than or equal to the second similarity threshold.

[0129] Step S1302: If not, the repetitive unit regions corresponding to the similarity greater than or equal to the second similarity threshold are regarded as repetitive unit regions meeting the similarity condition.

[0130] The comparison result of the first similarity and the second similarity with the second similarity threshold includes: the first similarity is greater than or equal to the second similarity threshold, the second similarity is less than the second similarity threshold, or the first similarity is less than the second similarity threshold, and the second similarity is greater than or equal to the second similarity threshold. Then, a region meeting the similarity condition is selected from the first repeating unit region and the second repeating unit region as the repeating unit region.

[0131] Step S1303: If yes, the repeating unit region with the largest preset size is taken as the repeating unit region that meets the similarity condition.

[0132] The comparison result of the first similarity and the second similarity with the second similarity threshold includes: the first similarity and the second similarity are both greater than or equal to the second similarity threshold. When the first similarity and the second similarity are both greater than or equal to the second similarity threshold, both cannot be used at the same time as the repeating unit region that meets the similarity condition, but one of the first repeating region and the second repeating region needs to be selected as the repeating unit region that meets the similarity condition. In this case, the larger repeating unit can be selected from the two as the repeating unit region that meets the similarity condition. For example, see Fig.10 and Fig.12 , Fig.10a is the first repeating unit region, b is the first adjacent region, and the length and width of the first repeating unit region are approximately 100 and 70, respectively; Fig.12 Where a is the second repeating unit region, b is the second adjacent region, and the length and width of the second repeating unit region are approximately 100 and 60, respectively, and when both the first similarity and the second similarity are greater than or equal to the second similarity threshold, select Fig.10 The first repeating unit region in is taken as the repeating unit region that meets the similarity condition.

[0133] See also Fig.14 , Fig.14 yes Figure 8 Specifically, the steps before step S104 may include the following steps S1401 to S1403.

[0134] Step S1401: Create a grid image using all region coordinates and similarities of the image to be detected.

[0135] After traversing and matching all regions including the image to be detected using the repeating unit region, the similarity between the repeating unit region and each region can be obtained. Since regions that are exactly the same as the repeating unit exist in different positions of the image to be detected, the similarity gradually decreases from the maximum value when the first repeating unit is found until the second repeating unit is encountered, and the maximum value is reached again, and this cycle repeats. The two-dimensional matrix composed of similarities is visualized as a grid image, and the points on the intersecting grid lines in the grid image are the coordinates of the upper left corner of each region.

[0136] Step S1402: Obtain a similarity histogram of the grid images according to the similarity distribution of the grid images.

[0137] According to the similarity distribution in the grid image, the similarity histogram in the grid image can be obtained, such as Fig.15 As shown, the minimum similarity histogram can be bimodal, where Fig.15 The horizontal axis of the similarity histogram in is the similarity value, and the vertical axis is the number of similarity values.

[0138] Step S1403: using the distribution of the similarity histogram, determine a first similarity threshold according to a preset threshold algorithm.

[0139] The preset threshold algorithm includes an OTSU (maximum between-class variance) algorithm, and the first similarity threshold can be determined by using the distribution of the obtained similarity histogram. Different similarity histograms have different values ​​for determining the first similarity threshold. For example, Fig.15 In the similarity histogram shown, the first similarity threshold may be 0.86.

[0140] See also Fig.16 , Fig.16 yes Figure 8 The flowchart of an embodiment of the steps after step S804 is shown in FIG.

[0141] Step S1601: Obtain a design drawing of a circuit board, and obtain a mapping matrix between the design drawing and the image to be detected.

[0142] The design drawing of the circuit board can be to import the circuit diagram of the circuit board into the software, and the design drawing of the circuit board can be automatically generated by the software based on the circuit diagram. After obtaining the design drawing of the circuit board, a circuit board identical to the design drawing of the circuit board can be generated according to the design drawing of the circuit board. The points of the image to be detected can be mapped to the design drawing using a mapping matrix.

[0143] Step S1602: using the mapping matrix to obtain the circuit element region of the abnormal region in the design diagram.

[0144] The abnormal area is marked in the image to be detected using the above steps, and after obtaining the mapping matrix between the image to be detected and the design drawing, the circuit component area corresponding to the abnormal area in the image to be detected and the abnormal area in the design drawing can be obtained based on the mapping matrix and the abnormal area.

[0145] Step S1603: using the circuit element area and the abnormal area, aligning the image to be inspected and the design drawing of the circuit board.

[0146] After determining the circuit element area and the abnormal area, the image to be inspected and the design drawing of the circuit board can be aligned. Therefore, this method can use the repeated unit area to find the abnormal area of ​​the image to be inspected with a high degree of repetition, simplifying the complexity of the algorithm; at the same time, the abnormal area can be accurately found from the image to be inspected with a high degree of repetition, making it easy to use the found abnormal area to align with the design drawing of the circuit board, and facilitating defect detection of the image to be inspected.

[0147] The above method is applied in automatic optical inspection equipment. Fig.17 , Fig.17 1 is a schematic diagram of the structure of an embodiment of the automatic optical inspection device of the present application. The automatic optical inspection device 90 of the present embodiment includes a processor 91 and a memory 92. The memory 92 stores a computer program, and the processor 91 is used to execute the computer program to implement the above-mentioned image processing method and abnormal area detection method.

[0148] The processor 91 may be an integrated circuit chip with signal processing capability. The processor 91 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0149] for Figure 1-Figure 16 The image processing method and abnormal region detection method of the illustrated embodiment can be presented in the form of a computer program. The present application proposes a computer storage medium carrying the computer program. Fig.18 , Fig.18 1 is a schematic diagram of the structure of an embodiment of a computer storage medium of the present application. The computer storage medium 100 of this embodiment includes a computer program 110, which can be executed to implement the above-mentioned image processing method and abnormal area detection method.

[0150] The computer storage medium 100 of this embodiment can be a medium that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, or it can also be a server that stores the program instructions. The server can send the stored program instructions to other devices for execution, or it can also execute the stored program instructions by itself.

[0151] In addition, if the above functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a storage medium readable by a mobile terminal, that is, the present application also provides a storage device storing program data, the program data can be executed to implement the method of the above embodiment, and the storage device can be, for example, a USB flash drive, an optical disk, a server, etc. In other words, the present application can be embodied in the form of a software product, which includes a number of instructions for enabling a smart terminal to execute all or part of the steps of the method described in each embodiment.

[0152] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other.

[0153] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0154] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0155] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (which can be a personal computer, server, network device or other system that can fetch instructions from the instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0156] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, and acquire image texture features of the image to be processed; Based on the gradient change of the image texture feature, the image to be processed is divided into a number of regions; Determine the area in the plurality of areas that meets the image gradient condition as the circuit board area, and determine the remaining areas as non-circuit board areas; Segmenting the image to be processed according to the circuit board area to obtain a circuit board image; The step of determining a region among the plurality of regions that meets the image gradient condition as a circuit board region includes: Determine a region among the plurality of regions that meets the image gradient condition as a first circuit board region; When the number of first circuit board areas in the image to be processed is greater than or equal to two, obtaining distances between the first circuit board areas; Merging two or more first circuit board areas whose distance is less than or equal to a preset distance threshold into a second circuit board area; The merged second circuit board area and the remaining first circuit board area are determined as the circuit board area.

2. The image processing method according to claim 1, characterized in that: The step of dividing the image to be processed into a plurality of regions based on the gradient change of the image texture feature comprises: Based on the image texture feature, calculating the image gradient of the image to be processed; The image to be processed is divided into a plurality of regions according to the image gradient of the image to be processed and a preset gradient threshold.

3. The image processing method according to claim 2, characterized in that: The step of determining the area among the plurality of areas that meets the image gradient condition as the circuit board area, and determining the remaining areas as the non-circuit board area, comprises: Determine the area where the image gradient is greater than or equal to the preset gradient threshold as the circuit board area; The area where the image gradient is less than the preset gradient threshold is determined as the non-circuit board area.

4. The image processing method according to claim 2 or 3, characterized in that: The preset gradient threshold includes a first preset gradient threshold and a second preset gradient threshold, wherein the first preset gradient threshold is greater than the second preset gradient threshold; and determining the area satisfying the image gradient condition among the plurality of areas as a circuit board area and determining the remaining areas as non-circuit board areas, comprises: Determine an area where the image gradient is greater than or equal to the first preset gradient threshold as the circuit board area; Determine a region where the image gradient is less than the first preset gradient threshold and greater than or equal to the second preset gradient threshold as a board edge region; Determine the area where the image gradient is less than the second preset gradient threshold as the non-circuit board area; The step of segmenting the image to be processed according to the circuit board area to obtain a circuit board image includes: The image to be processed is segmented according to the non-circuit board area, the circuit board area and the board edge area to obtain the circuit board image.

5. The image processing method according to claim 4, characterized in that: Before segmenting the image to be processed according to the non-circuit board area, the circuit board area and the board edge area to obtain the circuit board image, the image processing method further includes: Acquire a first area image of the board edge area; Extracting a first edge pixel point set of the first area image; An image processing process of performing an opening operation and / or a closing operation on the first edge pixel point set to update the first edge pixel point set; The board edge region is updated according to the updated first edge pixel point set.

6. The image processing method according to claim 1, characterized in that: Before segmenting the image to be processed according to the circuit board area to obtain the circuit board image, the image processing method further includes: Acquire a second area image of the circuit board area; Extracting a second edge pixel point set of the second region image; An image processing process of performing an opening operation and / or a closing operation on the second edge pixel point set to update the second edge pixel point set; The circuit board area is updated according to the updated second edge pixel point set.

7. The image processing method according to claim 1, characterized in that: The step of determining a region among the plurality of regions that meets the image gradient condition as a circuit board region comprises: Determine an area among the plurality of areas that meets the image gradient condition as a third circuit board area; Detecting whether the area size of the third circuit board area is greater than or equal to a preset size threshold; If not, deleting the third circuit board area whose size is smaller than the preset size threshold; The remaining third circuit board area after the size detection is determined as the circuit board area.

8. A method for detecting abnormal areas, characterized in that: The abnormal area detection method comprises: Acquire a circuit board image, wherein the circuit board image is obtained by processing the image processing method according to any one of claims 1 to 7; Get the repeating unit region; Using the repeating unit region to traverse and match all regions including the circuit board image, and obtaining the similarity between the repeating unit region and each region; The region in the circuit board image whose similarity is less than the first similarity threshold is marked as an abnormal region.

9. An automatic optical inspection device, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images