Abnormal area detection method, automatic optical detection equipment and storage medium

Abstract

The present application discloses an abnormal region detection method, an automatic optical detection device, and a storage medium. The abnormal region detection method includes: obtaining a first starting point in an image to be detected; generating a first repeating unit region based on the first starting point according to a first preset size; using the first repeating unit region to obtain a first adjacent region of the same size, and obtaining a first similarity between the first repeating unit region and the first adjacent region; when the first similarity is greater than or equal to a second similarity threshold, using the first repeating unit region as a repeating unit region that meets the similarity condition. Through the above implementation, the repeating unit region can be used to find abnormal regions in an image to be detected with a high degree of repetition, thereby simplifying the complexity of the algorithm; at the same time, the abnormal region can be accurately found from the image to be detected with a high degree of repetition, so that the found abnormal region can be used for subsequent image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an abnormal area detection method, an automatic optical detection device, and a storage medium. Background Art

[0002] Automated Optical Inspection (AOI) equipment in the printed circuit board (PCB) industry has very important practical significance. AOI refers to the detection and location of defects on PCBs on PCB production lines by collecting images through optical cameras and using image processing, machine learning and other methods.

[0003] There are inevitably a lot of defects in the PCB production process, and the circuits on the PCB are very small and dense. It is very time-consuming and laborious to check the defects manually, and it is easy to make mistakes. In addition, if the defects are not detected and the defective PCB boards are directly put into the next process for manufacturing, the subsequent repair costs will become higher and higher, and they are more likely to be scrapped, resulting in a lot of waste costs.

[0004] However, during the inspection process, the original PCB design and the image captured by the camera must first be aligned before they can be compared and then defect detection can be performed. For PCBs such as LED light boards, which are difficult to extract features and have high repetition, the existing alignment algorithms have poor performance, resulting in a large number of false point defect reports due to alignment failures, which require manual re-inspection. Therefore, optimizing the alignment algorithm has important practical significance for the accuracy of defect detection. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present application provides an abnormal area detection method, an automatic optical detection device and a storage medium.

[0006] To solve the above problems, the present application provides an abnormal area detection method, which includes: obtaining a first starting point in an image to be detected; generating a first repeating unit area based on the first starting point according to a first preset size; using the first repeating unit area to obtain a first adjacent area of ​​the same size, and obtaining a first similarity between the first repeating unit area and the first adjacent area; when the first similarity is greater than or equal to a second similarity threshold, using the first repeating unit area as a repeating unit area that meets the similarity condition.

[0007] To solve the above problems, the present application provides an automatic optical inspection device, which includes: a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the computer program to implement the above method.

[0008] To solve the above problems, the present application provides a computer-readable storage medium on which program instructions are stored, and the above method is implemented when the program instructions are executed by a processor.

[0009] Compared with the prior art, the abnormal area detection method of the present application includes: obtaining a first starting point in the image to be detected; generating a first repeating unit area based on the first starting point according to a first preset size; using the first repeating unit area to obtain a first adjacent area of ​​the same size, and obtaining a first similarity between the first repeating unit area and the first adjacent area; when the first similarity is greater than or equal to a second similarity threshold, the first repeating unit area is used as a repeating unit area that meets the similarity condition. Through the above implementation, the obtained repeating unit area is matched with the image to be detected, and the matched repeating unit is compared with the first similarity to find the abnormal area. This method can use the repeating unit area to find the abnormal area of ​​the image to be detected with a high degree of repetition, simplifying the complexity of the algorithm; at the same time, it can accurately find the abnormal area from the image to be detected with a high degree of repetition, so as to facilitate the use of the found abnormal area for subsequent image processing.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the abnormal area detection method provided by the present application;

[0013] Figure 2 is a schematic diagram of an embodiment of the first repeating unit region and the first adjacent region in the present application;

[0014] Figure 3 yes Figure 1 A flowchart of an embodiment of the steps after step S103;

[0015] Figure 4This is a schematic diagram of an embodiment of the second repeating unit region and the second adjacent region in the present application;

[0016] Figure 5 yes Figure 1 A flowchart of another embodiment of the steps after step S103;

[0017] Figure 6 yes Figure 1 A flowchart of an embodiment of the steps before step S106;

[0018] Figure 7 is a schematic diagram of an embodiment of a similarity histogram provided by the present application;

[0019] Figure 8 yes Figure 1 A flowchart of an embodiment of the steps after step S106;

[0020] Fig. 9 yes Figure 1 A flowchart of an embodiment of the steps before step S101;

[0021] Fig.10 yes Figure 1 A flowchart of another embodiment of the steps before step S101;

[0022] Fig.11 is a schematic diagram of an embodiment of an image to be processed provided by the present application;

[0023] Fig.12 yes Fig. 9 A schematic diagram of a flow chart of an embodiment before step S1106 is shown;

[0024] Fig.13 yes Fig. 9 A schematic diagram of a flow chart of an embodiment before step S1004 is shown;

[0025] Fig.14 It is a schematic diagram of a flow chart of an embodiment of determining a circuit board area provided by the present application;

[0026] Fig.15 is a schematic diagram of another embodiment of the process of determining the circuit board area provided by the present application;

[0027] Fig.16 It is a structural schematic diagram of an embodiment of the automatic optical inspection device of the present application;

[0028] Fig.17 It is a structural diagram of an embodiment of the computer storage medium of the present application. DETAILED DESCRIPTION

[0029] The present application is further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present application, but are not intended to limit the scope of the present application. Similarly, the following examples are only some embodiments of the present application rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present application.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] In the description of this application, it is necessary to explain that, unless otherwise clearly specified and limited, the terms "install", "set", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or connected through an intermediate medium. For ordinary technicians in this field, the above-mentioned specific meanings belonging to this application can be connected according to specific circumstances.

[0032] This application provides a method for detecting abnormal areas. Figure 1 , Figure 1 1 is a flow chart of an embodiment of the abnormal region detection method provided by the present application. Specifically, it may include the following steps S101 to S106.

[0033] Step S101: Acquire the first starting point in the image to be detected.

[0034] The first starting point can be obtained manually, for example, the first starting point is determined manually in the image to be detected. The first starting point can also be obtained automatically, for example, the center point of the image to be detected is the first starting point, or the first detection image is divided into nine equal areas, and the intersection between the areas is the first starting point. The first starting point can be a corner point of the repeating unit area.

[0035] Step S102: generating a first repeating unit region based on a first starting point and in accordance with a first preset size.

[0036] The first repeating unit area can be an image framed by a shape such as a rectangular frame. When the repeating unit area is a rectangular frame, the first preset size can include the length and width of the repeating unit. After determining the length size, width size and first starting point of the repeating unit area, the first repeating unit area can be generated.

[0037] Step S103: using the first repeating unit region to obtain a first adjacent region having the same size as the first repeating unit region, and obtaining a first similarity between the first repeating unit region and the first adjacent region.

[0038] After determining the size of the first repeating unit area, a first adjacent area with the same size as the first repeating unit area can be obtained, the first adjacent area is adjacent to the first repeating unit area, and a partial image area is framed from the image to be detected using the same size as the first repeating unit, and a first similarity between the first repeating unit area and the first adjacent area is obtained, that is, a first similarity between the image in the first repeating unit area and the image in the first adjacent area is obtained. For example, see Figure 2 , Figure 2 In the figure, a is the first repeating unit region, b is the first adjacent region, and the first repeating unit region and the first adjacent region have the same size, that is, the length and width of the first repeating unit region and the first adjacent region are approximately 100 and 70 respectively.

[0039] Step S104: when the first similarity is greater than or equal to the second similarity threshold, the first repeating unit region is regarded as a repeating unit region that meets the similarity condition.

[0040] The similarity condition is the similarity between an image framed by a repeating unit region and an image framed by another repeating unit region adjacent to the repeating unit region. When the similarity condition is met, the repeating unit region can be determined to be a repeating unit region that meets the similarity condition. Exemplarily, in the image to be detected, the first repeating unit region includes the first image, and the second repeating unit region adjacent to the first repeating unit region is selected in the image to be detected using the same size as the first repeating unit region. The similarity between the first repeating unit region and the second repeating unit region is compared. When the similarity meets the condition, the repeating unit region is determined to be a repeating unit region that meets the similarity condition. When the first similarity is greater than or equal to the second similarity threshold, it is determined that the first repeating unit region meets the similarity condition, and the first repeating unit region can be used to perform subsequent steps. Exemplarily, in Figure 2 In the example, the first repeating unit region and the first adjacent region are relatively similar, and the first similarity may be about 0.99. When the second similarity threshold includes 0.95, it is determined that the first similarity is greater than the second similarity threshold, and the Figure 2 The a in the figure is identified as the first repeating unit region that meets the similarity conditions.

[0041] In one embodiment, after obtaining the first similarity between the first repeating unit region and the first adjacent region (step S103), the abnormal region detection method further includes: generating a second repeating unit region based on the first starting point according to a second preset size when the first similarity is less than or equal to a second similarity threshold; and repeatedly executing: using the second repeating unit region to obtain a second adjacent region of the same size, and obtaining a second similarity between the second repeating unit region and the second adjacent region, until a second repeating unit region with a second similarity greater than or equal to the second similarity threshold is obtained as a repeating unit region that meets the similarity condition.

[0042] When the first similarity is less than the second similarity threshold, it indicates that the first repeating unit region does not meet the similarity condition, and the first repeating unit region cannot be used as the repeating unit region for subsequent processing. At this time, the first starting point can be used to generate the second repeating unit region according to the second preset size, wherein the second preset size is different from the first preset size. When the second repeating unit region meets the similarity condition, the second repeating unit region can be used as the repeating unit region for subsequent processing. When the second repeating unit also does not meet the similarity condition, the first starting point can be used to generate the third repeating unit region according to the third preset size until a repeating unit region that meets the similarity condition is finally found.

[0043] In one embodiment, after obtaining the first similarity between the first repeating unit region and the first adjacent region (step S103), the image processing method further includes: when the first similarity is less than a third similarity threshold, selecting a second starting point to replace the first starting point in the image to be detected; wherein the third similarity threshold is less than the second similarity threshold.

[0044] When the first similarity is less than the third similarity threshold, it may be because the first starting point is located near the abnormal area, and the abnormal area is selected according to the first repeating unit generated by the first starting point. At this time, it is still difficult to generate a first repeating unit that meets the similarity condition with the first starting point. In order to avoid the above situation, the second starting point can be selected to replace the first starting point. The second starting point can be obtained in the same way as the first starting point.

[0045] Step S105: using the repeated unit region to perform traversal matching on all regions of the image to be inspected including the circuit board, and obtaining the similarity between the repeated unit region and each region.

[0046] By using repeated unit areas to traverse and match all areas of the image to be inspected including the circuit board, a size frame corresponding to a number of non-overlapping repeated unit areas can be obtained, and a grid image to be inspected is divided. The size frame of each repeated unit area selects a regional image of the image to be inspected to obtain the similarity between the repeated unit area and each area selected by the size frame of the repeated unit area.

[0047] Step S106: Mark the region whose similarity is less than the first similarity threshold as an abnormal region.

[0048] When the similarity between the repeating unit region and the region selected by the size frame of the repeating unit region is less than the first similarity threshold, the partial region is an abnormal region, otherwise it is a non-abnormal region. Exemplarily, when the first similarity threshold is 0.9, the region selected by the size frame of the repeating unit region includes four regions, namely the first region, the second region, the third region and the fourth region, the similarities between the first region, the second region and the third region and the repeating unit are 0.91, 0.92 and 0.93 respectively, and the similarities between the fourth region and the repeating unit are 0.71 respectively, so the fourth region can be identified as an abnormal region.

[0049] Through the above implementation, the obtained repetitive unit area is matched with the image to be detected, and the matched repetitive unit is compared with the first similarity to find the abnormal area. This method can use the repetitive unit area to find the abnormal area in the image to be detected with a high degree of repetition, thereby simplifying the complexity of the algorithm; at the same time, it can accurately find the abnormal area from the image to be detected with a high degree of repetition, so as to facilitate the subsequent image processing using the found abnormal area.

[0050] See also Figure 3 , Figure 3 yes Figure 1 Schematic diagram of an embodiment of the process flow of the steps after step S103. Specifically, it can include the following steps S401 to S403.

[0051] Step S401: generating a second repeating unit region based on the first starting point and in accordance with a second preset size.

[0052] After obtaining the first similarity between the first repeating unit region and the first adjacent region, the first repeating unit is not directly used as the repeating unit region meeting the similarity condition, but a second repeating unit region is first generated based on the first starting point according to the second preset size.

[0053] Step S402: using the second repeating unit region to obtain a second adjacent region having the same size as the second repeating unit region, and obtaining a second similarity between the second repeating unit region and the second adjacent region.

[0054] After determining the size of the second repeating unit area, a second adjacent area with the same size as the second repeating unit area can be obtained, the second adjacent area is adjacent to the second repeating unit area, and a partial image area is framed from the image to be detected using the same size as the second repeating unit, and a second similarity between the second repeating unit area and the second adjacent area is obtained, that is, a second similarity between the image in the second repeating unit area and the image in the second adjacent area is obtained. For example, see Figure 4 , Figure 4 a is the second repeating unit region, b is the second adjacent region, and the second repeating unit region and the second adjacent region have the same size, that is, the length and width of the second repeating unit region and the second adjacent region are approximately 100 and 60 respectively.

[0055] Step S403: taking the repeating unit region corresponding to the greatest similarity between the first similarity and the second similarity as the repeating unit region meeting the similarity condition.

[0056] Among them, at least one of the first repeating unit region corresponding to the first similarity and the repeating unit region corresponding to the second similarity meets the similarity condition. The first similarity and the second similarity are compared, and the repeating unit region corresponding to the larger one of the first similarity and the second similarity is taken as the repeating unit region that meets the similarity condition. For example, see Figure 2 and Figure 4 , Figure 2 Where a is the first repeating unit region, b is the first adjacent region, and Figure 2 The first similarity between the first repeating unit region and the first adjacent region in is high, exemplarily, the first similarity is 0.99; Figure 2 Where a is the second repeating unit region, b is the second adjacent region, and Figure 4 The second similarity between the second repeating unit region and the adjacent region in is relatively low, and exemplarily, the second similarity is 0.89. In this case, the first similarity is greater than the second similarity, and Figure 2 The first repeating unit region in is taken as the repeating unit region that meets the similarity condition.

[0057] See also Figure 5 , Figure 5 yes Figure 1 Another embodiment of the process flow diagram of the steps after step S103 in the embodiment of the present invention can include the following steps 601 to S603.

[0058] Step S601: Determine whether the first similarity and the second similarity are both greater than or equal to a second similarity threshold.

[0059] At least one of the first similarity and the second similarity is greater than or equal to the second similarity threshold. Exemplarily, it includes: the first similarity is greater than or equal to the second similarity threshold, and the second similarity is less than the second similarity threshold; the first similarity is less than the second similarity threshold, and the second similarity is greater than or equal to the second similarity threshold; the first similarity and the second similarity are both greater than or equal to the second similarity threshold.

[0060] Step S602: If not, the repetitive unit regions corresponding to the similarity greater than or equal to the second similarity threshold are regarded as repetitive unit regions meeting the similarity condition.

[0061] The comparison result of the first similarity and the second similarity with the second similarity threshold includes: the first similarity is greater than or equal to the second similarity threshold, the second similarity is less than the second similarity threshold, or the first similarity is less than the second similarity threshold, and the second similarity is greater than or equal to the second similarity threshold. Then, a region meeting the similarity condition is selected from the first repeating unit region and the second repeating unit region as the repeating unit region.

[0062] Step S603: If yes, the repeating unit region with the largest preset size is used as the repeating unit region that meets the similarity condition.

[0063] The comparison result of the first similarity and the second similarity with the second similarity threshold includes: the first similarity and the second similarity are both greater than or equal to the second similarity threshold. When the first similarity and the second similarity are both greater than or equal to the second similarity threshold, both cannot be used at the same time as the repeating unit region that meets the similarity condition, but one of the first repeating region and the second repeating region needs to be selected as the repeating unit region that meets the similarity condition. In this case, the larger repeating unit can be selected from the two as the repeating unit region that meets the similarity condition. For example, see Figure 2 and Figure 4 , Figure 2 a is the first repeating unit region, b is the first adjacent region, and the length and width of the first repeating unit region are approximately 100 and 70, respectively; Figure 4 Where a is the second repeating unit region, b is the second adjacent region, and the length and width of the second repeating unit region are approximately 100 and 60, respectively, and when both the first similarity and the second similarity are greater than or equal to the second similarity threshold, select Figure 2 The first repeating unit region in is taken as the repeating unit region that meets the similarity condition.

[0064] See also Figure 6 , Figure 6 yes Figure 1 A flowchart of an embodiment of the steps before step S106 in the embodiment of the present invention. Specifically, it can include the following steps S701 to S703.

[0065] Step S701: Create a grid image using all region coordinates and similarities of the image to be detected.

[0066] After traversing and matching all regions including the image to be detected using the repeating unit region, the similarity between the repeating unit region and each region can be obtained. Since regions that are exactly the same as the repeating unit exist in different positions of the image to be detected, the similarity gradually decreases from the maximum value when the first repeating unit is found until the second repeating unit is encountered, and the maximum value is reached again, and this cycle repeats. The two-dimensional matrix composed of similarities is visualized as a grid image, and the points on the intersecting grid lines in the grid image are the coordinates of the upper left corner of each region.

[0067] Step S702: Obtain a similarity histogram of the grid images according to the similarity distribution of the grid images.

[0068] According to the similarity distribution in the grid image, the similarity histogram in the grid image can be obtained, such as Figure 7 As shown, the minimum similarity histogram can be bimodal, where Figure 7 The horizontal axis of the similarity histogram in is the similarity value, and the vertical axis is the number of similarity values.

[0069] Step S703: using the distribution of the similarity histogram, determine a first similarity threshold according to a preset threshold algorithm.

[0070] The preset threshold algorithm includes an OTSU (maximum between-class variance) algorithm, and the first similarity threshold can be determined by using the distribution of the obtained similarity histogram. Different similarity histograms have different values ​​for determining the first similarity threshold. For example, Figure 7 In the similarity histogram shown, the first similarity threshold may be 0.86.

[0071] See also Figure 8 , Figure 8 yes Figure 1 The flowchart of an embodiment of the steps after step S106 is shown in FIG. Specifically, the steps S901 to S903 may be included.

[0072] Step S901: Obtain a design drawing of a circuit board, and obtain a mapping matrix between the design drawing and the image to be detected.

[0073] The design drawing of the circuit board can be to import the circuit diagram of the circuit board into the software, and the design drawing of the circuit board can be automatically generated by the software based on the circuit diagram. After obtaining the design drawing of the circuit board, a circuit board identical to the design drawing of the circuit board can be generated according to the design drawing of the circuit board. The points of the image to be detected can be mapped to the design drawing using a mapping matrix.

[0074] Step S902: using the mapping matrix to obtain the circuit element region of the abnormal region in the design diagram.

[0075] The abnormal area is marked in the image to be detected using the above steps, and after obtaining the mapping matrix between the image to be detected and the design drawing, the circuit component area corresponding to the abnormal area in the image to be detected and the abnormal area in the design drawing can be obtained based on the mapping matrix and the abnormal area.

[0076] Step S903: using the circuit element area and the abnormal area, aligning the image to be inspected and the design drawing of the circuit board.

[0077] After determining the circuit element area and the abnormal area, the image to be inspected and the design drawing of the circuit board can be aligned. Therefore, this method can use the repeated unit area to find the abnormal area of ​​the image to be inspected with a high degree of repetition, simplifying the complexity of the algorithm; at the same time, the abnormal area can be accurately found from the image to be inspected with a high degree of repetition, making it easy to use the found abnormal area to align with the design drawing of the circuit board, and facilitating defect detection of the image to be inspected.

[0078] See also Fig. 9 , Fig. 9 yes Figure 1 Specifically, the steps before step S101 may include the following steps S1001 to S1004.

[0079] Step S1001: Acquire an image to be processed, and acquire image texture features of the image to be processed.

[0080] The image to be processed includes a circuit board image. The image to be processed can be obtained by an automatic optical inspection device, or by transmitting the image to be processed obtained by other means to the automatic optical inspection device for subsequent processing. For example, when the image to be processed is obtained by the automatic optical inspection device, the motion control module can be used to control the movement of the mechanical table and the motor, and the circuit board can be transferred to the image acquisition area, and the image of the circuit board can be acquired by the image acquisition module. In the process of image acquisition, the image acquisition area can be supplemented with light, such as by controlling the switch and brightness of the light source through the light source control module.

[0081] During the image acquisition process, in order to ensure that the circuit board image is not missed in the acquired image to be processed, the image acquisition module usually enlarges the acquisition area, that is, the area other than the circuit board area is acquired at the same time, and finally the circuit board area and the area other than the circuit board area together constitute the image to be processed. That is, the image to be processed includes both the circuit board area and the non-circuit board area. Among them, due to the many lines in the circuit board, the images of the circuit board area acquired are mostly dense patterns, and their image gradients are large; while the objects outside the circuit board are mostly blank areas, and the non-circuit board areas acquired usually have no dense patterns, and their image gradients are relatively small relative to the circuit board area image gradients. By acquiring the image texture features of the image to be processed, the image gradients of different areas in the image to be processed can be obtained.

[0082] Step S1002: Divide the image to be processed into several regions based on the gradient changes of the image texture features.

[0083] After obtaining the texture features of the image to be processed, the gradient change of the texture features on the image to be processed can be obtained. Based on the gradient change of the image texture features, the image to be processed is divided into several regions, that is, the region with larger image texture features and the region with smaller image texture features can be divided to obtain several regions.

[0084] Step S1003: determining the area that meets the image gradient condition among the plurality of areas as the circuit board area, and determining the remaining areas as the non-circuit board area.

[0085] After the image to be processed is divided into several regions, the image portion with larger image texture and the image portion with smaller image texture are respectively divided into different regions. Therefore, the region with larger image texture can be determined as the circuit board region; the region with smaller image texture can be determined as the non-circuit board region.

[0086] Step S1004: segment the image to be processed according to the circuit board area to obtain the image to be inspected of the circuit board.

[0087] After distinguishing the circuit board area and the non-circuit board area in the image to be processed, the image to be processed can be segmented to obtain a circuit board image that only includes the circuit board area. Thus, by obtaining the change of the image texture features of the image to be processed, the image to be processed is divided into the circuit board area and the non-circuit board area, and the area is segmented to obtain the circuit board image, so as to facilitate the alignment operation of the circuit board image with the original circuit board design image and facilitate defect detection of the circuit board image.

[0088] In one embodiment, the step of dividing the image to be processed into a number of regions based on the gradient change of the image texture feature (step S1002) includes: calculating the image gradient of the image to be processed based on the image texture feature; dividing the image to be processed into a number of regions according to the image gradient of the image to be processed and a preset gradient threshold.

[0089] After obtaining the texture features of the image to be processed, the gradients of the images of different parts in the image to be processed can be calculated. The image gradient of the circuit board area in the image to be processed is larger, and the image gradient of the non-circuit board area is smaller. There will be a large image gradient change from the circuit board area to the non-circuit board area, or from the non-circuit board area to the circuit board area. The image to be processed can be divided into several areas according to the image gradient and the preset gradient threshold.

[0090] In one embodiment, the step of determining an area among several areas that meets the image gradient condition as a circuit board area and determining the remaining areas as non-circuit board areas (step S1003) includes: determining an area with an image gradient greater than or equal to a preset gradient threshold as a circuit board area; and determining an area with an image gradient less than a preset gradient threshold as a non-circuit board area.

[0091] The image gradient of the circuit board area in the image to be processed is relatively large, and the image gradient of the non-circuit board area is relatively small. When the image gradient of a certain area is greater than the preset gradient threshold, it can be determined that the image gradient of the area is relatively large, that is, the area with the image gradient greater than the preset gradient threshold is determined as the circuit board area; when the image gradient of a certain area is less than the preset gradient threshold, it can be determined that the image gradient of the area is relatively small, that is, the area with the image gradient less than the preset gradient threshold is determined as the non-circuit board area.

[0092] In one embodiment, the preset gradient threshold includes a first preset gradient threshold and a second preset gradient threshold, wherein the first preset gradient threshold is greater than the second preset gradient threshold.

[0093] See also Fig.10 , Fig.10 yes Figure 1 Another embodiment of the process flow diagram of the steps before step S101 in the embodiment of the present invention can include the following steps S1101 to S1106.

[0094] Step S1101: Obtain an image to be processed, and obtain image texture features of the image to be processed.

[0095] Step S1102: Divide the image to be processed into several regions based on the gradient changes of the image texture features.

[0096] The implementation methods of step S1101 and step S1102 are respectively the same as those of step S1001 and step S1002, and will not be repeated here.

[0097] Step S1103: determining a region where the image gradient is greater than or equal to a first preset gradient threshold as a circuit board region.

[0098] Combination Fig.11 , Fig.11 is a schematic diagram of an embodiment of an image to be processed provided by the present application. Fig.11 As shown in the figure, the entire image is the image to be processed. According to the color, the entire image to be processed can be divided into four areas, namely, a white area, a gray area and two black areas. The black part is the circuit board area, the white part is the non-circuit board area, and the gray part is the board edge area of ​​the circuit board. The image gradient of the board area is the largest, the image gradient of the board edge area is the second largest, and the image gradient of the non-circuit board area is the smallest.

[0099] By acquiring the texture features of the image to be processed, the gradients of the images of different parts in the image to be processed can be calculated. The region where the image gradient is detected to be greater than the first preset gradient threshold is determined to be the circuit board region. Fig.11 In the image to be processed, the image gradient of the black area portion of the image is greater than the first preset gradient threshold, that is, the image of the black area portion is detected as the circuit board area.

[0100] Step S1104: determining a region where the image gradient is less than a first preset gradient threshold and greater than or equal to a second preset gradient threshold as a panel edge region.

[0101] The region where the detected image gradient is less than the first preset gradient threshold and greater than or equal to the second preset gradient threshold is determined as a non-board edge region. Fig.11 In the image to be processed, the image gradient of the gray area portion of the image is less than the first preset gradient threshold and greater than or equal to the second preset gradient threshold, that is, the image of the gray area portion is detected as the edge plate area.

[0102] Step S1105: Determine the area where the image gradient is less than the second preset gradient threshold as a non-circuit board area.

[0103] The region where the image gradient is detected to be less than the second preset gradient threshold is determined as a non-circuit board region. Fig.11 In the image to be processed, the image gradient of the white area portion of the image is less than the second preset gradient threshold, that is, the image of the white area portion is detected as a non-circuit board area.

[0104] Step S1106: segmenting the image to be processed according to the non-circuit board area, the circuit board area and the board edge area to obtain the circuit board image.

[0105] After distinguishing the circuit board area, the non-circuit board area, and the board edge area in the image to be processed, the image to be processed can be segmented to obtain a circuit board image that only includes the circuit board area.

[0106] See also Fig.12 , Fig.12 yes Fig. 9 The flowchart of an embodiment before step S1106 is shown. Specifically, it may include the following steps S1301 to S1304.

[0107] Step S1301: Acquire a first region image of a board edge region.

[0108] The first region image may be an image of the board edge region; or may be a partial image of the board edge region including the first edge pixel point set.

[0109] Step S1302: extracting a first edge pixel point set of the first region image.

[0110] The first edge pixel point set is a set of pixel points in the edge area of ​​the first area image. In this embodiment, the first edge pixel point set is a pixel point set that distinguishes the first area image from the circuit board area image.

[0111] Step S1303: performing an image processing flow of opening and / or closing operations on the first edge pixel point set to update the first edge pixel point set.

[0112] Both the opening operation and the closing operation use the erosion operation and the dilation operation. The erosion operation includes calculating the local minimum through the convolution operation, and then corroding the foreground image so that the edge of the image shrinks inward. The dilation operation is opposite to the erosion operation, and the local maximum is calculated through the convolution operation, and the foreground image is expanded so that the edge of the image expands outward. In this embodiment, the first edge pixel point is image processed by the opening operation and / or the closing operation, so as to update the first edge pixel point set. The opening operation and the closing operation can be used together. For example, the first edge pixel point can be image processed by the opening operation, the closing operation, and the opening operation again, so as to update the first edge pixel point set.

[0113] Step S1304: updating the board edge region according to the updated first edge pixel point set.

[0114] The board edge region is updated using the updated first edge pixel point set, that is, the first edge pixel point set between the board edge region and the circuit board region is updated, so as to facilitate segmentation processing of the board edge region and the circuit board region.

[0115] See also Fig.13 , Fig.13 yes Fig. 9The flowchart of an embodiment before step S1004 is shown. Specifically, it may include the following steps S1401 to S1404.

[0116] Step S1401: Acquire a second area image of the circuit board area.

[0117] The second region image may be an image of the circuit board region; or may be a partial image of the circuit board region including the second edge pixel point set.

[0118] Step S1402: extracting a second edge pixel point set of the second region image.

[0119] The second edge pixel set is a set of pixels in the edge region of the second region image. In this embodiment, the second edge pixel set is a set of pixels that distinguish the second region image from the non-circuit board region image.

[0120] Step S1403: performing an image processing flow of opening and / or closing operations on the second edge pixel point set to update the second edge pixel point set.

[0121] The second edge pixel points are image processed by an opening operation and / or a closing operation, thereby updating the second edge pixel point set. The opening operation and the closing operation can be used together. For example, the second edge pixel points can be image processed by an opening operation, a closing operation, and then an opening operation, thereby updating the second edge pixel point set.

[0122] Step S1404: updating the circuit board area according to the updated second edge pixel point set.

[0123] The circuit board area is updated using the updated second edge pixel point set, that is, the second edge pixel point set between the circuit board area and the non-circuit board area is updated, so as to facilitate segmentation processing of the circuit board area and the non-circuit board area.

[0124] See also Fig.14 , Fig.14 This is a schematic diagram of an embodiment of determining a circuit board area provided by the present application. Specifically, it may include the following steps S1501 to S1504.

[0125] Step S1501: Determine an area among a plurality of areas that meets the image gradient condition as a first circuit board area.

[0126] After the image to be processed is divided into several regions, a portion of the image with larger image texture is determined as the first circuit board region. That is, there may be multiple circuit board regions in the image to be processed, and the multiple circuit board regions are all determined as the first circuit board region.

[0127] Step S1502: when the number of first circuit board regions in the image to be processed is greater than or equal to two, obtaining distances between the first circuit board regions.

[0128] When there are multiple first circuit board areas in the image to be processed, there is a certain distance between two adjacent first circuit board areas. The distance can be the distance between the two nearest points of the two adjacent first circuit board areas, or the distance between the midpoints of the two side lines of the two adjacent first circuit board areas. For example, Fig.11 In the figure, the two black parts are two first circuit board areas respectively, and the distance between the midpoints of two adjacent side lines of the two black parts can be used as the distance between the two first circuit board areas.

[0129] Step S1503: merging two or more first circuit board regions whose distance is less than or equal to a preset distance threshold into a second circuit board region.

[0130] When it is detected that the distance between two or more first circuit board areas is less than or equal to a preset distance threshold, the first circuit board area can be identified as one circuit board area, and the two or more first circuit boards with a distance less than the preset distance threshold are merged into a second circuit board area. In other words, the multiple first circuit board areas are regarded as a whole as the second circuit board area.

[0131] Step S1504: Determine the merged second circuit board area and the remaining first circuit board area as the circuit board area.

[0132] After at least part of the first circuit board area is merged, the image to be processed may contain the first circuit board area, the second circuit board area, or both the first circuit board area and the second circuit board area. The first circuit board area and / or the second circuit board area of ​​the image to be processed are simultaneously determined as the circuit board area, so as to facilitate segmentation of the circuit board area and the non-circuit board area.

[0133] See also Fig.15 , Fig.15 This is another embodiment of the process flow diagram of determining the circuit board area provided by the present application. Specifically, it may include the following steps S1601 to S1604.

[0134] Step S1601: Determine an area among a plurality of areas that meets the image gradient condition as a third circuit board area.

[0135] After the image to be processed is divided into several regions, the image with larger image texture is determined as the third circuit board region. The image texture of the third circuit board region meets the image gradient condition.

[0136] Step S1602: Detect whether the area size of the third circuit board area is greater than or equal to a preset size threshold.

[0137] The third circuit board area includes the circuit board area. In some possible cases, there may be an area in the non-circuit board area that meets the image gradient condition, but if it is determined that the area is a circuit board area, the final segmented image will still include the non-circuit board area image. In order to avoid the above situation, it can be determined whether the area size of each third circuit board area is greater than or equal to a preset size threshold, so as to determine whether it is an image of the non-circuit board area by the size of the area of ​​the third circuit board.

[0138] Step S1603: If not, delete the third circuit board area whose size is smaller than the preset size threshold.

[0139] In some embodiments, the third circuit board area may include an interference area in the circuit board area and the non-circuit board area, the interference area meets the image gradient condition, but it is not the circuit board area, and the size of the interference area is smaller than the size of the circuit board area. When it is detected that the area size of the third circuit board is smaller than the preset size threshold, the area smaller than the preset size threshold is deleted.

[0140] Step S1604: Determine the remaining third circuit board area after the size detection as the circuit board area.

[0141] After deleting the third circuit board area whose size is smaller than the preset size threshold, the remaining third circuit board area in the image to be processed is the circuit board area, so as to facilitate segmentation processing of the circuit board area and the non-circuit board area.

[0142] The above method is applied in automatic optical inspection equipment. Fig.16 , Fig.16 1 is a schematic diagram of the structure of an embodiment of the automatic optical inspection device of the present application. The automatic optical inspection device 10 of the present embodiment includes a processor 11 and a memory 12. The memory 12 stores a computer program, and the processor 11 is used to execute the computer program to implement the above abnormal area detection method.

[0143] The processor 11 may be an integrated circuit chip having the ability to process signals. The processor 11 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0144] for Figure 1-Figure 15The abnormal area detection method of the embodiment shown can be presented in the form of a computer program. The present application proposes a computer storage medium carrying the computer program. Fig.17 , Fig.17 It is a schematic diagram of the structure of an embodiment of a computer storage medium of the present application. In this embodiment, the computer storage medium 20 includes a computer program 21, which can be executed to implement the above-mentioned abnormal area detection method.

[0145] The computer storage medium 20 of this embodiment can be a medium that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, or can also be a server that stores the program instructions. The server can send the stored program instructions to other devices for execution, or can also execute the stored program instructions by itself.

[0146] In addition, if the above functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a storage medium readable by a mobile terminal, that is, the present application also provides a storage device storing program data, the program data can be executed to implement the method of the above embodiment, and the storage device can be, for example, a USB flash drive, an optical disk, a server, etc. In other words, the present application can be embodied in the form of a software product, which includes a number of instructions for enabling a smart terminal to execute all or part of the steps of the method described in each embodiment.

[0147] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other.

[0148] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0149] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0150] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (which can be a personal computer, server, network device or other system that can fetch instructions from the instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0151] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal areas, characterized in that: The abnormal area detection method comprises: Obtaining a first starting point in the image to be detected; Generate a first repeating unit region based on the first starting point and according to a first preset size; Using the first repeating unit region to obtain a first adjacent region with the same size, and obtaining a first similarity between the first repeating unit region and the first adjacent region, wherein the first repeating unit region and the first adjacent region are two adjacent regions in the image to be detected; When the first similarity is greater than or equal to a second similarity threshold, taking the first repeating unit region as a repeating unit region that meets the similarity condition; Using the repeating unit region to perform traversal matching on all regions of the image to be detected including the circuit board, to obtain the similarity between the repeating unit region and each region; The region with a similarity less than the first similarity threshold is marked as an abnormal region.

2. The abnormal area detection method according to claim 1, characterized in that: After obtaining the first similarity between the first repeating unit region and the first adjacent region, the abnormal region detection method further includes: When the first similarity is less than or equal to the second similarity threshold, generating a second repeating unit region based on the first starting point and according to a second preset size; Repeated execution: using the second repeating unit region to obtain a second adjacent region of the same size, obtaining a second similarity between the second repeating unit region and the second adjacent region, until a second repeating unit region whose second similarity is greater than or equal to the second similarity threshold is obtained as a repeating unit region that meets the similarity condition.

3. The abnormal area detection method according to claim 1, characterized in that: After obtaining the first similarity between the first repeating unit region and the first adjacent region, the abnormal region detection method further includes: Generating a second repeating unit region based on the first starting point and according to a second preset size; Using the second repeating unit region to obtain a second adjacent region having the same size as the second repeating unit region, and obtaining a second similarity between the second repeating unit region and the second adjacent region; The repeating unit region corresponding to the greatest similarity between the first similarity and the second similarity is taken as the repeating unit region meeting the similarity condition.

4. The abnormal area detection method according to claim 3, characterized in that: After obtaining the second similarity between the second repeating unit region and the second adjacent region, the abnormal region detection method further includes: Determine whether the first similarity and the second similarity are both greater than or equal to the second similarity threshold; If not, taking the repeating unit region corresponding to the similarity greater than or equal to the second similarity threshold as the repeating unit region meeting the similarity condition; If so, the repeating unit region with the largest preset size is taken as the repeating unit region that meets the similarity condition.

5. The abnormal area detection method according to claim 1, characterized in that: After obtaining the first similarity between the first repeating unit region and the first adjacent region, the abnormal region detection method further includes: When the first similarity is less than a third similarity threshold, selecting a second starting point to replace the first starting point in the image to be detected; The third similarity threshold is smaller than the second similarity threshold.

6. The abnormal area detection method according to claim 1, characterized in that: Before marking the region with a similarity less than the first similarity threshold as an abnormal region, the abnormal region detection method further includes: Using all the region coordinates and similarities of the image to be detected, a grid image is established; According to the similarity distribution of the grid images, obtaining a similarity histogram of the grid images; The first similarity threshold is determined according to a preset threshold algorithm using the distribution of the similarity histogram.

7. The abnormal area detection method according to claim 1, characterized in that: The abnormal area detection method comprises: Acquire an image to be processed, and acquire image texture features of the image to be processed; Based on the gradient change of the image texture feature, the image to be processed is divided into a number of regions; Determine the area in the plurality of areas that meets the image gradient condition as the circuit board area, and determine the remaining areas as non-circuit board areas; The image to be processed is segmented according to the circuit board area to obtain the image to be detected of the circuit board.

8. The abnormal area detection method according to claim 1, characterized in that: After marking the region with a similarity less than the first similarity threshold as an abnormal region, the abnormal region detection method further includes: Acquire a design drawing of the circuit board, and acquire a mapping matrix between the design drawing and the image to be detected; Using the mapping matrix to obtain the circuit element region of the abnormal region in the design diagram; The image to be detected and the design drawing of the circuit board are aligned by using the circuit element area and the abnormal area.

9. An automatic optical inspection device, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images