Image processing method and device, equipment and medium

Abstract

The invention relates to the technical field of computers, in particular to an image processing method and device, equipment and a medium. The method comprises the following steps: acquiring a to-be-processed image and a standard image of a target circuit board by using AOI equipment; key point matching is carried out on the to-be-processed image and the standard image, and a reference target is determined; determining a corresponding relationship between a plurality of first targets in the to-be-processed image and a plurality of second targets in the standard image based on the position information of the reference target in the to-be-processed image and the position information of the second targets matched with the reference target in the standard image; based on the corresponding relationship and a preset screening rule, screening the plurality of first targets to obtain a plurality of key targets meeting the preset screening rule in the plurality of first targets; based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the second target corresponding to the plurality of key targets in the standard image, the to-be-processed image is corrected, and the accuracy of target circuit board detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to an image processing method and device, equipment and a medium. BACKGROUND

[0002] An AOI (Automatic Optic Inspection) device is a device for detecting common defects encountered in welding production based on optical principles. It is widely used in the field of electronic components, such as electronic components, PCBs (Printed Circuit Boards), display panels, semiconductor chips, etc., and its detection accuracy and requirements vary depending on the application field.

[0003] In related technologies, an AOI device acquires a PCB image to be detected in real time, and then detects the standard image and the PCB image to be detected in a sliding window manner. If the standard image and the PCB image to be detected are exactly the same, it is determined that the PCB to be detected is normal. However, due to the slight activity space of the PCB in the tray, the picture content in the standard image and the measured image pair generated after sliding is not exactly the same, and there may be a certain degree of offset between the pixels of the standard image and the measured image,

[0004] Therefore, in order to reduce the impact of the offset in the measured image of the PCB on the subsequent component detection, it is necessary to provide an image processing method to correct the offset of the measured image. SUMMARY

[0005] The embodiments of the present application provide an image processing method, device, equipment and medium to reduce the impact of the offset in the measured image of the PCB on the subsequent component detection.

[0006] In a first aspect, the present application provides an image processing method, comprising:

[0007] acquiring a target circuit board image to be processed and a standard image corresponding to the target circuit board by using an automatic optical inspection (AOI) device;

[0008] performing key point matching on the image to be processed and the standard image to determine a reference target; the reference target is a first target in the image to be processed, and the first target has the most matching key points with a second target in the standard image;

[0009] determining a correspondence relationship between a plurality of first targets in the image to be processed and a plurality of second targets in the standard image based on position information of the reference target in the image to be processed and position information of the second target matching the reference target in the standard image;

[0010] screening, a plurality of key targets in the plurality of first targets meeting the preset screening rule are obtained;

[0011] Based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the second targets corresponding to the plurality of key targets in the standard image, the to-be-processed image is corrected.

[0012] In a possible implementation, the key point matching of the to-be-processed image and the standard image and the determination of the reference target comprises:

[0013] The key point matching of the to-be-processed image and the standard image is performed using an image matching algorithm, and a matched key point pair in the to-be-processed image and the standard image is determined.

[0014] Based on the matched key point pair in the to-be-processed image and the standard image, the number of key point pairs corresponding to a plurality of first targets in the to-be-processed image is determined respectively.

[0015] The number of key point pairs corresponding to a plurality of first targets in the to-be-processed image is sorted, and a first target with the largest number of key point pairs in the to-be-processed image is determined.

[0016] The first target with the largest number of key point pairs in the to-be-processed image is taken as the reference target.

[0017] In a possible implementation, before the key point matching of the to-be-processed image and the standard image and the determination of the reference target, the method further comprises:

[0018] The to-be-processed image is detected using a rotating target detection algorithm, and a plurality of first targets in the to-be-processed image and position information and a rotation angle of each first target in the to-be-processed image are obtained.

[0019] The standard image is detected using the rotating target detection algorithm, and a plurality of second targets in the standard image and position information and a rotation angle of each second target in the standard image are obtained.

[0020] In a possible implementation, the determination of the correspondence between the plurality of first targets in the to-be-processed image and the plurality of second targets in the standard image based on the position information of the reference target in the to-be-processed image and the position information of the second target matched with the reference target in the standard image comprises:

[0021] For each first target of the plurality of first targets in the to-be-processed image, the following operations are performed:

[0022] Based on the position information of the reference target in the to-be-processed image and the position information of the first target in the to-be-processed image, a relative position of the first target and the reference target in the to-be-processed image is determined;

[0023] Based on the relative position of the first target and the reference target in the to-be-processed image, the position information of the second target matched with the reference target in the standard image, and a preset deviation threshold, a position information region of the second target corresponding to the first target in the standard image is determined;

[0024] The second target in the position information region in the standard image is taken as the second target corresponding to the first target.

[0025] In a possible implementation, the preset screening rule is:

[0026] The number of the first targets in the to-be-processed image that have a corresponding relationship with the plurality of second targets in the standard image is greater than a preset threshold;

[0027] The position information of the key target in the to-be-processed image is in a preset region;

[0028] The key target has a corresponding second target.

[0029] In a possible implementation, the correcting the to-be-processed image based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image, and the position information and the rotation angle of the second targets corresponding to the plurality of key targets in the standard image includes:

[0030] Based on the position information of the plurality of key targets in the to-be-processed image and the position information of the second targets corresponding to the plurality of key targets in the standard image, an offset position of the to-be-processed image is determined;

[0031] Based on the rotation angle of the plurality of key targets in the to-be-processed image and the rotation angle of the second targets corresponding to the plurality of key targets in the standard image, an offset rotation angle of the to-be-processed image is determined;

[0032] Based on the offset position and the offset rotation angle, the to-be-processed image is corrected.

[0033] In a possible implementation, the position information includes horizontal coordinate information and vertical coordinate information; and the determining the offset position of the to-be-processed image based on the position information of the plurality of key targets in the to-be-processed image and the position information of the second targets corresponding to the plurality of key targets in the standard image includes:

[0034] For each key target, the offset horizontal coordinate of the key target is determined based on the horizontal coordinate information of the key target in the to-be-processed image and the horizontal coordinate information of the second target corresponding to the key target in the standard image; and the offset vertical coordinate of the key target is determined based on the vertical coordinate information of the key target in the to-be-processed image and the vertical coordinate information of the second target corresponding to the key target in the standard image.

[0035] The mean value of the offset horizontal coordinates of the plurality of key targets is taken as the offset horizontal coordinate of the to-be-processed image; and the mean value of the offset vertical coordinates of the plurality of key targets is taken as the offset vertical coordinate of the to-be-processed image.

[0036] In a possible implementation, the determining the offset rotation angle of the to-be-processed image based on the rotation angles of the plurality of key targets in the to-be-processed image and the rotation angles of the second targets corresponding to the plurality of key targets in the standard image includes:

[0037] For each key target, the following operations are respectively performed:

[0038] The offset rotation angle of the key target is determined based on the rotation angle of the key target in the to-be-processed image and the rotation angle of the second target corresponding to the key target in the standard image.

[0039] In a second aspect, the present application provides an image processing device, which includes:

[0040] The acquisition module is configured to acquire, by using an automatic optical inspection (AOI) device, a to-be-processed image of a target circuit board and a standard image corresponding to the target circuit board.

[0041] The matching module is configured to perform key point matching on the to-be-processed image and the standard image, and determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most key points matched with a second target in the standard image.

[0042] The corresponding module is configured to determine a corresponding relationship between a plurality of first targets in the to-be-processed image and a plurality of second targets in the standard image based on position information of the reference target in the to-be-processed image and position information of the second target matched with the reference target in the standard image.

[0043] The screening module is configured to screen the first targets in the to-be-processed image based on the correspondence and a preset screening rule, to obtain a plurality of key targets in the first targets that meet the preset screening rule.

[0044] The correction module is configured to correct the to-be-processed image based on the position information and the rotation angle of the key targets in the to-be-processed image and the position information and the rotation angle of the second targets corresponding to the key targets in the standard image.

[0045] In a third aspect, the present application also provides an electronic device, which comprises a processor configured to implement the steps of the image processing method according to any one of the above aspects when executing a computer program stored in a memory.

[0046] In a fourth aspect, the present application also provides a computer-readable storage medium storing a computer program, which is configured to implement the steps of the image processing method according to any one of the above aspects when executed by a processor.

[0047] In a fifth aspect, the present application provides a computer program product comprising a computer program, which is configured to implement the image processing method according to the first aspect when executed by a processor.

[0048] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0049] In the embodiments of the present application, the AOI device is used to collect the to-be-processed image of the target circuit board and the standard image corresponding to the target circuit board; key point matching is performed on the to-be-processed image and the standard image to determine a reference target; the correspondence between the first targets in the to-be-processed image and the second targets in the standard image is determined based on the position information of the reference target in the to-be-processed image and the standard image; the first targets in the to-be-processed image are screened based on the correspondence and a preset screening rule, to obtain a plurality of key targets in the first targets that meet the preset screening rule; and the position information and the rotation angle of the key targets in the to-be-processed image and the position information and the rotation angle of the second targets corresponding to the key targets in the standard image are used to correct the to-be-processed image.

[0050] Therefore, on the one hand, the target with the most key points matched with the standard image is selected as the target in the to-be-processed image, and the targets in the to-be-processed image and the standard image are corresponded; on the other hand, the position information and the rotation angle of the key targets obtained through multiple screenings based on the correspondence are used to determine the offset position and the offset rotation angle, and the to-be-processed image is corrected, which increases the accuracy of the target circuit board detection and reduces the offset of the to-be-processed image of the PCB caused by the sliding window method. The influence of the offset of the to-be-processed image on the subsequent component defect detection is reduced.

[0051] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0053] Figure 1A A schematic diagram of a standard image provided for the embodiments of the present application;

[0054] Figure 1B A schematic diagram of a measured image provided for the embodiments of the present application;

[0055] Figure 2 A flowchart of an image processing method provided for the embodiments of the present application;

[0056] Figure 3 A flowchart of determining a reference target provided for the embodiments of the present application;

[0057] Figure 4 A schematic diagram of a key point matching result provided for the embodiments of the present application;

[0058] Figure 5 A flowchart of determining a corresponding relationship provided for the embodiments of the present application;

[0059] Figure 6 A schematic diagram of a preset region provided for the embodiments of the present application;

[0060] Figure 7 A flowchart of correcting a to-be-processed image provided for the embodiments of the present application;

[0061] Figure 8 A structural schematic diagram of an image processing device provided for the embodiments of the present application;

[0062] Figure 9 A structural schematic diagram of an electronic device provided for the embodiments of the present application. DETAILED DESCRIPTION

[0063] In order to make the purposes, technical solutions and advantages of the present application clearer, the following further describes the present application with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0064] It should be noted that the terms "comprising" and "having" and their conjugates involved in the present application are intended to cover the inclusion, not the exclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0065] The terms "first", "second" in the specification are only for descriptive purposes, and cannot be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0066] The word "exemplary" used in the following means "as an example, embodiment or illustration". Any embodiment described as "exemplary" is not necessarily interpreted as superior or better than other embodiments.

[0067] The AOI (Automatic Optic Inspection) device is a device for detecting common defects encountered in welding production based on optical principles. It is widely used in the field of electronic components, such as electronic components, PCB (Printed Circuit Board), display panel, semiconductor chip, etc., and its detection accuracy and demand vary with the application field.

[0068] In electronic component detection, the AOI device can detect size measurement of connectors, capacitors, resistors, etc., as well as defects such as PIN pin offset, deformation, shortage, etc., and can also detect printed characters, etc. In addition, the application of AOI device is not limited to the field of electronic components, but can also be applied to the detection of food packaging, pharmaceutical packaging, battery products and textile and garment accessories fields.

[0069] In display panel inspection, AOI (Automated Optical Inspection) equipment can be applied to multiple stages of the display panel manufacturing process. For example, during panel production, AOI equipment can perform real-time inspection of the panel, promptly identifying and addressing defects to prevent the production of defective products. Furthermore, AOI equipment plays a crucial role in the panel assembly and testing stages, ensuring assembly quality and the accuracy of test results. Specifically, AOI equipment can detect various defects in display panels, such as bright spots, dark spots, poor lines, and uneven color. These defects can affect the display panel's display performance and lifespan; therefore, timely detection and handling of these defects are essential for improving product quality and user satisfaction.

[0070] In semiconductor chip packaging processes, AOI (Automated Optical Inspection) equipment mainly performs wafer inspection, particle appearance defect inspection, die / wire bonding inspection, and molding appearance inspection. Especially in molding appearance inspection, the equipment detects and performs 3D measurements on potential defects generated during the chip manufacturing process, such as molding defects, plating defects, BGA balls, and leads, and sorts the inspected chips as OK / NG.

[0071] In the PCB industry, AOI equipment can detect defects such as bridging, displacement, lack of solder, and insufficient solder before surface mount after soldering. It can also detect problems such as displacement, missing material, polarity, skew, bent leads, and misaligned components before reflow soldering after surface mount. Furthermore, it can detect problems such as insufficient / excessive solder, short circuits due to lack of solder, solder balls, missing material, polarity, displacement, bent leads, and misaligned components after reflow soldering or wave soldering. In addition, it can also perform bare board inspection in the PCB industry.

[0072] In related technologies, Zhidong Precision's SMT-AOI equipment acquires images of the PCB board to be inspected in real time, and then uses a sliding window method to inspect the standard image and the measured image. If the standard image and the measured image are exactly the same, the PCB board to be inspected is determined to be normal. However, due to the slight movement space of the PCB board in the tray, the content of the standard image generated after sliding windowing is not completely consistent with the measured image. Figure 1A The standard image shown and Figure 1B The measured images shown have a pixel offset of several to a dozen pixels.

[0073] Therefore, in order to reduce the impact of offsets in measured PCB images on subsequent components, it is urgent to provide an image processing method to correct the offsets in the measured images.

[0074] In view of this, this application provides an image processing method, apparatus, device and medium to solve the problem that there is a certain degree of offset between the measured image of the PCB board and the standard image in the prior art, which affects the subsequent components.

[0075] The inventive concept of the embodiments of the present application can be summarized as follows: on the one hand, the target with the most matching key points in the standard image is selected from the to-be-processed image, and each target in the to-be-processed image and the standard image is corresponded; on the other hand, the position information and the rotation angle of the key target obtained through multiple screenings based on the corresponding relationship are used to determine the offset position and the offset rotation angle, and the to-be-processed image is corrected, thereby increasing the accuracy of the target circuit board detection and reducing the offset of the to-be-processed image of the PCB caused by the sliding window method. The influence of the to-be-processed image offset on the subsequent component defect detection is reduced.

[0076] The image processing method provided by the present application can be applied to the AOI device in the application scenarios such as electronic component detection, display panel detection, semiconductor chip packaging, PCB detection, etc., and can correct the images of electronic components, display panels, semiconductor chips, PCBs, etc.

[0077] After introducing the main inventive idea of the embodiments of the present application, the main inventive idea of the embodiments of the present application will be described in detail in combination with Figure 2 The image processing method provided by the present application will be introduced.

[0078] Referring to Figure 2 The flowchart of the image processing method of the embodiments of the present application is shown in the figure, and the method can be specifically executed as the steps shown in Figure 2

[0079] In step S201, the to-be-processed image of the target circuit board and the corresponding standard image of the target circuit board are collected by using the automatic optical detection (AOI) device.

[0080] The to-be-processed image and the corresponding standard image of the target circuit board both refer to the local image on the PCB.

[0081] In a possible implementation, before the key point matching of the to-be-processed image and the standard image and the determination of the reference target, the method further includes:

[0082] The to-be-processed image is detected by using a rotating target detection algorithm to obtain a plurality of first targets in the to-be-processed image and the position information and the rotation angle of each first target in the to-be-processed image.

[0083] The standard image is detected by using a rotating target detection algorithm to obtain a plurality of second targets in the standard image and the position information and the rotation angle of each second target in the standard image.

[0084] In specific implementation, the standard image and the to-be-processed image are input into the same rotating target detection algorithm YOLOv8-OBB, and the rotating target detection results of the standard image and the to-be-processed image are output.

[0085] ​The detection result of the to-be-processed image includes a plurality of first targets in the to-be-processed image, such as the categories of 30 kinds of components on the PCB, and a rotated rectangular frame (class_index, x1, y1, x2, y2, x3, y3, x4, y4) of each first target, and the position information and the rotation angle of each first target in the to-be-processed image are obtained based on the rotated rectangular frame.

[0086] The detection result of the standard image includes a plurality of second targets in the standard image, such as the categories of 30 kinds of components on the PCB, and a rotated rectangular frame of each second target, and the position information and the rotation angle of each second target in the standard image are obtained based on the rotated rectangular frame.

[0087] It should be noted that the position information of each target in the image in the embodiments of the present application refers to the position information of the center point of each target. The rotated rectangular frame of each first target in the to-be-processed image can be detected by the rotated target detection algorithm, so that the position information of each first target can be calculated according to the position coordinates of the rotated rectangular frame; and the rotated rectangular frame of each second target in the standard image can be detected, so that the position information of each second target can be calculated according to the position coordinates of the rotated rectangular frame.

[0088] In step S202, key point matching is performed on the to-be-processed image and the standard image to determine the reference target.

[0089] The reference target is one of the first targets in the to-be-processed image, and the first target has the most matching key points with the second targets in the standard image.

[0090] In one possible implementation, the key point matching is performed on the to-be-processed image and the standard image to determine the reference target, including the steps as shown in Figure 3

[0091] In step S301, the image matching algorithm is used to perform key point matching on the to-be-processed image and the standard image to determine the matching key point pairs in the to-be-processed image and the standard image;

[0092] In step S302, based on the matching key point pairs in the to-be-processed image and the standard image, the number of key point pairs corresponding to each of the plurality of first targets in the to-be-processed image is determined;

[0093] In step S303, the number of key point pairs corresponding to each of the plurality of first targets in the to-be-processed image is sorted to determine the first target with the most number of key point pairs in the to-be-processed image;

[0094] In step S304, the first target with the most number of key point pairs in the to-be-processed image is taken as the reference target.

[0095] ​In implementation, the image to be processed and the standard image are input into the image matching algorithm (SIFT) to match the corresponding positions, and the matching result is as shown in FIG. 8. Figure 4 Then, the number of key point pairs of the image to be processed and the standard image in the region where each first target in the image to be processed is counted, and the number of key point pairs of all first targets is sorted, and the target ranked first is selected as the reference target. For example, A1 shown in FIG. 8 is selected as the reference target, and the position information of the second target matched with the reference target A1 in the standard image is A2. Figure 4

[0096] In step S203, the corresponding relationship between the multiple first targets in the image to be processed and the multiple second targets in the standard image is determined based on the position information of the reference target in the image to be processed and the position information of the second target matched with the reference target in the standard image.

[0097] In a possible implementation, when the corresponding relationship between the multiple first targets in the image to be processed and the multiple second targets in the standard image is determined based on the position information of the reference target in the image to be processed and the position information of the second target matched with the reference target in the standard image, the steps shown in FIG. 9 are performed for each first target of the multiple first targets in the image to be processed. Figure 5

[0098] In step S501, the relative position of the first target and the reference target in the image to be processed is determined based on the position information of the reference target in the image to be processed and the position information of the first target in the image to be processed.

[0099] In step S502, the position information region of the second target corresponding to the first target in the standard image is determined based on the relative position of the first target and the reference target in the image to be processed, the position information of the second target matched with the reference target in the standard image and a preset deviation threshold.

[0100] In step S503, the second target in the position information region in the standard image is taken as the second target corresponding to the first target.

[0101] For example, it is assumed that the position information of the reference target A1 in the image to be processed is (x1, y1), the corresponding position information A2 of the second target matched with the reference target in the standard image is (x2, y2), and the position information of the first target B1 in the image to be processed is (x3, y3).

[0102] ​​Firstly, the relative position of the first target B1 and the reference target A1 in the image to be processed is determined as (Δx, Δy) = (x3-x1, y3-y1), and then based on the relative position of the first target B1 and the reference target A1 in the image to be processed, the position information of the reference target A1 in the standard image, and the preset deviation threshold, the x coordinate region of the second target corresponding to the first target B1 in the standard image is determined as (x2+Δx-ε, x2+Δx+ε), and the y coordinate region is (y2+Δy-ε, y2+Δy+ε), and ε is the preset deviation threshold, and the position information region of the second target corresponding to the first target in the standard image is (x2+Δx-ε, x2+Δx+ε) and (y2+Δy-ε, y2+Δy+ε).

[0103] Finally, the position information of the second target B2 is obtained by searching in the position information region, x4∈(x2+Δx-ε, x2+Δx+ε) and y4∈(y2+Δy-ε, y2+Δy+ε), so that the second target B2 is determined as the target corresponding to the first target B1, and the position information is (x4, y4).

[0104] Optionally, if there are multiple second targets in the position information region in the standard image, the distances between the multiple second targets in the position information region and the center coordinates of the position information region are determined based on the position information of the multiple second targets in the position information region and the center coordinates of the position information region, and the second target with the shortest distance between the position information and the center coordinates of the position information region is taken as the second target corresponding to the first target.

[0105] In specific implementation, for each second target in the position information region, the distance between the position information of the second target and the center coordinates of the position information region is calculated, and if the distance between the position information of the second target and the center coordinates of the position information region is the shortest among the multiple second targets in the position information region, the second target is taken as the second target corresponding to the first target.

[0106] For example, if the second target B2(x4, y4), the second target B3(x5, y5), and the second target B4(x6, y6) are obtained by searching in the position information region, and the center coordinates of the position information region are (x2+Δx, y2+Δy), then according to the Euclidean distance, the distance between the second target B2(x4, y4) and (x2+Δx, y2+Δy) is the shortest, and the second target B2 is determined as the target corresponding to the first target B1.

[0107] The preset deviation threshold can be set according to actual needs, or can be set according to experience, and the application does not limit this.

[0108] In step S204, based on the correspondence relationship and the preset screening rule, the plurality of first targets in the to-be-processed image are screened to obtain a plurality of key targets in the plurality of first targets that meet the preset screening rule.

[0109] In a possible implementation, the preset screening rule is:

[0110] The number of the first targets in the to-be-processed image that have the correspondence relationship with the plurality of second targets in the standard image is greater than a preset threshold value;

[0111] The position information of the key target in the to-be-processed image is within a preset region;

[0112] The key target has a corresponding second target.

[0113] In specific implementation, based on the correspondence relationship and the preset screening rule, the plurality of first targets in the to-be-processed image are screened to obtain a plurality of key targets in the plurality of first targets that meet the preset screening rule, which can be implemented as:

[0114] First, it is determined that the number of the first targets in the to-be-processed image that have the correspondence relationship with the plurality of second targets in the standard image is greater than a preset threshold value. In other words, it is determined that the ratio of the number of the first targets that have the correspondence relationship to the number of the first targets in the to-be-processed image is greater than a ratio threshold value.

[0115] The preset threshold value and the ratio threshold value can be set according to actual needs, or can be set according to experience, and the present application does not limit this.

[0116] Further, the key target is screened again, and the target whose position information is within the preset region on the to-be-processed image is retained. The preset region is a region in which the gray line as shown in Figure 6 is located. It can be understood that the target whose position information is x pixels away from the upper, lower, left and right edges of the to-be-processed image is deleted, and the target within the preset region is obtained. X is any positive number greater than 0.

[0117] The preset region can be set according to actual needs, or can be set according to experience, and the present application does not limit this.

[0118] Further, the first target having the corresponding second target is retained in the target within the preset region as shown in Figure 6 , as the key target. It can be understood that the first target within the region as shown in Figure 6 , which does not find the corresponding second target in the standard image, is removed.

[0119] If it is determined that the number of the first targets having the corresponding relationship with the plurality of second targets in the standard image in the to-be-processed image is less than the preset threshold value, or the ratio of the number of the first targets having the corresponding relationship to the number of the first targets in the to-be-processed image is less than the ratio threshold value, the front end of the industrial computer prompts the worker that the tooling may have a large deviation and needs to be manually adjusted by the worker.

[0120] If the number of the key targets is 0, the to-be-processed image does not need to be corrected; if the number of the key targets is greater than or equal to 1, the step S205 is continued to be executed.

[0121] In the step S205, the to-be-processed image is corrected based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the plurality of second targets corresponding to the plurality of key targets in the standard image.

[0122] In a possible implementation, the to-be-processed image is corrected based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the plurality of second targets corresponding to the plurality of key targets in the standard image, and the step is performed as shown in FIG. 7. Figure 7

[0123] In the step S701, the offset position of the to-be-processed image is determined based on the position information of the plurality of key targets in the to-be-processed image and the position information of the plurality of second targets corresponding to the plurality of key targets in the standard image.

[0124] In a possible implementation, the position information includes horizontal coordinate information and vertical coordinate information; the offset position of the to-be-processed image is determined based on the position information of the plurality of key targets in the to-be-processed image and the position information of the plurality of second targets corresponding to the plurality of key targets in the standard image, and the step includes:

[0125] For each key target, the offset horizontal coordinate of the key target is determined based on the horizontal coordinate information of the key target in the to-be-processed image and the horizontal coordinate information of the second target corresponding to the key target in the standard image; and the offset vertical coordinate of the key target is determined based on the vertical coordinate information of the key target in the to-be-processed image and the vertical coordinate information of the second target corresponding to the key target in the standard image.

[0126] The mean value of the offset horizontal coordinates of the plurality of key targets is taken as the offset horizontal coordinate of the to-be-processed image; and the mean value of the offset vertical coordinates of the plurality of key targets is taken as the offset vertical coordinate of the to-be-processed image.

[0127] ​In implementation, for the plurality of key targets of the to-be-processed image, first, the horizontal coordinate deviations between the plurality of key targets of the to-be-processed image and the corresponding second targets in the standard image are calculated according to formula (1), then the sum of all the deviations is divided by the number of the key targets, and the offset horizontal coordinate of the to-be-processed image is obtained:

[0128]

[0129] wherein, n is the number of the key targets, Δx i is the horizontal coordinate deviation of the i-th key target between the to-be-processed image and the corresponding second target in the standard image.

[0130] According to formula (2), first, the vertical coordinate deviations between the plurality of key targets of the to-be-processed image and the corresponding second targets in the standard image are calculated, then the sum of all the deviations is divided by the number of the key targets, and the offset vertical coordinate of the to-be-processed image is obtained:

[0131]

[0132] wherein, n is the number of the key targets, Δy i is the vertical coordinate deviation of the i-th key target between the to-be-processed image and the corresponding second target in the standard image.

[0133] In step S702, based on the rotation angles of the plurality of key targets in the to-be-processed image and the rotation angles of the corresponding second targets in the standard image, the offset rotation angle of the to-be-processed image is determined.

[0134] In one possible implementation, based on the rotation angles of the plurality of key targets in the to-be-processed image and the rotation angles of the corresponding second targets in the standard image, the offset rotation angle of the to-be-processed image is determined, including:

[0135] For each key target, the following operations are performed respectively:

[0136] Based on the rotation angle of the key target in the to-be-processed image and the rotation angle of the corresponding second target in the standard image, the offset rotation angle of the key target is determined.

[0137] In implementation, the offset rotation angle Δθ i of each key target is calculated as follows: i1 Δθ i2 , Δθ i is the offset rotation angle of the i-th key target, Δθ i1 is the rotation angle of the i-th key target in the to-be-processed image, and Δθ i2 is the rotation angle of the corresponding second target of the i-th key target in the standard image.

[0138] In step S703, the to-be-processed image is corrected based on the offset position and the offset rotation angle.

[0139] In implementation, the position information (x, y) of each first target in the to-be-processed image is subtracted by (Aw, Ah) as the corrected coordinate of the first target, and each first target in the to-be-processed image is corrected. The corrected coordinate of each first target is used for subsequent detection of offset and missing parts and other defect types.

[0140] Meanwhile, the offset rotation angles of all key targets are counted. If the number of key targets whose offset rotation angles are greater than the rotation angle threshold is greater than the number threshold, the worker is prompted at the front end of the industrial computer that the PCB board may have an offset, and the worker manually adjusts the angle and other information of the batch of PCB boards. If the number of key targets whose offset rotation angles are greater than the rotation angle threshold is less than the number threshold, no processing is needed.

[0141] The rotation angle threshold and the number threshold can be set according to actual needs or experience, and the present application does not limit this.

[0142] In the embodiment of the present application, the AOI device is used to collect the to-be-processed image of the target circuit board and the standard image corresponding to the target circuit board; the key point matching is performed on the to-be-processed image and the standard image to determine the reference target; the corresponding relationship between the plurality of first targets in the to-be-processed image and the plurality of second targets in the standard image is determined based on the position information of the reference target in the to-be-processed image and the standard image; the plurality of first targets in the to-be-processed image are screened based on the corresponding relationship and the preset screening rule to obtain a plurality of key targets in the plurality of first targets that meet the preset screening rule; and the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the second targets corresponding to the plurality of key targets in the standard image.

[0143] Therefore, on the one hand, the target with the most matching key points in the to-be-processed image and the standard image is selected to correspond each target in the to-be-processed image and the standard image; on the other hand, the offset position and the offset rotation angle are determined based on the position information and the rotation angle of the key targets obtained by multiple screening based on the corresponding relationship, and the to-be-processed image is corrected, which increases the accuracy of detecting the target circuit board and reduces the offset of the to-be-processed image of the PCB board caused by the sliding window method. The influence of the to-be-processed image offset on the subsequent component defect detection is reduced.

[0144] Based on the same inventive concept, the embodiment of the present application also provides an image processing device, Figure 8 The structure diagram of the image processing device provided by the embodiment of the present application is shown in the figure, and the device comprises:

[0145] The acquisition module 801 is configured to acquire a to-be-processed image of a target circuit board and a standard image corresponding to the target circuit board by using an automatic optical inspection (AOI) device.

[0146] The matching module 802 is configured to perform key point matching on the to-be-processed image and the standard image, and determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most key points matched with a second target in the standard image.

[0147] The correspondence module 803 is configured to determine a correspondence between a plurality of first targets in the to-be-processed image and a plurality of second targets in the standard image based on position information of the reference target in the to-be-processed image and position information of the second target matched with the reference target in the standard image.

[0148] The screening module 804 is configured to screen the plurality of first targets in the to-be-processed image based on the correspondence and a preset screening rule, to obtain a plurality of key targets in the plurality of first targets that meet the preset screening rule.

[0149] The correction module 805 is configured to correct the to-be-processed image based on position information and a rotation angle of the plurality of key targets in the to-be-processed image and position information and a rotation angle of the second targets corresponding to the plurality of key targets in the standard image.

[0150] In a possible implementation, the matching module 802 is specifically configured to:

[0151] perform key point matching on the to-be-processed image and the standard image by using an image matching algorithm, to determine a key point pair matched in the to-be-processed image and the standard image.

[0152] determine a number of key point pairs corresponding to the plurality of first targets in the to-be-processed image based on the key point pair matched in the to-be-processed image and the standard image.

[0153] sort the number of key point pairs corresponding to the plurality of first targets in the to-be-processed image, to determine a first target with the most key point pairs in the to-be-processed image.

[0154] determine the first target with the most key point pairs in the to-be-processed image as the reference target.

[0155] In a possible implementation, the matching module 802 is further configured to: detect the to-be-processed image using a rotating target detection algorithm to obtain a plurality of first targets in the to-be-processed image and position information and a rotating angle of each first target in the to-be-processed image.

[0156] detect the standard image using the rotating target detection algorithm to obtain a plurality of second targets in the standard image and position information and a rotating angle of each second target in the standard image.

[0157] In a possible implementation, the corresponding module 803 is specifically configured to:

[0158] for each first target of the plurality of first targets in the to-be-processed image, perform the following operations:

[0159] determine a relative position of the first target and the reference target in the to-be-processed image based on the position information of the reference target in the to-be-processed image and the position information of the first target in the to-be-processed image;

[0160] determine a position information region of the second target corresponding to the first target in the standard image based on the relative position of the first target and the reference target in the to-be-processed image, the position information of the second target matched with the reference target in the standard image, and a preset deviation threshold;

[0161] take the second target in the position information region in the standard image as the second target corresponding to the first target.

[0162] In a possible implementation, the correction module 805 is specifically configured to:

[0163] determine an offset position of the to-be-processed image based on the position information of the plurality of key targets in the to-be-processed image and the position information of the plurality of key targets corresponding second targets in the standard image;

[0164] determine an offset rotating angle of the to-be-processed image based on the rotating angle of the plurality of key targets in the to-be-processed image and the rotating angle of the plurality of key targets corresponding second targets in the standard image;

[0165] correct the to-be-processed image based on the offset position and the offset rotating angle.

[0166] In a possible implementation, the correction module 805 is specifically configured to:

[0167] For each key target, the offset horizontal coordinate of the key target is determined based on the horizontal coordinate information of the key target in the image to be processed and the horizontal coordinate information of the second target corresponding to the key target in the standard image; the offset vertical coordinate of the key target is determined based on the vertical coordinate information of the key target in the image to be processed and the vertical coordinate information of the second target corresponding to the key target in the standard image.

[0168] The average of the offset x-coordinates of the multiple key targets is used as the offset x-coordinate of the image to be processed; the average of the offset y-coordinates of the multiple key targets is used as the offset y-coordinate of the image to be processed.

[0169] In one possible implementation, the correction module 805 is specifically used for:

[0170] For each key objective, perform the following operations:

[0171] The offset rotation angle of the key target is determined based on the rotation angle of the key target in the image to be processed and the rotation angle of the second target corresponding to the key target in the standard image.

[0172] Based on the same inventive concept, this application also provides an electronic device. Figure 9 This application provides a schematic diagram of an electronic device structure, such as... Figure 9 As shown, it includes: processor 901, communication interface 902, memory 903 and communication bus 904, wherein processor 901, communication interface 902 and memory 903 communicate with each other through communication bus 904.

[0173] The memory 903 stores a computer program, which, when executed by the processor 901, causes the processor 901 to perform the steps of any of the image processing methods provided in the embodiments of this application.

[0174] Since the problem-solving methods of the above-mentioned electronic devices are similar to those of image processing methods, the implementation of the above-mentioned electronic devices can be found in the embodiments of the methods, and repeated details will not be described again.

[0175] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 902 is used for communication between the above electronic device and other devices. The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0176] The processor mentioned above can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; can also be a Digital Signal Processing (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0177] On the basis of the above embodiments, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program executable by a processor. When the program runs on the processor, the processor executes the steps of any one of the image processing methods provided by the embodiments of the present application.

[0178] Based on the same inventive concept, the embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any one of the image processing methods provided by the embodiments of the present application are implemented.

[0179] Since the principle of solving the problem of the above computer readable storage medium is similar to the image processing method, the implementation of the above computer readable storage medium can refer to the embodiments of the method, and the repeated parts will not be described here.

[0180] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, various software modules in accordance with embodiments of the present application are stored in a memory such as a computer memory or disk storage for use by, or in connection with, the software on the computer system. The software can provide for programs to be transferred to another computer readable medium (e.g., a removable medium, or a medium conveyed through a computer network) for use in a different system.

[0181] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0182] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks.

[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0184] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a to-be-processed image of a target circuit board and a standard image corresponding to the target circuit board by using an automatic optical inspection (AOI) device; performing key point matching on the to-be-processed image and the standard image to determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most matching key points with a second target in the standard image; determining a correspondence between a plurality of first targets in the to-be-processed image and a plurality of second targets in the standard image based on position information of the reference target in the to-be-processed image and position information of the second target corresponding to the reference target in the standard image; performing screening on the plurality of first targets in the to-be-processed image based on the correspondence and a preset screening rule to obtain a plurality of key targets in the plurality of first targets that meet the preset screening rule; correcting the to-be-processed image based on position information and a rotation angle of the plurality of key targets in the to-be-processed image and position information and a rotation angle of the second targets corresponding to the plurality of key targets in the standard image.

2. The method of claim 1, wherein, The method comprises: performing key point matching on the to-be-processed image and the standard image to determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most matching key points with a second target in the standard image; performing key point matching on the to-be-processed image and the standard image to determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most matching key points with a second target in the standard image; performing key point matching on the to-be-processed image and the standard image to determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most matching key points with a second target in the standard image; performing key point matching on the to-be-processed image and the standard image to determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most matching key points with a second target in the standard image.

3. The method of claim 2, wherein, The method comprises: for each of the plurality of first targets in the to-be-processed image, performing the following operations: determining a relative position of the first target and the reference target in the to-be-processed image based on position information of the reference target in the to-be-processed image and position information of the first target in the to-be-processed image; determining a position information region of the second target corresponding to the first target in the standard image based on the relative position of the first target and the reference target in the to-be-processed image, position information of the second target corresponding to the reference target in the standard image, and a preset deviation threshold; taking the second target in the position information region in the standard image as the second target corresponding to the first target.

4. The method of claim 1, wherein, The preset screening rule is: the number of the first targets in the to-be-processed image that have a correspondence with the plurality of second targets in the standard image is greater than a preset threshold. The position information of the key target in the to-be-processed image is in a preset region; The key target has a corresponding second target.

5. The method of claim 1, wherein, The correction of the to-be-processed image based on the position information and the rotation angle of the plurality of key targets in the to-be-processed image and the position information and the rotation angle of the plurality of second targets corresponding to the plurality of key targets in the standard image comprises: Determining the offset position of the to-be-processed image based on the position information of the plurality of key targets in the to-be-processed image and the position information of the plurality of second targets corresponding to the plurality of key targets in the standard image; Determining the offset rotation angle of the to-be-processed image based on the rotation angle of the plurality of key targets in the to-be-processed image and the rotation angle of the plurality of second targets corresponding to the plurality of key targets in the standard image; Correcting the to-be-processed image based on the offset position and the offset rotation angle.

6. The method of claim 5, wherein, The position information comprises horizontal coordinate information and vertical coordinate information; and the determination of the offset position of the to-be-processed image based on the position information of the plurality of key targets in the to-be-processed image and the position information of the plurality of second targets corresponding to the plurality of key targets in the standard image comprises: For each key target, determining the offset horizontal coordinate of the key target based on the horizontal coordinate information of the key target in the to-be-processed image and the horizontal coordinate information of the second target corresponding to the key target in the standard image, and determining the offset vertical coordinate of the key target based on the vertical coordinate information of the key target in the to-be-processed image and the vertical coordinate information of the second target corresponding to the key target in the standard image; Taking the average of the offset horizontal coordinates of the plurality of key targets as the offset horizontal coordinate of the to-be-processed image, and taking the average of the offset vertical coordinates of the plurality of key targets as the offset vertical coordinate of the to-be-processed image.

7. The method of claim 5, wherein, The determination of the offset rotation angle of the to-be-processed image based on the rotation angle of the plurality of key targets in the to-be-processed image and the rotation angle of the plurality of second targets corresponding to the plurality of key targets in the standard image comprises: For each key target, the following operations are performed respectively: Determining the offset rotation angle of the key target based on the rotation angle of the key target in the to-be-processed image and the rotation angle of the second target corresponding to the key target in the standard image.

8. An image processing apparatus characterized by comprising: The apparatus comprises: The acquisition module is configured to acquire, by using an automatic optical inspection (AOI) device, a to-be-processed image of a target circuit board and a standard image corresponding to the target circuit board; The matching module is configured to perform key point matching on the to-be-processed image and the standard image, and determine a reference target; the reference target is a first target in the to-be-processed image, and the first target has the most key points matched with a second target in the standard image. A corresponding module is configured to determine a corresponding relationship between the first targets in the to-be-processed image and the second targets in the standard image based on the position information of the reference target in the to-be-processed image and the position information of the second target matched with the reference target in the standard image; A screening module is configured to screen the first targets in the to-be-processed image based on the corresponding relationship and a preset screening rule, and obtain a plurality of key targets in the first targets that meet the preset screening rule; A correction module is configured to correct the to-be-processed image based on the position information and the rotation angle of the key targets in the to-be-processed image and the position information and the rotation angle of the second targets corresponding to the key targets in the standard image.

9. An electronic device, comprising: The electronic device comprises a processor configured to implement the steps of the image processing method according to any one of claims 1-7 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is configured to implement the steps of the image processing method according to any one of claims 1-7 when executed by the processor.

Images