PCB (Printed Circuit Board) detection method and equipment based on image recognition
Through the image recognition method, the target recognition area of the PCB board is confirmed and the recognition model is trained, which solves the problem of rapid recognition of complex traceable PCB boards, and an efficient and accurate recognition process is achieved.
Patent Information
- Application Number
- CN202411818965.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-03-28
AI Technical Summary
The prior art is difficult to quickly and accurately identify complex trace PCB boards, resulting in high labor costs and large computing resources.
Using an image recognition method, the target recognition area of the PCB board is confirmed by marking the recognition model, and the target training image is obtained, and the PCB recognition model is trained using the image to reduce the calculation amount and improve the recognition accuracy.
It realizes fast and accurate identification of PCB boards, reduces labor costs and computing resource usage, and improves recognition accuracy.
Smart Images
Figure CN119723016A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of PCB board detection, and in particular to a PCB board detection method and equipment based on image recognition. Background Art
[0002] The inspection of PCB (Printed Circuit Board) generally relies on the professional knowledge and experience of technical workers, using visual inspection or inspection instruments, which will bring very high labor costs.
[0003] The development of image recognition technology has made it possible to automatically identify PCB boards. However, the wiring of PCB boards is complex. A whole PCB board has many complex wirings. The current CNN (Convolutional Neural Networks) is difficult to extract features on the entire PCB board and identify anomalies. Summary of the invention
[0004] Aiming at the problem of rapid identification of PCB boards, the present invention proposes a PCB board detection method and device based on image recognition.
[0005] In some embodiments, a PCB board detection method based on image recognition includes: Using the marker recognition model to recognize the marker information of the PCB training image, confirming that the first target area or the second target area of the PCB training image is the target recognition area and the coordinate information of the target recognition area; wherein the PCB training image includes the first target area and the second target area; The PCB training image is cut based on the coordinate information of the target recognition area to obtain a target training image; the image content of the target training image is the image content of the target recognition area; When the identified marking information corresponds to a first PCB recognition model, the target training image is used to train the first PCB recognition model; when the identified marking information corresponds to a second PCB recognition model, the target training image is used to train the second PCB recognition model; wherein the first PCB recognition model is used to perform defect recognition on the image content of the first target area, and the second PCB recognition model is used to perform defect recognition on the image content of the second target area; The defects of the PCB image are identified using the trained PCB recognition model, wherein the PCB recognition model includes a mark recognition model, a first PCB recognition model, and a second PCB recognition model.
[0006] In some embodiments, when the identified marking information corresponds to the first PCB recognition model, the first PCB recognition model is trained using the target training image; when the identified marking information corresponds to the second PCB recognition model, the second PCB recognition model is trained using the target training image, including: According to the label information of the identified PCB training image, the target training image is divided into a first PCB training database and a second PCB training database; wherein the first PCB training database includes the target training image and the label information, and the second PCB training database includes the target training image and the label information; The first PCB training database is used to train the first PCB recognition model, and the second PCB training database is used to train the second PCB recognition model.
[0007] In some embodiments, the marking information includes shape information and color information; using the marking recognition model to recognize the marking information of the PCB training image, confirming the target recognition area of the PCB training image and the coordinate information of the target recognition area, includes: Confirming the category of the PCB training image based on color information, where the color information includes at least two colors, corresponding to two target recognition areas of the PCB training image respectively; The coordinate information of the target recognition area is confirmed based on the category and shape information of the PCB training image.
[0008] In some embodiments, the tag information includes: first shape information, located at a first corner point of the target recognition area, the first shape information including a first ray and a second ray perpendicular to each other; and The second shape information is located at the second corner point of the target identification area, and the first corner point and the second corner point are located on the same diagonal of the target identification area. The second shape information includes a third ray and a fourth ray that are perpendicular to each other; wherein the extended lines of the first ray, the second ray, the third ray and the fourth ray enclose a first rectangular area.
[0009] In some embodiments, confirming the coordinate information of the target recognition area based on the category and shape information of the PCB training image includes: Based on the category of the PCB training image, confirm the coordinate information of the initial recognition area; Expand the width and height of the initial recognition area by the first length to obtain a second rectangular area; Use the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirm the target recognition area; the target recognition area includes the union area of the first rectangular area and the second rectangular area.
[0010] In some embodiments, using the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirming the target recognition area includes: Binarize the second rectangular area to obtain a binary image; Calculate the connected areas of the binary image; The union area of the connected area and the first rectangular area is used as the target recognition area.
[0011] In some embodiments, using the trained PCB recognition model to recognize defects in a PCB image includes: Controlling the camera to capture the PCB image, the PCB board has marking information, so that the captured PCB image has the marking information; Using the mark recognition model to recognize the mark information of the PCB image, confirming the target recognition area of the PCB image and the coordinate information of the target recognition area; The PCB image is cut based on the coordinate information of the target recognition area to obtain an image to be recognized; the image content of the image to be recognized is the image content of the target recognition area; According to the marking information of the PCB image, the corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
[0012] In some embodiments, controlling the camera to capture the PCB image includes: continuously capturing multiple PCB images; and calling the corresponding first PCB recognition model or the second PCB recognition model to recognize the image to be recognized according to the marking information of the PCB image to confirm the defect of the PCB image, including: Confirm the target recognition area of multiple PCB images; Confirm the image to be identified based on the target identification area of at least two PCB images; The corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
[0013] In some embodiments, based on the target recognition area of at least two PCB images, confirming the image to be recognized includes: The target recognition areas corresponding to the PCB images at the current moment and the previous moment are sent to the twin network, and the twin network outputs the similarity between the two target recognition areas; If the similarity between the two target recognition areas is greater than a first preset threshold, the target recognition area corresponding to the PCB image at the previous moment is cropped to obtain the image to be recognized; if the similarity between the two target recognition areas is less than the first preset threshold, the similarity between the target recognition areas corresponding to the PCB images at the next moment and the current moment is obtained, until the similarity between the two target recognition areas is less than the first preset threshold, the target recognition area corresponding to the PCB image at the current moment is cropped to obtain the image to be recognized.
[0014] The present application also proposes a device, which comprises: Camera, used to take PCB images; A memory storing a PCB board detection program based on image recognition; One or more processors are connected to the camera and the memory respectively. When the PCB board detection program based on image recognition is executed by the processor, the above-mentioned PCB board detection method based on image recognition is implemented.
[0015] When training a PCB recognition model, the present application first confirms the marking information through the marking recognition model, and cuts out the corresponding target training image without manually cutting the image one by one, which greatly reduces the labor cost. Then, based on the marking information, the target training image is used to train the first PCB recognition model or the second PCB recognition model, which reduces the amount of calculation, avoids mutual interference of training data (the first recognition area and the second recognition area), and improves the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a flowchart of an embodiment of a PCB board detection method based on image recognition of the present invention; Figure 2 A PCB image of an embodiment of a PCB board detection method based on image recognition of the present invention; Figure 3 It is a flowchart of another embodiment of the PCB board detection method based on image recognition of the present invention; Figure 4 A flowchart of another embodiment of a PCB board detection method based on image recognition according to the present invention; Figure 5 It is a flowchart of another embodiment of the PCB board detection method based on image recognition of the present invention; Figure 6 The figure is a flowchart of yet another embodiment of the PCB board detection method based on image recognition of the present invention. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0018] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0019] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.
[0020] Example 1 PCB automatic inspection equipment can identify whether the PCB board is qualified based on the collected images and output OK (normal) or NG (abnormal) as the identification result.
[0021] In actual use, in order to simplify the production process of PCB boards, multiple PCB wiring diagrams are often produced on the same PCB board. Later, users can choose the corresponding area to load parts (install resistors, capacitors, processors and other components) or cut the PDB board according to their needs.
[0022] In the above scenario, if the entire PCB board is directly tested, several problems will arise: 1. There is no abnormality in the area of the PCB board that needs to be used, but there is an abnormality in the area of the PCB board that does not need to be used. In this case, the PCB board recognition result should be OK, not NG; but the traditional recognition model will identify it as NG.
[0023] 2. Directly identifying the entire PCB board will result in a very large amount of calculation for the model at a time, occupying a lot of computing resources.
[0024] For the above problems, refer to Figure 1 The present application proposes a PCB board detection method based on image recognition, and the PCB board detection method based on image recognition may include training of a PCB recognition model and calling of a PCB recognition model. Specifically, in some embodiments, the PCB board detection method based on image recognition includes: S100, using a marker recognition model to recognize marker information of a PCB training image, confirming that a first target area or a second target area of the PCB training image is a target recognition area and coordinate information of the target recognition area; wherein the PCB training image includes a first target area and a second target area; Reference Figure 2 In the process of manufacturing PCB boards, the target identification area can be divided according to the area to be loaded or the corresponding PCB wiring diagram. For example, the PCB board includes areas AB, which correspond to the first PCB wiring diagram and the second PCB wiring diagram respectively. Then, on the PCB board, the area corresponding to the first PCB wiring diagram can be divided into the first identification area 1, and the area corresponding to the second PCB wiring diagram can be divided into the second identification area 2; similarly, the PCB board is adapted to 2 different products through different loading methods, then on the PCB board, the area corresponding to the first loading method can be divided into the first identification area 1, and the area corresponding to the second loading method can be divided into the second identification area 2; it should be noted that the above-mentioned first identification area 1 and the second identification area 2 can overlap or not overlap, which is not limited here.
[0025] The marking information may include color information. For example, if the marking is red, it means that the target identification area of the current PCB board is the first identification area 1. If the marking is blue, it means that the target identification area of the current PCB board is the second identification area 2. Regarding the coordinate information of the target identification area, the coordinate information of the first identification area 1 and the second identification area 2 can be stored in the memory in advance. Once the target identification area is confirmed to be the first target identification area, the coordinate information of the first identification area 1 is confirmed to be the coordinate information of the target identification area. Similarly, once the target identification area is confirmed to be the second identification area 2, the coordinate information of the second identification area 2 is confirmed to be the coordinate information of the target identification area.
[0026] Exemplarily, the marker recognition model can correspond to the complexity of the marker information. If the marker information only includes color and shape information, the RGB pixel value of the marker information can be directly analyzed, and the target training image can be binarized to obtain the connected area of the binarized image to obtain the shape of the marker information.
[0027] In the present application, the marker recognition module is set as a twin network. The input of the twin network includes marker information and pre-stored shape information. The twin network compares the similarity between the acquired marker information and the pre-stored shape information. When the similarity is greater than a threshold, it is confirmed that the shape of the marker information is the shape of the shape information.
[0028] Compared with convolutional neural networks and recurrent neural networks, this application uses twin networks for comparison, which has the advantages of small computational complexity and small number of model parameters.
[0029] S200, cutting the PCB training image based on the coordinate information of the target recognition area to obtain a target training image; the image content of the target training image is the image content of the target recognition area; It should be noted that if the entire PCB image is used to train the PCB recognition model, the amount of data for the PCB recognition model will be very large, making it difficult or impossible to embed it into the PCB detection equipment. At the same time, the PCB recognition model will also occupy a lot of computing resources when working.
[0030] This application first crops the PCB training image to obtain a target training image with a smaller resolution, and then uses the target training image to train the PCB recognition model. At this time, the PCB recognition model only inputs the target training image, so it does not carry other interference information (non-target recognition area of the PCB training image), which greatly reduces the number of model parameters and the amount of calculation during operation.
[0031] S300: When the identified marking information corresponds to a first PCB recognition model, the target training image is used to train the first PCB recognition model; when the identified marking information corresponds to a second PCB recognition model, the target training image is used to train the second PCB recognition model; wherein the first PCB recognition model is used to perform defect recognition on the image content of the first target area, and the second PCB recognition model is used to perform defect recognition on the image content of the second target area; Furthermore, in order to better reduce the amount of calculation of the model, the present application also reuses the tag information, and uses the recognition result of the tag information to select and call the first PCB recognition model and the second PCB recognition model; in this embodiment, the first PCB recognition model is a model trained using the first recognition area 1, so it can only recognize defects in the first recognition area 1, and the second PCB recognition model is a model trained using the second recognition area 2, so it can only recognize defects in the second recognition area 2; However, this also means that the first PCB recognition model only needs to recognize two types of PCB boards: normal PCB boards in the first recognition area 1 and abnormal PCB boards in the first recognition area 1. Similarly, the second PCB recognition model only needs to recognize normal PCB boards in the second recognition area 2 and abnormal PCB boards in the second recognition area 2. The original four categories (normal first recognition area 1, abnormal first recognition area 1, normal second recognition area 2, abnormal second recognition area 2) are transformed into two binary categories.
[0032] In addition, because the first PCB recognition model only needs to recognize two types of PCB boards, the recognition process will not be interfered by the second recognition area 2. Similarly, the second PCB recognition model will not be interfered by the first recognition area 1, thereby improving the recognition accuracy of the model.
[0033] Exemplarily, the first PCB recognition model may use a CNN+DNN+softmax structure, or an existing image recognition model such as Resnet50, or a LeNet-5 with a simpler structure; In this embodiment, the structures of the first PCB recognition model and the second PCB recognition model may be the same or different, and may be specifically selected according to the complexity of the first target area and the complexity of the second target area. For example, the wiring complexity of the first target area is much greater than the wiring complexity of the second target area. At this time, the number of network layers of the first PCB recognition model is greater than the number of network layers of the second PCB recognition model, or in other words, the number of parameters of the first PCB recognition model is greater than the number of parameters of the second PCB recognition model, so that the recognition ability of the first PCB recognition model is stronger than that of the second PCB recognition model. For example, the first PCB recognition model uses Resnet50 and the second PCB recognition model uses LeNet-5.
[0034] For example, This is not limited here.
[0035] S400: Use the trained PCB recognition model to recognize defects in the PCB image, wherein the PCB recognition model includes a mark recognition model, a first PCB recognition model, and a second PCB recognition model.
[0036] In practical applications, it can be divided into two steps: the tag recognition model recognizes the tag information, and then calls the first PCB recognition model or the second PCB recognition model corresponding to the tag information to identify the image.
[0037] The calling process can correspond to the training process. Similarly, using the PCB recognition model will also bring the same effect (improving accuracy and reducing calculation amount).
[0038] It should be noted that this application does not need to identify the type of defect, but only needs to identify whether there is a defect, or whether the defect can affect the use. This can be achieved by setting the division criteria of the training data when the standard is established.
[0039] When training a PCB recognition model, the present application first confirms the marking information through the marking recognition model, and cuts out the corresponding target training image without manually cutting the image one by one, which greatly reduces the labor cost. Then, based on the marking information, the target training image is used to train the first PCB recognition model or the second PCB recognition model, which reduces the amount of calculation and avoids mutual interference of training data (first recognition area 1 and second recognition area 2), thereby improving accuracy.
[0040] Reference Figure 3 In some embodiments, S300, when the identified tag information corresponds to the first PCB recognition model, the first PCB recognition model is trained using the target training image; when the identified tag information corresponds to the second PCB recognition model, the second PCB recognition model is trained using the target training image, including: S301, dividing the target training image into a first PCB training database and a second PCB training database according to the tag information of the identified PCB training image; wherein the first PCB training database includes the target training image and the tag information, and the second PCB training database includes the target training image and the tag information; S302: Use the first PCB training database to train a first PCB recognition model, and use the second PCB training database to train a second PCB recognition model.
[0041] In this embodiment, when preparing training data, the operator only needs to identify whether the corresponding target recognition area is abnormal based on the marking information, and then label it (abnormal / normal).
[0042] There is no need to mark each training data as the first recognition area 1 is normal, the first recognition area 1 is abnormal, the second recognition area 2 is normal, and the second recognition area 2 is normal, which greatly reduces the labor cost of labeling.
[0043] Reference Figure 3 In some embodiments, the marking information includes shape information and color information; S100, using a marking recognition model to recognize the marking information of a PCB training image, confirming a target recognition area of the PCB training image and coordinate information of the target recognition area, including: S101, confirming the category of the PCB training image based on color information, where the color information includes at least two colors, which respectively correspond to two target recognition areas of the PCB training image; The categories of PCB training images refer to different categories of areas to be loaded. When there are 3 or more types of loading areas, 3 or more colors can be set.
[0044] S102: confirming the coordinate information of the target recognition area based on the category and shape information of the PCB training image.
[0045] The coordinate information of the target recognition area can be confirmed by confirming the category of the PCB training image. However, the coordinate information of the target recognition area is further corrected by the shape design of the marking information in this embodiment.
[0046] Reference Figure 2 Specifically, the marking information includes: First shape information, located at a first corner point of the target recognition area, the first shape information includes a first ray 31 and a second ray 32 that are perpendicular to each other; and The second shape information is located at the second corner point of the target identification area, and the first corner point and the second corner point are located on the same diagonal of the target identification area. The second shape information includes a third ray 33 and a fourth ray 34 that are perpendicular to each other; wherein the extension lines of the first ray 31, the second ray 32, the third ray 33 and the fourth ray 34 enclose a first rectangular area.
[0047] In other words, the two marking information confirms a rectangular box, and no text information is needed to assist; Of course, in other embodiments, a corner point or a center point may be marked, and two parameters of the width and height of the rectangle may be added. However, this requires adjustment of the width and height parameters when used, whereas this is not necessary when using the marking information of two corner points, and the model can automatically confirm.
[0048] Reference Figure 4 In some embodiments, S102, confirming the coordinate information of the target recognition area based on the category and shape information of the PCB training image includes: S1021, confirming the coordinate information of the initial recognition area based on the category of the PCB training image; S1022, expanding the width and height of the initial recognition area by a first length to obtain a second rectangular area; In this embodiment, the first length may be related to the width or height of the initial recognition area. Specifically, the first length may be 10%, 20%, 30%, or 50% of the width, or 10%, 20%, 30%, or 50% of the height. Different length values may also be set along the width and height of the initial recognition area.
[0049] S1023, using the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirming a target recognition area; the target recognition area includes a union area of the first rectangular area and the second rectangular area.
[0050] It should be noted that the initial recognition area is a preset value. After confirming the category of the PCB training image, the model knows the coordinate information of the initial recognition area. At this time, it is possible to use the initial recognition area as the target recognition area; however, in order to improve the accuracy, this application uses the shape information of the marking information to correct the initial recognition area to ensure that the target recognition area completely includes the object to be recognized.
[0051] First, in order to ensure that all marks are included, the initial recognition area is expanded to a second rectangular area. At this time, the second rectangular area contains all the mark information. Then, the first rectangular area is confirmed based on the mark information (first shape information and second shape information). Then, the union area of the first rectangular area and the second rectangular area is calculated to obtain the corrected target recognition area.
[0052] Furthermore, the present application also uses image recognition technology to confirm the target recognition area. Specifically, S1023, using the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, confirming the target recognition area also includes: Binarize the second rectangular area to obtain a binary image; Calculate the connected areas of the binary image; The union area of the connected area and the first rectangular area is used as the target recognition area.
[0053] Reference Figure 5 In some embodiments, S400, using the trained PCB recognition model to identify defects in the PCB image includes: S401, controlling a camera to capture a PCB image, wherein the PCB board has marking information, so that the captured PCB image has the marking information; S402, using the mark recognition model to recognize the mark information of the PCB image, and confirming the target recognition area of the PCB image and the coordinate information of the target recognition area; S403, cutting the PCB image based on the coordinate information of the target recognition area to obtain an image to be recognized; the image content of the image to be recognized is the image content of the target recognition area; S404: According to the marking information of the PCB image, the corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
[0054] The steps of this embodiment correspond to the training steps of the PCB recognition model. The marking information of the PCB image can be first identified to confirm the category of the PCB image, and then the target recognition area of the PCB image can be confirmed. The target recognition area is then cut out to obtain the image to be recognized. Finally, according to the category of the PCB image, the image to be recognized is sent to the first PCB recognition model or the second PCB recognition model for recognition, so that the defects of the PCB image can be confirmed.
[0055] Reference Figure 6 In some embodiments, controlling the camera to capture the PCB image includes: continuously capturing multiple PCB images. For example, during operation, the camera can capture PCB images continuously, for example, 30 images per second. In this embodiment, in order to ensure the accuracy of the target recognition area, in addition to using the target recognition area confirmation method in the training process, the present application also confirms whether the target recognition area is accurate based on the difference between the target recognition areas at the previous and next moments. Specifically, S404: According to the marking information of the PCB image, calling the corresponding first PCB recognition model or the second PCB recognition model to recognize the image to be recognized, so as to confirm the defects of the PCB image, including: S4041, confirming the target recognition area of the multiple PCB images; The confirmation method refers to the confirmation method based on the marking information, the first rectangular area, and the second rectangular area during the training process, which will not be described in detail here.
[0056] S4042, confirming the image to be identified based on the target identification area of at least two PCB images; After confirming the target recognition area of at least two PCB images, it can be confirmed whether the difference between the target recognition areas of at least two PCB images meets the requirements. If the requirements are met, the image to be recognized is cropped according to the target recognition area of the current PCB image. If not, continue to shoot images until the requirements are met or the shooting reaches a preset time (for example, 2 minutes).
[0057] S4043. Call the corresponding first PCB recognition model or the second PCB recognition model to recognize the image to be recognized, so as to confirm defects of the PCB image.
[0058] Specifically, the first PCB recognition model or the second PCB recognition model may be called based on the type of the PCB board, which will not be described in detail here.
[0059] In some embodiments, S4042, based on the target recognition area of at least two PCB images, confirming the image to be recognized includes: The target recognition areas corresponding to the PCB images at the current moment and the previous moment are sent to the twin network, and the twin network outputs the similarity between the two target recognition areas; If the similarity between the two target recognition areas is greater than a first preset threshold, the target recognition area corresponding to the PCB image at the previous moment is cropped to obtain the image to be recognized; if the similarity between the two target recognition areas is less than the first preset threshold, the similarity between the target recognition areas corresponding to the PCB images at the next moment and the current moment is obtained, until the similarity between the two target recognition areas is less than the first preset threshold, the target recognition area corresponding to the PCB image at the current moment is cropped to obtain the image to be recognized.
[0060] It should be noted that compared with conventional models, the twin network can compare the Euclidean distance between the feature vectors obtained from two inputs, and then confirm the similarity between the two inputs; In this embodiment, the image to be identified is cropped only when the similarity between the two target identification areas before and after is confirmed to be high enough, for example, greater than 85%, 88%, 90%, 95%, or 98%. Otherwise, the image is collected continuously until a PCB image that meets the requirements is obtained, and the target identification area corresponding to the PCB image at this time is cropped to obtain the image to be identified.
[0061] Using multiple frames for comparison can avoid recognition errors caused by abnormalities in a single frame, which is also called device misdetection.
[0062] Example 2 The present application also proposes a device, which comprises: Camera, used to take PCB images; A memory storing a PCB board detection program based on image recognition; One or more processors are connected to the camera and the memory respectively. When the PCB board detection program based on image recognition is executed by the processor, the PCB board detection method based on image recognition is implemented. The method includes: Using the marker recognition model to recognize the marker information of the PCB training image, confirming that the first target area or the second target area of the PCB training image is the target recognition area and the coordinate information of the target recognition area; wherein the PCB training image includes the first target area and the second target area; The PCB training image is cut based on the coordinate information of the target recognition area to obtain a target training image; the image content of the target training image is the image content of the target recognition area; When the identified marking information corresponds to a first PCB recognition model, the target training image is used to train the first PCB recognition model; when the identified marking information corresponds to a second PCB recognition model, the target training image is used to train the second PCB recognition model; wherein the first PCB recognition model is used to perform defect recognition on the image content of the first target area, and the second PCB recognition model is used to perform defect recognition on the image content of the second target area; The defects of the PCB image are identified using the trained PCB recognition model, wherein the PCB recognition model includes a mark recognition model, a first PCB recognition model, and a second PCB recognition model.
[0063] In some embodiments, when the identified marking information corresponds to the first PCB recognition model, the first PCB recognition model is trained using the target training image; when the identified marking information corresponds to the second PCB recognition model, the second PCB recognition model is trained using the target training image, including: According to the label information of the identified PCB training image, the target training image is divided into a first PCB training database and a second PCB training database; wherein the first PCB training database includes the target training image and the label information, and the second PCB training database includes the target training image and the label information; The first PCB training database is used to train the first PCB recognition model, and the second PCB training database is used to train the second PCB recognition model.
[0064] In some embodiments, the marking information includes shape information and color information; using the marking recognition model to recognize the marking information of the PCB training image, confirming the target recognition area of the PCB training image and the coordinate information of the target recognition area, includes: Confirming the category of the PCB training image based on color information, where the color information includes at least two colors, corresponding to two target recognition areas of the PCB training image respectively; The coordinate information of the target recognition area is confirmed based on the category and shape information of the PCB training image.
[0065] In some embodiments, the tag information includes: First shape information, located at a first corner point of the target recognition area, the first shape information includes a first ray 31 and a second ray 32 that are perpendicular to each other; and The second shape information is located at the second corner point of the target identification area, and the first corner point and the second corner point are located on the same diagonal of the target identification area. The second shape information includes a third ray 33 and a fourth ray 34 that are perpendicular to each other; wherein the extension lines of the first ray 31, the second ray 32, the third ray 33 and the fourth ray 34 enclose a first rectangular area.
[0066] In some embodiments, confirming the coordinate information of the target recognition area based on the category and shape information of the PCB training image includes: Based on the category of the PCB training image, confirm the coordinate information of the initial recognition area; Expand the width and height of the initial recognition area by the first length to obtain a second rectangular area; Use the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirm the target recognition area; the target recognition area includes the union area of the first rectangular area and the second rectangular area.
[0067] In some embodiments, using the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirming the target recognition area includes: Binarize the second rectangular area to obtain a binary image; Calculate the connected areas of the binary image; The union area of the connected area and the first rectangular area is used as the target recognition area.
[0068] In some embodiments, using the trained PCB recognition model to recognize defects in a PCB image includes: Controlling the camera to capture the PCB image, the PCB board has marking information, so that the captured PCB image has the marking information; Using the mark recognition model to recognize the mark information of the PCB image, confirming the target recognition area of the PCB image and the coordinate information of the target recognition area; The PCB image is cut based on the coordinate information of the target recognition area to obtain an image to be recognized; the image content of the image to be recognized is the image content of the target recognition area; According to the marking information of the PCB image, the corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
[0069] In some embodiments, controlling the camera to capture the PCB image includes: continuously capturing multiple PCB images; and calling the corresponding first PCB recognition model or the second PCB recognition model to recognize the image to be recognized according to the marking information of the PCB image to confirm the defect of the PCB image, including: Confirm the target recognition area of multiple PCB images; Confirm the image to be identified based on the target identification area of at least two PCB images; The corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
[0070] In some embodiments, based on the target recognition area of at least two PCB images, confirming the image to be recognized includes: The target recognition areas corresponding to the PCB images at the current moment and the previous moment are sent to the twin network, and the twin network outputs the similarity between the two target recognition areas; If the similarity between the two target recognition areas is greater than a first preset threshold, the target recognition area corresponding to the PCB image at the previous moment is cropped to obtain the image to be recognized; if the similarity between the two target recognition areas is less than the first preset threshold, the similarity between the target recognition areas corresponding to the PCB images at the next moment and the current moment is obtained, until the similarity between the two target recognition areas is less than the first preset threshold, the target recognition area corresponding to the PCB image at the current moment is cropped to obtain the image to be recognized.
[0071] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0072] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0073] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0076] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0077] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A PCB board detection method based on image recognition, characterized in that: The method comprises: Using the marker recognition model to recognize the marker information of the PCB training image, confirming that the first target area or the second target area of the PCB training image is the target recognition area and the coordinate information of the target recognition area; wherein the PCB training image includes the first target area and the second target area; The PCB training image is cut based on the coordinate information of the target recognition area to obtain a target training image; the image content of the target training image is the image content of the target recognition area; When the identified marking information corresponds to a first PCB recognition model, the target training image is used to train the first PCB recognition model; when the identified marking information corresponds to a second PCB recognition model, the target training image is used to train the second PCB recognition model; wherein the first PCB recognition model is used to perform defect recognition on the image content of the first target area, and the second PCB recognition model is used to perform defect recognition on the image content of the second target area; The defects of the PCB image are identified using the trained PCB recognition model, wherein the PCB recognition model includes a mark recognition model, a first PCB recognition model, and a second PCB recognition model.
2. The PCB board detection method based on image recognition as claimed in claim 1, characterized in that: When the identified marking information corresponds to a first PCB recognition model, training the first PCB recognition model using the target training image; When the identified marking information corresponds to a second PCB recognition model, training the second PCB recognition model using the target training image includes: According to the label information of the identified PCB training image, the target training image is divided into a first PCB training database and a second PCB training database; wherein the first PCB training database includes the target training image and the label information, and the second PCB training database includes the target training image and the label information; The first PCB training database is used to train the first PCB recognition model, and the second PCB training database is used to train the second PCB recognition model.
3. The PCB board detection method based on image recognition as claimed in claim 2, characterized in that: The marking information includes shape information and color information; the step of using the marking recognition model to recognize the marking information of the PCB training image, and confirming the target recognition area of the PCB training image and the coordinate information of the target recognition area includes: Determining the category of the PCB training image based on the color information, wherein the color information includes at least two colors, respectively corresponding to two target recognition areas of the PCB training image; The coordinate information of the target recognition area is confirmed based on the category and shape information of the PCB training image.
4. The PCB board detection method based on image recognition as claimed in claim 3, characterized in that: The marking information includes: first shape information, located at a first corner point of the target recognition area, the first shape information comprising a first ray and a second ray perpendicular to each other; and The second shape information is located at the second corner point of the target identification area, and the first corner point and the second corner point are located on the same diagonal of the target identification area, and the second shape information includes a third ray and a fourth ray that are perpendicular to each other; wherein the extended lines of the first ray, the second ray, the third ray and the fourth ray enclose a first rectangular area.
5. The PCB board detection method based on image recognition as claimed in claim 4, characterized in that: The step of confirming the coordinate information of the target recognition area based on the category and shape information of the PCB training image includes: Based on the category of the PCB training image, confirming the coordinate information of the initial recognition area; Expand the width and height of the initial recognition area by a first length to obtain a second rectangular area; Use the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirm the target recognition area; the target recognition area includes the union area of the first rectangular area and the second rectangular area.
6. The PCB board detection method based on image recognition as claimed in claim 5, characterized in that: The step of using the recognition model to recognize the screen content of the second rectangular area and the first rectangular area confirmed by the first shape information and the second shape information, and confirming the target recognition area comprises: Binarizing the second rectangular area to obtain a binary image; Calculating the connected areas of the binary image; A union area of the connected area and the first rectangular area is used as a target recognition area.
7. The PCB board detection method based on image recognition as claimed in claim 6, characterized in that: The method of using the trained PCB recognition model to recognize defects in the PCB image includes: Controlling a camera to capture a PCB image, wherein the PCB board has marking information, so that the captured PCB image has the marking information; Using the mark recognition model to recognize the mark information of the PCB image, confirm the target recognition area of the PCB image and the coordinate information of the target recognition area; The PCB image is cut based on the coordinate information of the target recognition area to obtain an image to be recognized; the image content of the image to be recognized is the image content of the target recognition area; According to the marking information of the PCB image, the corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
8. The PCB board detection method based on image recognition as claimed in claim 7, characterized in that: The controlling the camera to shoot the PCB image includes: shooting a plurality of PCB images continuously; and calling the corresponding first PCB recognition model or the second PCB recognition model to recognize the image to be recognized according to the marking information of the PCB image to confirm the defect of the PCB image, including: confirming target identification areas of the plurality of PCB images; Confirming an image to be identified based on target identification areas of at least two of the PCB images; The corresponding first PCB recognition model or the second PCB recognition model is called to recognize the image to be recognized, so as to confirm the defects of the PCB image.
9. The PCB board detection method based on image recognition as claimed in claim 8, characterized in that: The step of confirming the image to be identified based on the target identification area of at least two of the PCB images includes: The target recognition areas corresponding to the PCB images at the current moment and the previous moment are sent to the twin network, and the twin network outputs the similarity between the two target recognition areas; If the similarity between the two target recognition areas is greater than a first preset threshold, the target recognition area corresponding to the PCB image at the previous moment is cut to obtain the image to be recognized; if the similarity between the two target recognition areas is less than the first preset threshold, the similarity between the target recognition areas corresponding to the PCB image at the next moment and the current moment is obtained, until the similarity between the two target recognition areas is less than the first preset threshold, the target recognition area corresponding to the PCB image at the current moment is cut to obtain the image to be recognized.
10. A device, characterized in that: include: Camera, used to take PCB images; A memory storing a PCB board detection program based on image recognition; One or more processors are connected to the camera and the memory respectively. When the image recognition-based PCB board detection program is executed by the processor, the image recognition-based PCB board detection method according to any one of claims 1 to 9 is implemented.