Process method for realizing automatic comparison of PCB characters

Abstract

The invention discloses a process method for realizing automatic comparison of PCB characters. The process method comprises the steps of image preprocessing, image registration based on serial number positions, text detection, text recognition, character extraction and contrastive analysis which are carried out in sequence. The graph line preprocessing comprises the following steps: cutting a redundant region, and meanwhile, carrying out binarization processing on an acquired PCB real object image to obtain a scanning graph; image registration based on serial number positions: using a deep learning algorithm to extract characters in the PCB as feature points to carry out registration on the schematic diagram and the scanning diagram; a multi-classification and attention mechanism is introduced to improve the network, and text detection, text recognition and character extraction are performed on the schematic diagram and the scanning diagram after registration; and comparing and analyzing, summarizing character extraction results of the schematic diagram and the scanning diagram, and judging error types. Therefore, the workload of workers is greatly reduced, the characters on the PCB can be accurately and efficiently detected, and errors such as wrong printing, through printing, printing deviation and excessive printing can be detected.

Description

Technical Field

[0001] The present invention is an innovation in the inspection method during the production process of electronic products, and is applicable to the detection of the correctness and accuracy of character printing on physical PCB boards. Background Art

[0002] A printed circuit board (PCB) is a carrier for various electronic components. By detecting the physical characters on the PCB board and comparing them with the schematic diagram, errors and defects that occur during the production process of the printed circuit board can be discovered in advance, thereby avoiding quality problems in the subsequent production process and greatly improving production efficiency. The complexity, density, component miniaturization, and area of aerospace product PCB boards are also increasing day by day. The traditional manual inspection of PCB boards mainly relies on the naked eye and can no longer meet the time efficiency requirements for the development of aerospace equipment. In order to improve the production efficiency of products and ensure the delivery nodes of aerospace model products, it is urgent to make the inspection of PCB characters intelligent.

[0003] In the published literature and periodicals, no invention patents that can solve the above problems have been found. There are the following several inventions that are slightly related, and the specific content is as follows:

[0004] "An Industrial Silk Screen Defect Detection Method and System Based on Sample Comparison", Application No.: CN202311264156.6, Authorization Publication Date: April 30, 2024. This patent establishes a positive sample library. For each sample to be detected, the top 3 positive samples that are most similar to it in the positive sample library are found for comparison, and the absolute value difference between each positive sample and the sample to be detected is obtained to get a difference map. This method is only for the detection of appearance defects and cannot detect and identify characters. The present invention uses a deep learning method for character recognition and comparison in the case of too many combinations of characters, which is different from the present invention.

[0005] "A Casting Text Detection and Recognition Method Based on Deep Learning", Application No.: CN202311175374.2, Application Publication Date: December 12, 2023. This patent proposes an automated casting text detection and recognition method. However, this method requires matching the recognition results with a database, and this method may have a situation where the database does not match the PCB due to the schematic diagram being modified but the database not being updated in time; at the same time, the method of matching through the database can only be used to detect characters and cannot be extended to other detection purposes, such as pad detection and polarity detection.

[0006] "A PCB Product Appearance Detection Device and Its Detection Method", Application No.: CN202311598948.7, Authorization Announcement Date: 2024-01-26. This invention proposes a PCB product appearance detection device that only targets the physical appearance of the PCB physical object and does not involve character recognition and comparison, which is different from the present invention. Summary of the Invention

[0007] The present invention proposes a process method for realizing automatic comparison of PCB characters. The correctness and accuracy of character printing on the PCB board are often detected manually by visual inspection. This detection method not only has low efficiency, but also is prone to detection errors due to the fatigue caused by the workers' long-term staring. To improve this situation, the present invention proposes a fast automatic comparison process method for PCB board characters, which is improved from the original manual visual inspection to machine automatic inspection, reducing labor consumption and improving detection efficiency and accuracy.

[0008] To achieve the above object, the technical solutions adopted by the present invention include:

[0009] A process method for realizing automatic comparison of PCB characters, including image preprocessing, image registration based on numbered positions, text detection, text recognition, character extraction, and comparative analysis in sequence;

[0010] The figure line preprocessing includes: trimming the redundant areas in the collected PCB physical object image and the schematic diagram, and at the same time performing binarization processing on the collected PCB physical object image to obtain a scanned image;

[0011] Image registration based on numbered positions: Using a deep learning algorithm to extract the characters in the PCB board as feature points to register the schematic diagram and the scanned image;

[0012] Then, a multi-classification and attention mechanism is introduced into the deep learning algorithm to improve the network, and text detection, text recognition, and character extraction are performed on the registered schematic diagram and scanned image;

[0013] Comparative analysis: Summarize the character extraction results of the schematic diagram and the scanned image, and based on the character positions in the schematic diagram, perform character search in a certain neighborhood of the corresponding position in the scanned image and discriminate the error types.

[0014] Optionally, the image registration based on numbered positions specifically includes:

[0015] First, use the target detection network Yolo-v5 to detect the schematic diagram and the PCB board scanned image, take the positions of the characters in the schematic diagram and the PCB board scanned image as text feature points, then take the content of the characters as feature descriptors for feature point matching, and finally calculate the transformation matrix through the matched feature points to register the schematic diagram and the PCB board scanned image.

[0016] Optionally, the text detection specifically includes:

[0017] Continuously train the target detection network Yolo-v5 for text position detection;

[0018] Add an SE attention module to the last layer of the backbone network CSPDarknet of the target detection network Yolo-v5 to adjust the weights of each channel of the feature map, enabling the model to pay more attention to channels with a large amount of information and suppressing irrelevant channels;

[0019] Optionally, multi-classification is also used to distinguish texts in different directions, and then the text is rotated to 0° according to the detected angle for text recognition. The specific method is as follows: use the angle of the text as its class label, and the specific classes are 0°, 45°, 90°, 135°, and 180°. The network outputs the angle of the text while detecting the text position.

[0020] Optionally, the text recognition includes:

[0021] Adopt the text recognition network Paddleocr, establish a sample set for the text types in the schematic diagram and scanned image, and then train and test the recognition model to obtain the text recognition network model.

[0022] Optionally, the character extraction specifically includes:

[0023] After text recognition, a number extraction module is needed to screen out reliable results from the text recognition results. The component numbers follow the combination method of "letter + number". Judge each character of the text recognition result one by one. If the text recognition conforms to the above format, the result is determined to be valid and the subsequent comparison and analysis link is carried out; if not, it is not processed.

[0024] Optionally, the use of the number extraction module to screen out reliable results from the text recognition results specifically includes:

[0025] Adopt the text recognition network Paddleocr, and the recognition module extracts the text recognition result to obtain the recognition result Y;

[0026] Judge whether the length of Y is greater than 1. If it is greater than 1, perform subsequent operations; otherwise, recognize it as not a number;

[0027] For the result where the length of Y is greater than 1, recognize whether the first character of Y is a letter. If it is not a letter, recognize it as not a number; if it is a letter, then recognize whether all characters of Y except the first letter are numbers. If they are numbers, recognize it as a number; otherwise, recognize it as not a number;

[0028] For the result where the length of Y is greater than 1, it is also necessary to identify whether the first two letters of Y are XB / XJ. If they are not XB / XJ, it is identified as not a number; if they are XB / XJ, then it is necessary to identify whether all characters of Y except the first two are numbers. If they are numbers, it is identified as a number, otherwise it is identified as not a number.

[0029] Optionally, the comparative analysis specifically includes:

[0030] Summarize the character extraction results of the schematic diagram and the scanned diagram, including the positions and contents of the characters. Based on the character positions in the schematic diagram, search for characters in a certain neighborhood of the corresponding positions in the scanned diagram and discriminate the error types. The error types include printing deviation, misprinting, missing printing, and / or overprinting.

[0031] The beneficial effects of the present invention compared with the prior art are as follows:

[0032] (1) Greatly reduce the workload of manual labor, can accurately and efficiently detect the characters on the PCB, and can detect errors such as misprinting, missing printing, printing deviation, and overprinting.

[0033] (2) According to the character characteristics in the PCB board, the present invention constructs a complete PCB character database, and replaces the way of manually collecting data with the way of machine-synthesizing training data, which greatly reduces the human consumption in the data collection stage and improves the recognition accuracy at the same time.

[0034] (3) Based on the text detection fusion strategy of "attention algorithm + detection network", the present invention combines the advantages of machine vision algorithms and deep learning. Through fusion and mutual verification, the reliability of text detection is improved. At the same time, a multi-classification strategy is introduced to realize the detection of texts at different angles, further improving the reliability and practicability of this patent. Description of the Drawings

[0035] The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:

[0036] Figure 1 is the image registration flowchart of the PCB character detection and comparison process method based on deep learning of the present invention;

[0037] Figure 2 is the original image and the binarized result image of the PCB scanned image, corresponding to the image preprocessing link in the technical solution;

[0038] Figure 3 is the registration effect schematic diagram of the schematic diagram and the scanned diagram of the present invention, corresponding to the image registration link based on the number position in the technical solution;

[0039] Figure 4 Schematic diagram of the text recognition effect of the present invention, corresponding to the text recognition step in the technical solution;

[0040] Figure 5 Flowchart of the number extraction algorithm of the present invention;

[0041] Figure 6 Diagram showing the character comparison effect of the present invention;

[0042] Figure 7 Comparison result between the scanned image and the schematic diagram of the present invention. Detailed implementation mode

[0043] The following detailed description of the present invention is exemplary and does not limit the scope of implementation of the present invention. Based on the embodiments in the present invention, for those of ordinary skill in the art, all improvements that do not make creative contributions should be regarded as the protection scope of the present invention.

[0044] In recent years, the development and application of machine vision and artificial intelligence technologies have become increasingly widespread, but there are relatively few related applications in the aerospace field. Therefore, for the characteristics and requirements of aerospace product PCBs, a process method for realizing automatic comparison of PCB characters is studied and designed based on machine vision and artificial intelligence technologies. Through the research on machine vision and artificial intelligence technologies of the present invention, a process method for realizing automatic comparison of PCB characters is designed, which significantly improves the inspection efficiency of PCB characters used in aerospace electronic products.

[0045] The process method for realizing automatic comparison of PCB characters of the present invention mainly includes image preprocessing, image registration based on the number position, text detection, text recognition, character extraction, and comparative analysis. First, the redundant areas in the collected PCB physical image and the schematic diagram are trimmed, and at the same time, the collected PCB physical image is binarized. Then, a deep learning algorithm is used to stably extract the characters in the PCB board as feature points to register the schematic diagram and the scanned image. Then, a multi-classification and attention mechanism is introduced into the detection network to improve the network for text detection and extraction. Finally, the character extraction results (including the position and content of the characters) of the assembly drawing and the scanned image are summarized, and based on the character position in the schematic diagram (such as the upper left coordinate, width, height, etc.), character search is performed within a certain neighborhood of the corresponding position in the scanned image and the error type is discriminated.

[0046] The specific implementation steps of the present invention are as follows:

[0047] (1) Image preprocessing

[0048] Since the schematic diagram is a binary image while the PCB image to be detected is generally in color and their image types are different, preprocessing and binarization are required. At the same time, there are redundant regions in both the schematic diagram and the PCB board scan image. Therefore, image preprocessing needs to be performed on both of them to crop the existing redundant regions and binarize the scan image to obtain a binary PCB board scan image. For details, see Figure 2 , Figure 2 The left side is a color picture, and the right side is the scan image obtained after binarization.

[0049] (2) Image registration based on the numbered positions

[0050] The traditional SIFT feature point detection and matching effect is not stable enough, and there are phenomena such as inaccurate feature point detection and false matching of feature points. In addition, due to the inconsistent sources of the input images and possible problems such as local printing tolerances and temporary modifications of the layout and wiring in the scan image. Therefore, for this matching scenario, first use the object detection network Yolo-v5 to detect the schematic diagram and the PCB board scan image, take the positions of the characters in the schematic diagram and the PCB board scan image as text feature points, then take the content of the characters as feature descriptors for feature point matching, and finally calculate the transformation matrix through the matched feature points to register the schematic diagram and the PCB board scan image. The proposed registration method can stably achieve the registration between the schematic diagram and the PCB board scan image. The flowchart of the registration algorithm is as shown in Figure 1 shown, and the effect display can be seen in Figure 3 .

[0051] (3) Text detection

[0052] Due to the dense and multi-directional characteristics of the text areas on the PCB board, it is easy to have situations such as incomplete characters, incomplete numbers, and incorrect text directions in the detected text areas. To address the above problems, the idea of "attention algorithm + detection network" is adopted. On the one hand, samples are gradually synthesized, and the target detection network Yolo-v5 is continuously trained for text position detection to improve the integrity and accuracy of text detection. On the other hand, considering the complex texture and dense text on the PCB board, an attention mechanism is proposed to be introduced into the detection network to make the network pay more attention to the text areas. The specific approach is as follows: an SE attention module is added to the last layer of the Yolo-v5 backbone network CSPDarknet. The SE attention module can adjust the weights of each channel of the feature map, enabling the model to pay more attention to the channels with a large amount of information and suppressing the features of irrelevant channels. In addition, multi-classification is used to distinguish texts in different directions, and then the text is rotated to 0° according to the detected angle for text recognition. The specific approach is as follows: the angle of the text is used as its class label, and the specific classes are 0°, 45°, 90°, 135°, and 180°. Therefore, the network can output the angle of the text while detecting the text position. Through training with an expanded training set and adding an attention mechanism to the detection network Yolo-v5, the accuracy and reliability of text detection have been significantly improved.

[0053] (4) Text recognition

[0054] This module is used to recognize the detected text areas. Here, the text recognition network Paddleocr is adopted. A sample set is established for the text types in assembly drawings and scanned images, and then the recognition model is trained and tested to achieve a fast and accurate text recognition network model. The display results are shown in Figure 4 .

[0055] In the initial stage, the number of training samples, especially the number of difficult samples, is small, and it is difficult to form big data, which affects model training. Therefore, on the basis of trying to accumulate samples, a large number of samples are synthesized through algorithms to meet the needs of the initial stage. For misrecognized characters, corresponding sample augmentation strategies can be established to suppress background noise and interference, thereby improving the network's recognition ability for such characters.

[0056] (5) Number extraction

[0057] Different from general text, component numbers are special format texts starting with letters such as R / L / C, followed by a string of numbers, representing resistors, capacitors, inductors and other components on the PCB board. After text recognition, a number extraction module is required to filter out reliable results from the text recognition results. Component numbers follow the combination pattern of "letter + number". Therefore, we design an algorithm to judge each character of the text recognition result. If the text recognition conforms to the above format, the result is determined to be valid and the subsequent comparison and analysis process is carried out; if not, it is not processed. Specifically, it includes:

[0058] First, the recognition module extracts the text recognition result to obtain the recognition result Y, for example, using the text recognition network Paddleocr;

[0059] Judge whether the length of Y is greater than 1. If it is greater than 1, perform subsequent operations; otherwise, it is recognized as not a number;

[0060] For the result with the length of Y greater than 1, recognize whether the first character of Y is a letter. If it is not a letter, it is recognized as not a number; if it is a letter, then recognize whether all characters of Y except the first letter are numbers. If they are numbers, it is recognized as a number; otherwise, it is recognized as not a number;

[0061] For the result with the length of Y greater than 1, also recognize whether the first two letters of Y are XB / XJ. If they are not XB / XJ, it is recognized as not a number; if they are XB / XJ, then recognize whether all characters of Y except the first two characters are numbers. If they are numbers, it is recognized as a number; otherwise, it is recognized as not a number.

[0062] Through the above number extraction algorithm (the algorithm flow is as Figure 5 shown), the situation of false detection can be effectively reduced.

[0063] (6) Comparison and analysis

[0064] Table 1 Discrimination rules for character error types

[0065]

[0066]

[0067] This module first summarizes the character extraction results (including the position and content of characters) of the assembly drawing and the scanned drawing to form a table, as shown in Table 1. Then, based on the character positions (such as the upper left corner coordinates, width, height, etc.) of the schematic diagram, character search is carried out in a certain neighborhood of the corresponding position of the scanned drawing. Taking the string R123 as an example, the error type is described and the discrimination rules are explained, where T is a preset parameter.

[0068] Applying the method of the present invention to the character extraction processing of a specific PCB board, we getFigure 6 and 7 The results shown below:

[0069] Figure 6 show four types of errors that can be detected by the present invention. The comparison between the schematic diagram and the scanned diagram is shown in the grid. In the detection of this PCB board, there are actually 13 printing problems, including 10 misprints, 1 printing deviation, and 2 missing prints. Among them Figure 7 the detection results show that there are 11 misprints, and the misprint detection rate is 100%; 1 printing deviation, and the printing deviation detection rate is 100%; 2 missing prints, and the missing print detection rate is 100%; there is only one false detection of misprint.

[0070] Although the present invention has been described in detail with general descriptions and specific embodiments above, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the protection scope of the present invention.

Claims

1. A process method for realizing automatic comparison of PCB characters, characterized in that: It includes image preprocessing, image registration based on number position, text detection, text recognition, character extraction and comparative analysis in sequence; The image line preprocessing includes: cutting the redundant areas in the collected PCB physical image and the schematic diagram, and binarizing the collected PCB physical image to obtain a scanned image; Image registration based on number position: Use deep learning algorithm to extract characters in PCB board as feature points to register schematic diagram and scanned image; Then, multi-classification and attention mechanisms are introduced into the deep learning algorithm to improve the network, and text detection, text recognition and character extraction are performed on the registered schematics and scanned images; Comparative analysis is performed, and the character extraction results of the schematic diagram and the scanned image are summarized. Based on the character position of the schematic diagram, a character search is performed within a certain neighborhood of the corresponding position of the scanned image, and the error type is identified.

2. The process method for realizing automatic PCB character comparison according to claim 1, characterized in that: The image registration based on number position specifically includes: Firstly, the Yolo-v5 target detection network is used to detect the schematic diagram and the PCB board scan. The positions of the characters in the schematic diagram and the PCB board scan are used as text feature points. Then the content of the characters is used as the feature descriptor for feature point matching. Finally, the transformation matrix is ​​calculated through the matched feature points to align the schematic diagram and the PCB board scan.

3. The process for realizing automatic PCB character comparison according to claim 1 or 2, characterized in that: The text detection specifically includes: Continue to train the object detection network Yolo-v5 for text location detection; The SE attention module is added to the last layer of the CSPDarknet backbone network of the target detection network Yolo-v5 to adjust the weight of each channel of the feature map, so that the model pays more attention to channels with large amounts of information and suppresses irrelevant channels.

4. The process method for realizing automatic PCB character comparison according to claim 3, characterized in that: Multi-classification is also used to distinguish texts in different directions, and then the text is rotated to 0° according to the detected angle for text recognition. The specific approach is: the angle of the text is used as its category label, and the specific categories are 0°, 45°, 90°, 135° and 180°. The network outputs the angle of the text while detecting the text position.

5. The process for realizing automatic PCB character comparison according to claim 1 or 2, characterized in that: The text recognition includes: The text recognition network Paddleocr is used to establish a sample set for the text types in schematics and scanned images, and then the recognition model is trained and tested to obtain the text recognition network model.

6. The process for realizing automatic PCB character comparison according to claim 1 or 2, characterized in that: The character extraction specifically includes: After text recognition, you need to use the number extraction module to filter out reliable results from the text recognition results. The component number follows the combination of "letters + numbers" and the text recognition results are judged character by character. If the text recognition conforms to the above format, the result is judged to be valid and the subsequent comparison and analysis is carried out; if it does not conform, it will not be processed.

7. The process for realizing automatic PCB character comparison according to claim 6, characterized in that: The method of using the serial number extraction module to select reliable results from the text recognition results specifically includes: The text recognition network Paddleocr is used, and the recognition module extracts the text recognition result to obtain the recognition result Y; Determine whether the length of Y is greater than 1. If so, perform subsequent operations; otherwise, identify it as not a number. For the result that the length of Y is greater than 1, identify whether the first character of Y is a letter. If it is not a letter, identify it as not a number. If it is a letter, identify whether all characters of Y except the first letter are numbers. If it is a number, identify it as a number. Otherwise, identify it as not a number. For the result that the length of Y is greater than 1, it is also identified whether the first two letters of Y are XB / XJ. If they are not XB / XJ, it is identified as not a number. If they are XB / XJ, it is identified whether all characters of Y except the first two are numbers. If they are numbers, it is identified as a number, otherwise it is identified as not a number.

8. The process for realizing automatic PCB character comparison according to claim 1 or 2, characterized in that: The comparative analysis specifically includes: The character extraction results of the schematic diagram and the scanned image, including the position and content of the characters, are summarized. Based on the character position of the schematic diagram, a character search is performed within a certain neighborhood of the corresponding position of the scanned image and the error type is identified. The error types include printing deviation, misprinting, missing printing and / or over-printing.

Images