PCB multi-parameter synchronous detection method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of visual inspection, and discloses a PCB multi-parameter synchronous detection method, electronic equipment and a storage medium, and the method comprises the steps: generating a PCB elevation map through the fusion of a 2D image and 3D point cloud data, carrying out the synchronous comparison analysis of the height and contour of a component installation position through the combination of the standard height and contour information in a design file, and carrying out the detection of the 3D point cloud data. And integrated detection is realized. According to the method, by introducing the elevation map generated by the three-dimensional point cloud data, the defect that 2D detection lacks depth information is made up; meanwhile, due to space-time alignment fusion of 2D and 3D data, high-resolution details of a two-dimensional image are reserved, height dimension information is increased, and the coverage dimension and precision of detection are remarkably improved; meanwhile, according to the method, through multi-parameter synchronous analysis, matching judgment of the height and the contour is completed at the same time in one detection process, the detection efficiency is improved, and the missed detection risk caused by multiple times of independent detection is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual detection, and in particular to a PCB multi-parameter synchronous detection method, an electronic device and a storage medium. BACKGROUND

[0002] As the core carrier of electronic devices, the mounting quality and physical structure stability of printed circuit boards (PCB) directly determine the reliability and service life of terminal products. With the rapid development of the electronic information industry towards high density, miniaturization and integration, the mounting density of components on PCBs is continuously increasing, and the pin pitch is continuously decreasing, which puts forward strict requirements on the precision, efficiency and multi-dimensional coverage of mounting quality detection. At present, in the production and manufacturing process of PCBs, the quality detection of mounted components has become a key link to ensure product yield.

[0003] In related technologies, traditional 2D visual detection methods are mainly used. This method analyzes the collected planar image information, but due to the limited parameter dimension, it cannot comprehensively evaluate the production quality of PCBs, and it is difficult to meet the high requirements of the current industry for defect detection rate. SUMMARY

[0004] The main purpose of the present application is to provide a PCB multi-parameter synchronous detection method, an electronic device and a storage medium, which aims to solve the technical problem that the 2D visual detection method in the prior art cannot comprehensively evaluate the production quality of PCBs.

[0005] To achieve the above-mentioned purpose, in a first aspect, a PCB multi-parameter synchronous detection method is provided in the embodiments of the present application, comprising:

[0006] Obtain two-dimensional detection data and three-dimensional detection data of a target PCB, to obtain 2D image data and 3D point cloud data;

[0007] According to the conversion matrix, the 2D image data and the 3D point cloud data are spatio-temporally aligned and fused to obtain a PCB elevation map of the target PCB, wherein the pixel value of a pixel point in the PCB elevation map represents the height value of the corresponding pixel point;

[0008] Obtain the design file of the target PCB, determine the position coordinates of all preset component mounting positions on the target PCB, and the height threshold range and standard 2D contour of the corresponding standard components;

[0009] According to the position coordinates, traverse the pixel region corresponding to all preset component mounting positions in the PCB elevation map, extract the pixel value of each mounting position pixel region, and convert the actual height value of each mounting position according to the mapping relationship between the pixel value and the height value;

[0010] The actual height value of each mounting position is compared with the height threshold range of the corresponding standard component to obtain a height matching result.

[0011] An actual contour corresponding to each mounting position in the PCB elevation map is extracted and matched with a standard 2D contour of the corresponding mounting position in the design file to obtain a contour matching result. The first detection result is obtained by synchronous detection according to the contour matching result and the height matching result.

[0012] In a possible implementation, the spatio-temporal alignment and fusion of the 2D image data and the 3D point cloud data according to the conversion matrix to obtain the PCB elevation map of the target PCB board comprises:

[0013] Each 3D point in the 3D point cloud data is mapped to a corresponding image coordinate system of the 2D image data by using the conversion matrix to obtain a 2D pixel coordinate corresponding to each 3D point.

[0014] A two-dimensional grid corresponding to each 2D image pixel is constructed based on the resolution of the 2D image data, and each 3D point is assigned to a corresponding grid cell according to the 2D pixel coordinate thereof.

[0015] For each grid cell, if multiple 3D points fall into the grid cell, the average value of the height values of all the 3D points is calculated as the final height value of the grid cell. If no 3D point falls into the grid cell, a preset filling rule is used for filling the missing data.

[0016] According to a preset height-pixel value mapping rule, the final height value of each grid cell is converted into a corresponding pixel value to generate the PCB elevation map of the target PCB board.

[0017] In a possible implementation, the extraction of the actual contour corresponding to each mounting position in the PCB elevation map and the matching of the actual contour with the standard 2D contour of the corresponding mounting position in the design file to obtain a contour matching result comprises:

[0018] The pixel region corresponding to the mounting position in the PCB elevation map is subjected to a binaryzation process to extract edge information of the actual contour.

[0019] The similarity of the actual contour and the standard 2D contour is calculated according to the edge information.

[0020] The contour matching result of the 2D contour is obtained by comparing the similarity with a preset contour similarity. The contour matching result of the 2D contour is obtained by comparing the similarity with a preset contour similarity.

[0021] In a possible implementation, the edge information comprises an edge contour line and a plurality of key edge points. The similarity of the actual contour and the standard 2D contour is calculated according to the edge information.

[0022] According to the ratio between the area surrounded by the actual contour and the area surrounded by the standard 2D contour, the area matching is performed;

[0023] In the case of area matching, a plurality of vector pairs are determined according to a plurality of key edge points;

[0024] According to the included angle value between each vector pair in the plurality of vector pairs, the contour similarity is determined.

[0025] In a possible implementation, the plurality of vector pairs are determined according to a plurality of key edge points, including:

[0026] The key edge points of the actual contour are subjected to affine transformation correction to compensate for the rotation and translation deviation of the PCB in the imaging process, and the affine transformation correction is based on the centroid alignment and the principal axis alignment of the actual contour and the standard 2D contour;

[0027] N key edge points are extracted from the edge information of the actual contour after the affine transformation correction, and the N key edge points are sorted in a clockwise or counterclockwise direction;

[0028] N standard key points are extracted from the standard 2D contour, and the N standard key points are sorted according to the same sequence rule as the actual contour;

[0029] The sorted actual contour key points are sequentially connected to adjacent two points to construct N contour vectors;

[0030] The sorted standard contour key points are sequentially connected to adjacent two points to construct N standard vectors, so that the N contour vectors and the N standard vectors form N vector pairs.

[0031] In a possible implementation, the contour similarity is determined according to the included angle value between each vector pair in the plurality of vector pairs, including:

[0032] Based on the vector cosine value, the included angle between each vector pair is obtained, and N included angle values are obtained;

[0033] The length of each standard vector is calculated, and the weight corresponding to each vector pair is determined according to the length of the standard vector, wherein the longer the length of the standard vector, the greater the corresponding weight;

[0034] Based on the included angle value of each vector pair and the corresponding weight, a weighted average included angle value is calculated;

[0035] The similarity value is determined based on the weighted average included angle value.

[0036] In a possible implementation, the synchronously detecting according to the profile matching result and the height matching result to obtain a first detection result comprises:

[0037] determining that the profile matches and the height matches, judging that the corresponding mounting position is normal installation of the component;

[0038] determining that the profile does not match and the height does not match, judging that the corresponding mounting position is missing installation of the component;

[0039] determining that the profile does not match and the height matches, judging that the corresponding mounting position is misinstallation of the component;

[0040] determining that the profile matches and the height does not match, judging that the corresponding mounting position is virtual installation of the component.

[0041] In a possible implementation, the method further comprises:

[0042] in a case where the first detection result indicates that all the mounting positions are normal installation of the component, dividing the target PCB board into M*N grid regions in a two-dimensional plane based on the PCB elevation map;

[0043] calculating an average value of height values of all the pixel points in each grid region as a representative height of the grid region;

[0044] fitting a reference plane according to the representative heights of all the grid regions and two-dimensional plane coordinates of the grid regions;

[0045] calculating distances from the representative heights of each grid region to the reference plane to obtain local warping values of the regions;

[0046] calculating an overall warping degree evaluation value of the target PCB board based on the local warping values of all the regions;

[0047] comparing the overall warping degree evaluation value with a preset warping degree threshold to generate a second detection result, the second detection result being used to judge whether the physical structure of the target PCB board is qualified.

[0048] In a second aspect, the embodiments of the present application further provide an electronic device, comprising a memory and a processor, the memory being used to store program code, and the processor being used to call the program code to execute the method in the first aspect.

[0049] In a third aspect, the embodiments of the present application further provide a storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method in the first aspect.

[0050] Different from the prior art, the PCB multi-parameter synchronous detection method provided by the embodiment of the application generates a PCB elevation map by fusing 2D images and 3D point cloud data, combines the standard height and contour information in the design file, and synchronously compares and analyzes the height and contour of the component mounting position, thereby realizing integrated detection. The method generates an elevation map by introducing 3D point cloud data, thereby making up for the lack of depth information in 2D detection. Meanwhile, the spatio-temporal alignment and fusion of 2D and 3D data not only retains the high-resolution details of the two-dimensional image, but also increases the height dimension information, thereby significantly improving the coverage dimension and accuracy of detection. Meanwhile, the method realizes the matching and judgment of the height and contour at the same time in one detection process through multi-parameter synchronous analysis, thereby not only improving the detection efficiency, but also effectively reducing the risk of missed detection caused by multiple separate detections, thereby providing reliable technical support for improving the PCB assembly yield and product reliability. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to the structures shown in the drawings without creative labor.

[0052] Figure 1 The flowchart of the PCB multi-parameter synchronous detection method in some embodiments of the present application is shown in the figure.

[0053] Figure 2 The flowchart of step S200 of the PCB multi-parameter synchronous detection method in some embodiments of the present application is shown in the figure.

[0054] Figure 3 The flowchart of step S600 of the PCB multi-parameter synchronous detection method in some embodiments of the present application is shown in the figure.

[0055] Figure 4 The hardware structure diagram of the electronic device in some embodiments of the present application is shown in the figure.

[0056] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0058] It should be noted that all the direction indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, motion condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the direction indications will also change accordingly.

[0059] In addition, the descriptions involving "first", "second", etc. in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" can be explicitly or implicitly included at least one of the features. In addition, "and / or" throughout the text includes three schemes, taking A and / or B as an example, including A technical solution, B technical solution, and A and B technical solution that meet at the same time. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of the ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0060] As the core carrier of electronic equipment, the mounting quality and physical structure stability of the components on the printed circuit board (PCB) directly determine the reliability and service life of the terminal product. With the rapid development of the electronic information industry towards high density, miniaturization and integration, the mounting density of components on the PCB is continuously increasing, and the pin pitch is continuously decreasing, which puts forward strict requirements on the precision, efficiency and multi-dimensional coverage of the mounting quality detection. At present, in the production and manufacturing process of the PCB, the quality detection of the mounted components has become a key link to ensure the yield of the product.

[0061] In the related art, the traditional 2D visual detection method is mainly relied on. This method analyzes the collected planar image information, and due to the small number of parameters, it is difficult to comprehensively evaluate the production quality of the PCB, and it is difficult to meet the high requirements of the current industry on the defect detection rate.

[0062] To solve the above technical problems, the present application provides a PCB multi-parameter synchronous detection method, which can be applied to an electronic device. The electronic device can be a detection system, which can include a detection placement platform, a 2D detector, a 3D detector, etc. as shown in Figures 1-3 The following will take the electronic device executing the PCB multi-parameter synchronous detection method as an example for description. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here. Please refer to the accompanying Figure 1 The method includes the following steps S100-S600:

[0063] Step S100, respectively acquiring two-dimensional detection data and three-dimensional detection data of the target PCB board, obtaining 2D image data and 3D point cloud data;

[0064] Specifically, the 2D image data can be acquired by an RGB camera, and the 3D point cloud data can be acquired by an RGB-D camera or a laser radar.

[0065] For example, the PCB board to be detected is placed on a detection table, and an RGB camera and an RGB-D camera or a laser radar are started to synchronously collect 2D image data and 3D point cloud data.

[0066] Step S200, spatio-temporal alignment and fusion of the 2D image data and the 3D point cloud data according to a conversion matrix, to obtain a PCB elevation map of the target PCB board, wherein a pixel value of a pixel point in the PCB elevation map represents a height value of the corresponding pixel point;

[0067] Specifically, the conversion matrix refers to a mathematical transformation matrix for spatial mapping of a three-dimensional coordinate system (such as a depth coordinate system of an RGB-D camera) in which the 3D point cloud data is located and a two-dimensional image coordinate system (such as a pixel coordinate system of an RGB camera) in which the 2D image data is located, which contains key parameters such as a rotation matrix and a translation vector, and can realize accurate alignment of the two types of data in space. The conversion matrix can be determined by the installation positions of the RGB camera and the RGB-D camera and the corresponding intrinsic parameters: the intrinsic parameters include the focal length, the principal point coordinates and other parameters reflecting the imaging characteristics of the device itself, and the installation positions determine the relative spatial attitude (such as the translation distance and the rotation angle) between the two devices. After the installation positions of the RGB camera and the RGB-D camera and the corresponding intrinsic parameters are determined, the conversion matrix can be determined according to the coordinate system transformation principle (such as calculating the rotation and translation relationship between the coordinate systems of the two devices by a camera calibration algorithm), to ensure that the spatial points in the 3D point cloud data can be accurately mapped to the corresponding pixel positions of the 2D image.

[0068] Therefore, after the collection of the 2D image data and the 3D point cloud data of the target PCB board is completed in step S100, the pre-determined conversion matrix can be used to perform spatio-temporal alignment and fusion processing on the two types of data. The device coordinate system difference and the time sequence deviation in collection are eliminated by the conversion matrix, the height information of the 3D point cloud data is accurately associated with the pixel positions of the 2D image data, and finally the PCB elevation map of the target PCB board is generated. In the elevation map, the pixel value of each pixel point is uniquely mapped with the height value of the corresponding physical position, realizing the unified carrying of “planar position-height information” in a single data source.

[0069] In an embodiment, please refer to the accompanying drawings Figure 2, the step S200: according to the conversion matrix, the 2D image data and 3D point cloud data are spatio-temporally aligned and fused, and the PCB elevation map of the target PCB board is obtained, comprising:

[0070] S210, using a conversion matrix, each 3D point in the 3D point cloud data is mapped to the corresponding image coordinate system of the 2D image data, and the 2D pixel coordinates corresponding to each 3D point are obtained;

[0071] S220, taking the resolution of the 2D image data as a reference, a two-dimensional grid corresponding to each 2D image pixel is constructed, and each 3D point is assigned to the corresponding grid cell according to its 2D pixel coordinates;

[0072] S230, for each grid cell, if multiple 3D points fall into it, the average height value of all 3D points is calculated as the final height value of the grid cell; if the grid cell has no 3D points falling into it, a preset filling rule is used to fill in the missing data;

[0073] S240, according to a preset height-pixel value mapping rule, the final height value of each grid cell is converted into the corresponding pixel value to generate the PCB elevation map of the target PCB board.

[0074] Specifically, first, the three-dimensional space coordinates (such as X, Y, Z) of each 3D point in the 3D point cloud data are mapped to the image coordinate system corresponding to the 2D image data by using a pre-calibrated conversion matrix, and the pixel coordinates (such as u, v) of each 3D point in the 2D image are calculated. The conversion matrix eliminates the coordinate system differences (such as the installation position deviation of the RGB camera and the RGB-D camera) of the two types of data, ensures the accurate spatial correspondence between the 3D points and the 2D pixel points, and lays a foundation for subsequent data fusion. Then, a two-dimensional grid corresponding to each pixel of the 2D image is constructed based on the resolution (such as pixel size, row and column number) of the 2D image data. Each grid cell corresponds to a pixel position in the 2D image, and all 3D points are then assigned to the corresponding grid cells one by one according to the 2D pixel coordinates obtained in S210, so that the height information of the 3D points is preliminarily associated with the 2D plane position. Then, height value processing is performed on each grid cell to ensure data validity: if multiple 3D points fall into the same grid cell (due to point cloud data density or detection angle), the average value of the height values (Z coordinates) of these 3D points is calculated as the final height value of the grid cell, avoiding the influence of single-point noise; if no 3D point falls into the grid cell (such as incomplete coverage of point cloud data), a preset filling rule (such as neighborhood grid height interpolation, default reference height assignment, etc.) is used to supplement the missing data, ensuring the integrity of the height information of the entire height map. Finally, according to a preset height-pixel value mapping rule (such as linear conversion of height value to gray value, or color coding mapping to RGB value), the final height value of each grid cell is converted to the corresponding pixel value. For example, a higher height can be set to a brighter pixel value, and a lower height can be set to a darker pixel value, so that the PCB height map not only maintains the plane position accuracy of the 2D image, but also intuitively reflects the height information of the corresponding position through the pixel value, and finally generates a target PCB board PCB height map that integrates two-dimensional and three-dimensional features.

[0075] In this way, the embodiments of the present application realize coordinate mapping, grid assignment, height processing and pixel conversion in steps, complete the spatio-temporal alignment and fusion of 2D image data and 3D point cloud data, and generate a PCB height map that can be used as a unified data carrier to provide plane contour and height information simultaneously, laying a data foundation for subsequent efficient detection, thereby improving the accuracy and efficiency of PCB board multi-parameter detection.

[0076] Step S300, obtaining a design file of a target PCB board, determining position coordinates of all preset component mounting positions on the target PCB board, and corresponding standard component height threshold range and standard 2D contour;

[0077] After the PCB elevation map of the target PCB board is generated in step S200, the design file (such as CAD design drawing or BOM list) of the PCB board can be further obtained, and key information is extracted therefrom: first, the accurate position coordinates (such as pixel coordinate range in the image coordinate system) of all preset component mounting positions on the board are determined; second, the standard component parameters corresponding to each mounting position are obtained, including the height threshold range (i.e. the upper and lower limits of the height in the qualified installation state) and the standard 2D contour (i.e. the planar shape and size defined in the design). These information will be used as the reference data for subsequent comparison of actual contour and standard contour, and verification of actual height and standard height range, and provide reliable basis for accurate determination of the component installation state.

[0078] In step S400, the pixel area corresponding to each preset component mounting position in the PCB elevation map is traversed according to the position coordinates, the pixel value of each mounting position pixel area is extracted, and the actual height value of each mounting position is obtained by conversion according to the mapping relationship between the pixel value and the height value.

[0079] After the reference information (position coordinates of the preset component mounting position, and height threshold range and standard 2D contour of the corresponding standard component) of the target PCB board is obtained in step S300, data extraction and conversion can be performed on the generated PCB elevation map based on these information. First, according to the position coordinates of each preset component mounting position, the pixel area corresponding to each mounting position is accurately positioned and traversed in the PCB elevation map, ensuring that no area to be detected is missed. Then, the pixel values of all pixels in each positioned pixel area are extracted, and these pixel values are the original data reflecting the physical height of the area. Finally, the mapping relationship between the pixel value and the height value is established in advance, and the extracted pixel values are converted into corresponding actual height values one by one, which will be used for subsequent comparison with the standard height threshold range and is the core data for determining whether the component height is qualified.

[0080] It should be noted that the average pixel value of each mounting position pixel area can be used to determine the actual height value of the mounting position, and the actual height value of the corresponding mounting position can also be determined by other means.

[0081] In step S500, the actual height value of each mounting position is compared with the height threshold range of the corresponding standard component to obtain a height matching result.

[0082] After obtaining the actual height value of each mounting position in step S400, the actual data can be compared with the reference data. Specifically, the actual height value corresponding to each preset component mounting position is compared with the standard component height threshold range (i.e. the height upper and lower limits in the qualified installation state) determined in step S300 respectively: if the actual height value is within the threshold range, the height matching result is qualified (matched); if the actual height value exceeds the threshold range (too high or too low), the height matching result is unqualified (not matched). The height matching result is a direct basis for determining whether the component installation meets the height requirement, and provides a key reference for subsequent comprehensive judgment of the component installation state.

[0083] Step S600, extracting the actual contour corresponding to each mounting position in the PCB elevation map, and matching the standard 2D contour of the corresponding mounting position in the design file to obtain a contour matching result, and synchronously detecting according to the contour matching result and the height matching result to obtain a first detection result.

[0084] After obtaining the height matching result in step S500, the contour information of the component can be further detected and comprehensively judged. First, the actual contour corresponding to each preset component mounting position is extracted from the PCB elevation map, which reflects the planar shape and size characteristics of the component after installation. Then, the extracted actual contour is matched and compared with the standard 2D contour (i.e. the planar shape and size defined in the design) of the corresponding mounting position obtained from the design file in step S300, and the contour matching result (such as matching qualified or unqualified) is obtained according to the shape similarity, size deviation and other indicators of the contour. Finally, the height matching result and the above contour matching result are synchronously detected in combination: when both are qualified, the first detection result is that the component installation is qualified; if any one of them is unqualified, the first detection result is that the component installation is unqualified. The first detection result integrates the detection information of height and contour, and provides a comprehensive basis for accurately determining the component installation state.

[0085] In an embodiment, please refer to the accompanying drawings Figure 3 , the step S600: extracting the actual contour corresponding to each mounting position in the PCB elevation map, and matching the standard 2D contour of the corresponding mounting position in the design file to obtain a contour matching result, comprising:

[0086] S610, performing binaryzation processing on the pixel area corresponding to the mounting position in the PCB elevation map, and extracting the edge information of the actual contour;

[0087] S620, calculating the similarity of the actual contour and the standard 2D contour according to the edge information;

[0088] S630, comparing the similarity with a preset contour similarity to obtain a 2D contour matching result of contour matching or contour non-matching.

[0089] Specifically, first, the pixel region corresponding to each mounting position in the PCB elevation map is binarized, and by setting a suitable pixel value threshold, the pixels in the region are divided into effective pixels corresponding to the target component (such as pixels higher than the reference surface of the PCB) and background pixels (such as the reference surface of the PCB), thereby highlighting the edge features of the actual contour and extracting edge information reflecting the planar shape of the component, laying the foundation for subsequent contour comparison. Subsequently, according to the extracted edge information, the similarity between the actual contour and the corresponding mounting position standard 2D contour obtained in step S300 is calculated, which can be quantitatively calculated through indicators such as the degree of coincidence of the edge point set, the deviation degree of the shape geometric parameters (such as the perimeter, area, inflection point position, etc.), to obtain a similarity value representing the shape matching degree of the two. Finally, the similarity is compared with a preset contour similarity threshold: if the similarity is greater than or equal to the threshold, it is determined as contour matching; if the similarity is less than the threshold, it is determined as contour non-matching, thereby obtaining the 2D contour matching result.

[0090] In this way, through the processing of the mounting position pixel region in the PCB elevation map, the similarity calculation and the threshold comparison, the matching result of the actual contour and the standard 2D contour can be accurately obtained, providing a reliable contour detection basis for subsequent comprehensive judgment of the component installation state.

[0091] In an embodiment, the edge information includes an edge contour line and a plurality of key edge point positions, and the step of calculating the similarity between the actual contour and the standard 2D contour according to the edge information includes: calculating the ratio between the area enclosed by the actual contour and the area enclosed by the standard 2D contour according to the edge contour line, and performing area matching; in the case of area matching, determining a plurality of vector pairs according to the plurality of key edge point positions; and determining the contour similarity according to the included angle value between each vector pair in the plurality of vector pairs.

[0092] Specifically, the ratio between the area enclosed by the actual contour and the area enclosed by the standard 2D contour can be calculated according to the edge contour line first, and the area matching degree of the two can be judged through the ratio, if the area ratio is within a preset reasonable range (such as close to 1), it is considered that the area is matched, which provides a premise for subsequent more detailed contour comparison; if the area deviation is too large, it can be directly determined that the contour is not matched.

[0093] In the case of area matching, the similarity can be further calculated by key edge points. First, the key edge points of the actual contour are corrected by affine transformation, which is based on the alignment of the centroids of the actual contour and the standard 2D contour (adjusting the coordinates of the centroids of the two to coincide) and the alignment of the principal axes (keeping the principal directions of the two consistent), so as to compensate for the position deviation of the PCB board in the imaging process, such as rotation, translation, etc., and ensure that the actual contour and the standard contour are in a comparable state in the same coordinate system. Then, N key edge points (such as inflection points, end points, etc. representative feature points) are extracted from the actual contour edge information after affine transformation correction, and are sequentially arranged in a clockwise or counterclockwise direction; at the same time, N standard key points are extracted from the standard 2D contour, and the same sequential rules (clockwise or counterclockwise) as the actual contour are used for sorting, to ensure the correspondence of the points. Subsequently, the N contour vectors are constructed by sequentially connecting adjacent points of the sorted actual contour key points; the same adjacent point connection operation is performed on the sorted standard contour key points to construct N standard vectors, so that the N contour vectors of the actual contour and the N standard vectors of the standard contour correspond one-to-one, forming N vector pairs. Finally, the contour similarity is determined according to the angle values between each of the N vector pairs, for example, the average deviation value of all vector pairs is calculated, and the smaller the deviation, the higher the similarity.

[0094] In this way, the embodiments of the present application can accurately quantify the similarity between the actual contour and the standard 2D contour by area matching preliminary screening and combining the corrected key points to construct vector pairs to calculate the angle deviation, providing a scientific basis for the judgment of the contour matching result, and improving the accuracy of the PCB board component installation detection.

[0095] To further improve the accuracy of contour matching, in an embodiment, the step of determining the contour similarity according to the angle values between each of the plurality of vector pairs comprises: obtaining the angle between each of the vector pairs based on the vector cosine value, obtaining N angle values; calculating the length of each standard vector, and determining the weight corresponding to each vector pair according to the length of the standard vector, wherein the longer the length of the standard vector, the greater the corresponding weight; calculating the weighted average angle value based on the angle value of each vector pair and the corresponding weight; determining the similarity value based on the weighted average angle value.

[0096] Specifically, first, the angle between each of the vector pairs is calculated based on the vector cosine value, through the vector dot product formula (cosθ = a·b / (|a| |b|)) ) the included angle θ is deduced, and the N included angle values corresponding to the N vector pairs are calculated one by one, which directly reflect the direction deviation degree of each vector pair. Then, the length of each standard vector is calculated. The lengths of the N standard vectors constructed in the standard 2D contour are calculated by coordinate difference; the corresponding weight of each vector pair is determined according to the length of the standard vector, which follows the rule that "the longer the standard vector length, the greater the corresponding weight". Because the longer vector usually represents a more critical structural feature in the contour and has a greater impact on the overall shape of the contour, giving a higher weight can improve the contribution of the key feature to the matching result. Then, based on the included angle value of each vector pair and its corresponding weight, the weighted average included angle value is calculated. The sum of the product of each included angle value and the corresponding weight is divided by the sum of all weights to obtain the weighted average included angle value that can comprehensively reflect the direction deviation of all vector pairs. This value is more consistent with the actual feature importance of the contour than the ordinary average included angle value. Finally, the similarity value is determined based on the weighted average included angle value, and the mapping relationship between the weighted average included angle value and the similarity is set (such as the smaller the included angle value, the higher the similarity; when the included angle value is 0, the similarity is 100%). The quantitative included angle deviation is converted into an intuitive similarity result through the mapping relationship, which provides an accurate basis for contour matching determination.

[0097] Therefore, the embodiment of the present application can highlight the influence of key contour features on the matching result by introducing the vector length weight to calculate the weighted average included angle value, effectively reduce the interference of secondary feature deviation (such as deviation of short side) on the determination, and further improve the accuracy of contour matching, thereby providing stronger support for the reliability of PCB board component installation detection.

[0098] In an embodiment, the step of obtaining a first detection result by simultaneously detecting the contour matching result and the height matching result comprises:

[0099] determining that the contour matches and the height matches, and judging that the corresponding installation position is a normally installed component;

[0100] determining that the contour does not match and the height does not match, and judging that the corresponding installation position is a missed component;

[0101] determining that the contour does not match and the height matches, and judging that the corresponding installation position is a misinstalled component;

[0102] determining that the contour matches and the height does not match, and judging that the corresponding installation position is a virtual component.

[0103] The embodiments of the present application determine one by one for different combinations of the contour matching result and the height matching result. Specifically, when the contour matching and the height matching are determined, it is indicated that the actual planar shape and size of the component are consistent with the standard 2D contour, and the actual height is within the standard height threshold range, thereby judging that the component of the corresponding mounting position is normally installed, i.e., meeting the design installation requirements. When the contour mismatching and the height mismatching are determined, it is indicated that neither the actual contour nor the height of the component of the corresponding mounting position meets the standard 2D contour and the height threshold range of the standard component (usually, the height is close to the reference surface of the PCB), thereby judging that the component of the corresponding mounting position is missing, i.e., the mounting position is not installed with the component. When the contour mismatching and the height matching are determined, it is indicated that the component is installed in the corresponding mounting position (the height meets the standard range), but the actual contour is greatly different from the standard 2D contour, which indicates that the installed component is not the type or specification specified in the design, thereby judging that the component of the corresponding mounting position is misinstalled. When the contour matching and the height mismatching are determined, it is indicated that the planar shape and size of the component meet the standard, but the actual height is out of the standard threshold range (too high or too low), which is usually caused by the fact that the component is not firmly attached to the PCB or is warped, offset, etc., thereby judging that the component of the corresponding mounting position is virtually installed.

[0104] In this way, by simultaneously detecting and combining the contour matching result and the height matching result, the embodiments of the present application can accurately distinguish different installation states of the component, such as normal installation, missing, misinstallation, and virtual installation, thereby providing a clear and reliable first detection result for the comprehensive detection of the installation quality of the component of the PCB, and improving the accuracy and refinement of the detection.

[0105] After the first detection is performed on each mounting position of the target PCB and the first detection result is obtained, the PCB multi-parameter synchronous detection method of the present application further includes:

[0106] In the case where the first detection result indicates that all the mounting positions are normally installed with the component, the target PCB is divided into MxN grid regions in a two-dimensional plane based on the PCB elevation map;

[0107] The average value of the height values of all the pixel points in each grid region is calculated as the representative height of the grid region;

[0108] A reference plane is fitted according to the representative heights of all the grid regions and the two-dimensional plane coordinates thereof;

[0109] The distance from the representative height of each grid region to the reference plane is calculated to obtain the local warping value of each region;

[0110] The overall warping degree evaluation value of the target PCB is calculated based on the local warping values of all the regions;

[0111] The overall warping degree evaluation value is compared with a preset warping degree threshold value to generate a second detection result, which is used to determine whether the physical structure of the target PCB is qualified.

[0112] The embodiments of the present application first detect the warping degree of the physical structure of the target PCB when all mounting positions are normally installed with components, avoiding interference of component installation abnormalities on the evaluation of the warping degree of the board. Based on the generated PCB elevation map, the two-dimensional plane is divided into MxN grid regions according to a preset size (such as a fixed pixel side length set according to the size of the PCB), so that each grid region can uniformly cover the surface of the PCB, ensuring the comprehensiveness of subsequent detection.

[0113] Then, the average value of the height values of all pixel points in each grid region is calculated, and the average value is taken as the representative height of the grid region. By taking the average of the pixel height in the region, the noise interference that may exist in a single pixel can be eliminated, and the actual height level of the local region can be more accurately reflected, providing reliable local height data for subsequent reference plane fitting.

[0114] Next, according to the representative heights of all grid regions and their corresponding two-dimensional plane coordinates (i.e., the position coordinates of each grid on the two-dimensional plane of the PCB), a reference plane is fitted using a plane fitting algorithm (such as the least squares method). The reference plane can be regarded as the theoretical plane of the PCB in the ideal non-warping state, serving as a reference for measuring the warping degree of each region of the board.

[0115] Then, the distance from the representative height of each grid region to the reference plane is calculated, and the absolute value of the distance is the local warping value of each region. The greater the local warping value, the more serious the deviation of the region from the ideal plane, which intuitively reflects the local warping of the PCB.

[0116] After that, based on the local warping values of all regions, the overall warping degree evaluation value of the target PCB is calculated by a pre-designed calculation rule (such as taking the maximum value, average value or variance of all local warping values), and the scattered local warping data is converted into a quantitative index that can represent the overall warping state of the board.

[0117] Finally, the overall warping degree evaluation value is compared with a preset warping degree threshold value (a qualified and unqualified critical value set according to the application scenario, material requirements, etc. of the PCB): if the evaluation value is less than or equal to the threshold value, the second detection result is that the physical structure of the PCB is qualified; if the evaluation value is greater than the threshold value, the second detection result is that the physical structure of the PCB is unqualified. The second detection result can determine whether the physical structure of the PCB meets the use requirements, filling the gap of only detecting the installation state of components.

[0118] Thus, this embodiment of the application, while ensuring the normal installation of components, further detects the overall warpage of the PCB board, realizing dual detection of component installation status and board physical structure, comprehensively ensuring the quality of the target PCB board, and improving the completeness and practicality of the detection scheme.

[0119] Based on this, the PCB board multi-parameter synchronous inspection method of this application generates a PCB elevation map by fusing 2D images and 3D point cloud data. Combined with the standard height and contour information in the design file, it performs synchronous comparison and analysis of the height and contour of component mounting positions, achieving integrated inspection. This method compensates for the lack of depth information in 2D inspection by introducing an elevation map generated from 3D point cloud data. Simultaneously, the spatiotemporal alignment and fusion of 2D and 3D data preserves the high-resolution details of the 2D image while adding height dimension information, significantly improving the coverage and accuracy of the inspection. Furthermore, through multi-parameter synchronous analysis, this method achieves simultaneous height and contour matching in a single inspection process, improving inspection efficiency and effectively reducing the risk of missed detections caused by multiple separate inspections. In addition, while ensuring proper component mounting, this application further inspects the overall warpage of the PCB board, achieving dual inspection of component mounting status and board physical structure, comprehensively ensuring the quality of the target PCB board and improving the completeness and practicality of the inspection solution.

[0120] like Figure 4 As shown, Figure 4 The diagram below shows the hardware structure of an electronic device in some embodiments of this application. The electronic device provided in the embodiments of this application includes a memory 1000 and a processor 2000. The memory 1000 is used to store computer-readable instructions, and the processor 2000 is used to call the computer-readable instructions to execute the PCB board multi-parameter synchronous detection method as described above.

[0121] The processor 2000 is configured to provide computing and control capabilities to control the electronic device to perform corresponding tasks, for example, to control the electronic device to perform the PCB multi-parameter synchronous detection method in any of the above method embodiments. The method includes: obtaining two-dimensional detection data and three-dimensional detection data of a target PCB respectively to obtain 2D image data and 3D point cloud data; performing space-time alignment fusion on the 2D image data and the 3D point cloud data according to a conversion matrix to obtain a PCB elevation map of the target PCB, wherein a pixel value of a pixel point in the PCB elevation map represents a height value of the corresponding pixel point; obtaining a design file of the target PCB, determining position coordinates of all preset component mounting positions on the target PCB, and height threshold ranges and standard 2D contours of corresponding standard components; traversing all pixel regions corresponding to the preset component mounting positions in the PCB elevation map according to the position coordinates, extracting pixel values of each mounting position pixel region, and converting the pixel values to obtain actual height values of each mounting position according to a mapping relationship between the pixel values and the height values; comparing the actual height values of each mounting position with the height threshold ranges of the corresponding standard components to obtain height matching results; extracting actual contours corresponding to each mounting position in the PCB elevation map, and matching the actual contours with the standard 2D contours of the corresponding mounting positions in the design file to obtain contour matching results, and performing synchronous detection according to the contour matching results and the height matching results to obtain a first detection result.

[0122] The processor 2000 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0123] The memory 1000, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the PCB multi-parameter synchronous detection method in the embodiments of the present application. The processor 2000 can implement the PCB multi-parameter synchronous detection method in any of the method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 1000.

[0124] Specifically, the memory 1000 can include volatile memory (VM), such as random access memory (RAM); the memory 1000 can also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD) or other non-transitory solid-state storage devices; the memory 1000 can also include a combination of the above types of memory.

[0125] In summary, the electronic device of the present application adopts the technical solutions of any one of the above PCB multi-parameter synchronous detection method embodiments, and therefore has at least the beneficial effects brought by the technical solutions of the above embodiments, which will not be repeated here.

[0126] The embodiments of the present application also provide a computer readable storage medium, such as a memory including program code, which can be executed by a processor to complete the PCB multi-parameter synchronous detection method in the above embodiments. For example, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CDROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0127] The embodiments of the present application also provide a computer program product, which includes one or more program codes stored in a computer readable storage medium. The processor of the early warning system reads the program code from the computer readable storage medium, and the processor executes the program code to complete the steps of the PCB multi-parameter synchronous detection method provided in the above embodiments.

[0128] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program code related hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0129] It should be noted that the above-described device embodiments are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0130] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0131] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made under the inventive concept of the present application, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A method for simultaneous detection of multiple parameters on a PCB board, characterized in that, include: Two-dimensional and three-dimensional detection data of the target PCB board are acquired respectively to obtain 2D image data and 3D point cloud data; The 2D image data and 3D point cloud data are spatiotemporally aligned and fused according to the transformation matrix to obtain the PCB elevation map of the target PCB board, wherein the pixel value of the pixel in the PCB elevation map represents the height value of the corresponding pixel. Obtain the design file of the target PCB board, and determine the position coordinates of all preset component mounting positions on the target PCB board, as well as the height threshold range and standard 2D contour of the corresponding standard components. Based on the location coordinates, traverse the pixel regions corresponding to all preset component mounting positions in the PCB elevation map, extract the pixel values ​​of each mounting position pixel region, and convert the actual height value of each mounting position according to the mapping relationship between the pixel values ​​and the height values. The actual height value of each mounting position is compared with the height threshold range of the corresponding standard component to obtain the height matching result; Extract the actual contour corresponding to each mounting position in the PCB elevation map, and match it with the standard 2D contour of the corresponding mounting position in the design file to obtain the contour matching result. Based on the contour matching result and the height matching result, perform synchronous detection to obtain the first detection result.

2. The PCB board multi-parameter synchronous detection method as described in claim 1, characterized in that, The step of performing spatiotemporal alignment and fusion of the 2D image data and 3D point cloud data according to the transformation matrix to obtain the PCB elevation map of the target PCB board includes: Using a transformation matrix, each 3D point in the 3D point cloud data is mapped to the image coordinate system corresponding to the 2D image data, thus obtaining the 2D pixel coordinates corresponding to each 3D point; Based on the resolution of 2D image data, a two-dimensional grid is constructed that corresponds one-to-one with the pixels of the 2D image, and each 3D point is assigned to the corresponding grid cell according to its 2D pixel coordinates. For each grid cell, if multiple 3D points fall into it, the average height value of all 3D points is calculated as the final height value of the grid cell; if no 3D points fall into the grid cell, the missing data is filled using a preset filling rule. According to the preset height-pixel value mapping rules, the final height value of each grid cell is converted into the corresponding pixel value to generate the PCB elevation map of the target PCB board.

3. The PCB board multi-parameter synchronous detection method as described in claim 1, characterized in that, The step of extracting the actual contour corresponding to each mounting position in the PCB elevation map and matching it with the standard 2D contour of the corresponding mounting position in the design file to obtain the contour matching result includes: The pixel region corresponding to the mounting position in the PCB elevation map is binarized to extract the edge information of the actual contour. The similarity between the actual contour and the standard 2D contour is calculated based on the edge information; Based on the comparison between the aforementioned similarity and a preset contour similarity, a 2D contour matching result is obtained, indicating whether the contours match or do not match.

4. The PCB board multi-parameter synchronous detection method as described in claim 3, characterized in that, The edge information includes edge contour lines and multiple key edge points. The step of calculating the similarity between the actual contour and the standard 2D contour based on the edge information includes: The area is matched by calculating the ratio between the area enclosed by the actual contour and the area enclosed by the standard 2D contour based on the edge contour lines. In the case of area matching, multiple vector pairs are determined based on multiple key edge points; Contour similarity is determined based on the angle between each vector pair in a plurality of vector pairs.

5. The PCB board multi-parameter synchronous detection method as described in claim 4, characterized in that, The determination of multiple vector pairs based on multiple key edge points includes: Affine transformation correction is performed on the key edge points of the actual contour to compensate for the rotation and translation deviation of the PCB board during the imaging process. The affine transformation correction is based on the centroid alignment and principal axis alignment of the actual contour and the standard 2D contour. Extract N key edge points from the edge information of the actual contour after affine transformation correction, and sort them in a clockwise or counterclockwise direction; Extract N corresponding standard key points from the standard 2D contour and sort them using the same order rules as the actual contour; For the sorted actual contour key points, connect adjacent two points in sequence to construct N contour vectors; For the sorted standard contour key points, perform the same operation to construct N standard vectors, so that the N contour vectors and the N standard vectors form N vector pairs.

6. The PCB board multi-parameter synchronous detection method as described in claim 5, characterized in that, The step of determining contour similarity based on the angle between each vector pair in multiple vector pairs includes: Based on the cosine value of the vector, the angle between each vector pair is obtained, resulting in N angle values; Calculate the length of each standard vector, and determine the weight of each vector pair based on the length of the standard vector. The longer the standard vector, the greater the corresponding weight. Calculate the weighted average angle value based on the included angle value of each vector pair and its corresponding weight; The similarity value is determined based on the weighted average angle value.

7. The PCB board multi-parameter synchronous detection method as described in claim 1, characterized in that, The step of simultaneously detecting based on the contour matching result and the height matching result to obtain the first detection result includes: If the contour matches and the height matches, it is determined that the corresponding mounting position is a normal installation of the component; If the outline and height do not match, it is determined that the corresponding mounting position is a missing component. If the contours do not match but the heights do, it is determined that the corresponding mounting position is due to incorrect component mounting. If the contour matches but the height does not, it is determined that the corresponding mounting position is a dummy component.

8. The PCB board multi-parameter synchronous detection method as described in claim 7, characterized in that, The method further includes: If the first test result indicates that all mounting positions are where the components are installed normally, the target PCB board is divided into M×N grid areas on a two-dimensional plane based on the PCB elevation map. Calculate the average height value of all pixels within each grid region, and use it as the representative height of that grid region; A reference plane is fitted based on the representative heights of all grid regions and their two-dimensional planar coordinates. Calculate the distance from the representative height of each grid region to the reference plane to obtain the local warping value of each region; Based on the local warpage values ​​of all regions, the overall warpage assessment value of the target PCB board is calculated. The overall warpage assessment value is compared with a preset warpage threshold to generate a second detection result, which is used to determine whether the physical structure of the target PCB board is qualified.

9. An electronic device, characterized in that, include: Memory and processor, wherein the memory is used to store program code; The processor is used to call the program code to perform the method as described in any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images