Multi-source sensor fusion pcba circuit board processing quality real-time detection method

Abstract

The application discloses a multi-source sensor fusion PCBA circuit board processing quality real-time detection method, relates to the technical field of PCBA circuit board processing quality detection, and comprises the following steps: collecting the image of a PCBA circuit board after processing through a multi-source sensor, and simultaneously performing image preprocessing on the image after processing; identifying the solder joint coordinates in the PCBA circuit board through a 3D laser profile image, simultaneously obtaining the pixel points belonging to the solder joints in a high-definition surface image according to the solder joint coordinates, and naming the pixel points as image solder joints; extracting the morphological features of each solder joint according to the position distribution of the image solder joints, simultaneously analyzing the color features of the solder joints according to the gray values of the image solder joints; and detecting whether the PCBA circuit board processing quality has problems in real time according to the morphological features and color features of the solder joints. The application is used for solving the problems that the existing PCBA circuit board processing quality detection technology still needs too much manual intervention and the defect recognition of the solder joints is not comprehensive enough, and thus the PCBA circuit board processing quality detection is prone to errors.

Description

Technical Field

[0001] This invention relates to the field of PCBA circuit board processing quality inspection technology, specifically a real-time inspection method for PCBA circuit board processing quality based on multi-source sensor fusion. Background Technology

[0002] PCBA circuit board processing quality inspection technology refers to the technology of inspecting the quality of solder joints on the surface of PCBA circuit boards during the PCBA circuit board processing process and evaluating whether there are defects in the solder joints.

[0003] Existing PCBA circuit board processing quality inspection technologies typically require excessive manual intervention and focus too much on the morphological characteristics of solder joints. However, solder joints also exhibit color characteristics on their surface, as normal solder joint surfaces are usually smooth. A smooth surface implies a certain regularity in color variation. If the solder joint surface is not smooth, it indicates a problem during soldering, which is essentially a processing defect. Some defects can also manifest in surface color. Relying solely on morphological characteristics to inspect the processing quality of PCBA circuit boards makes a comprehensive assessment difficult. For example, the invention patent with publication number CN117571743A discloses a "PCBA circuit board processing defect detection and analysis system based on image recognition." This solution only judges whether there are defects in solder joints based on their morphological characteristics, ignoring the smoothness of the solder joint surface, resulting in an incomplete defect identification. Existing PCBA circuit board processing quality inspection technologies still suffer from excessive manual intervention and incomplete defect identification, leading to errors in PCBA circuit board processing quality inspection. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art. It acquires processed images of PCBA circuit boards using multi-source sensors, performs image preprocessing on the processed images, identifies solder joint coordinates in the PCBA circuit board using height information at different coordinates in a 3D laser contour map, and then obtains the pixels belonging to the solder joints in a high-definition surface image based on the solder joint coordinates to obtain the image solder joints. Next, it extracts the morphological features of each solder joint based on the region formed by the image solder joints, and analyzes the color features of the solder joints based on the distribution and changes in their grayscale values. Finally, it detects the presence of PCBA circuit board processing quality problems in real time based on the morphological and color features of the solder joints. This invention addresses the problems of existing PCBA circuit board processing quality inspection technologies, which require excessive manual intervention and lack comprehensive identification of solder joint defects, leading to errors in PCBA circuit board processing quality inspection.

[0005] To achieve the above objectives, in a first aspect, this application provides a method for real-time detection of PCBA circuit board processing quality based on multi-source sensor fusion, comprising the following steps:

[0006] Images of PCBA circuit boards after processing are acquired using multi-source sensors, and image preprocessing is performed on the processed images.

[0007] The coordinates of solder joints in the PCBA circuit board are identified by the 3D laser contour map in the processed image. At the same time, the pixels belonging to the solder joints in the high-definition surface map of the processed image are obtained based on the solder joint coordinates and named as image solder joints.

[0008] The morphological features of each weld point are extracted based on the location distribution of the weld points in the image, and the color features of the weld points are analyzed based on the grayscale values ​​of the weld points in the image.

[0009] Real-time detection of PCBA circuit board processing quality issues based on the morphological and color characteristics of solder joints.

[0010] Furthermore, the step of acquiring processed images of the PCBA circuit board through multi-source sensors and simultaneously performing image preprocessing on the processed images also includes the following sub-steps:

[0011] Multi-source sensors are installed on the PCBA circuit board processing production line, including a 3D line laser profilometer and a high-definition industrial camera;

[0012] The processed PCBA circuit board was scanned using a 3D line laser profilometer to obtain the three-dimensional profile of the PCBA circuit board, which was named 3D laser profiling image.

[0013] A high-definition industrial camera is used to photograph the processed PCBA circuit board to obtain a high-definition surface image of the PCBA circuit board. At the same time, the high-definition surface image is processed into grayscale. The 3D laser contour image and the high-definition surface image are the processed images.

[0014] Furthermore, the step of identifying the solder joint coordinates in the PCBA circuit board through the 3D laser contour map in the processed image, and simultaneously obtaining the pixels belonging to the solder joints in the high-resolution surface image of the processed image based on the solder joint coordinates, further includes the following sub-steps:

[0015] Identify solder joint coordinates in PCBA circuit boards by using height information at different coordinates in 3D laser contour maps;

[0016] The image of the weld point is obtained by acquiring the pixels belonging to the weld point in the high-definition surface image based on the weld point coordinates.

[0017] The advantage lies in its ability to directly identify solder joints in high-definition surface images without requiring manual provision of soldering coordinates. It can also automatically adapt to different PCBA circuit board models, eliminating the need to re-identify solder joints based on the provided soldering coordinates each time the model is changed, thus improving the intelligence and effectiveness of PCBA circuit board processing quality inspection.

[0018] Furthermore, the step of identifying solder joint coordinates in the PCBA circuit board using height information at different coordinates in the 3D laser contour map also includes the following sub-steps:

[0019] Obtain the highest height in the 3D laser contour map, and take the median of the highest height as the weld point identification parameter;

[0020] The 3D laser contour map is sliced ​​and layered to obtain different height layers. The height layer with a height equal to the solder joint recognition parameter is obtained and named the solder joint recognition layer.

[0021] The solder joint identification layer contains different closed regions enclosed by different curves, which are named solder joint subordinate regions.

[0022] Obtain the coordinates of the point located on the slave area of ​​the solder joint, and name it the solder joint coordinates.

[0023] Furthermore, the step of obtaining the pixels belonging to the solder joints in the high-definition surface image based on the solder joint coordinates to obtain the image solder joint also includes the following sub-steps:

[0024] Place the top view of the 3D laser contour map and the high-definition surface map in the same reference coordinate system, find the pixel point corresponding to the coordinate of the weld point in the high-definition surface map, and name it the image weld point.

[0025] The grayscale value of the welding point in the image is obtained and named as the welding grayscale value. At the same time, the grayscale values ​​of all pixels in the high-definition surface image are clustered to obtain different grayscale clusters. The grayscale cluster in which the welding grayscale value is located is named as the welding grayscale cluster. All grayscale values ​​in the welding grayscale cluster are defined as welding grayscale values, and the pixels to which the welding grayscale values ​​belong are defined as image welding points.

[0026] Furthermore, the step of extracting the morphological features of each weld point based on its positional distribution in the image, and simultaneously analyzing its color features based on its grayscale values, further includes the following sub-steps:

[0027] The morphological features of each weld point are extracted from the region formed by the weld points in the image.

[0028] Analyze the color characteristics of the weld points based on the distribution and changes in the grayscale values ​​of the weld points in the image;

[0029] The advantage lies in the ability to simultaneously determine whether there are defects in solder joints through morphological and color characteristics. This is because solder joints usually have regular shapes, and the color of the solder joint surface usually shows a gradient change. By judging the processing quality of solder joints through both, the processing quality of PCBA circuit boards can be inspected more comprehensively, thus improving the accuracy and comprehensiveness of PCBA circuit board processing quality inspection.

[0030] Furthermore, the step of extracting the morphological features of each weld point from the region formed by the weld points in the image also includes the following sub-steps:

[0031] The region consisting of consecutive adjacent image welding points is named the welding point region, and is also labeled with the symbol WP. n,m This represents the image welding point located in the nth row and mth column of the high-resolution surface image, where n represents the row number and m represents the column number;

[0032] The outermost image welding point in the welding point area is named the welding contour point, and the morphological features of each welding point are extracted based on the distribution of the welding contour points.

[0033] Furthermore, the step of analyzing the color features of the weld joints based on the distribution and changes in the grayscale values ​​of the weld joints in the image also includes the following sub-steps:

[0034] Draw the circumcircle of the weld area and name it the weld area circle. Obtain the center of the weld area circle and name it the weld center.

[0035] With welding center and H i Draw a line segment with the geometric center as its endpoint, name it the grayscale direction line, and use the symbol L. i H represents i The corresponding grayscale direction line;

[0036] The color features of the weld points are extracted based on the gray values ​​of the weld points along the gray-scale trend line in the image.

[0037] Furthermore, the step of real-time detection of PCBA circuit board processing quality issues based on the morphological and color characteristics of solder joints also includes the following sub-steps:

[0038] PCBA circuit boards with no processing defects were obtained as quality inspection samples. The morphological and color characteristics of the quality inspection samples were analyzed and named as normal morphological characteristics and normal color characteristics, respectively.

[0039] Deep learning is performed on normal morphological features and normal color features to obtain morphological reference features and color reference features;

[0040] We use morphological and color reference features as references to analyze whether there are problems with the processing quality of PCBA circuit boards.

[0041] Furthermore, the step of analyzing whether there are problems with the PCBA circuit board processing quality using morphological reference features and color reference features as references also includes the following sub-steps:

[0042] The range of the morphological reference feature on the X-axis is named the morphological range, and the range of the color reference feature on the X-axis is named the color range.

[0043] If the morphological features of the PCBA circuit board are within the morphological reference features within the morphological range, and the color features are within the color reference features within the color range, then the PCBA circuit board is marked as having good processing quality; otherwise, the PCBA circuit board is marked as having defects in processing quality.

[0044] In a second aspect, this application provides an electronic device, the electronic device including a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the method in the first aspect.

[0045] Thirdly, this application provides a storage medium storing computer instructions thereon, characterized in that, when the computer instructions are executed by a processor, the steps of the method as described in the first aspect are performed. Attached Figure Description

[0046] The following describes the accompanying drawings used in this application.

[0047] Figure 1 This is a flowchart illustrating the steps of the method of the present invention;

[0048] Figure 2 This is a schematic diagram of the solder joint area of ​​the present invention;

[0049] Figure 3 This is a schematic diagram of the welding contour points of the present invention;

[0050] Figure 4 This is a schematic diagram of the welding contour points used as an example of the present invention.

[0051] Figure 5 This is a schematic diagram illustrating the numbering of the welding contour points according to the present invention;

[0052] Figure 6 This is a schematic diagram of the morphological analysis coordinate system of the present invention;

[0053] Figure 7 This is a schematic diagram of the welding center of the present invention;

[0054] Figure 8This is a schematic diagram of the coordinate system for grayscale gradient analysis in this invention;

[0055] Figure 9 This is a schematic diagram of the feature learning coordinate system of the present invention;

[0056] Figure 10 This is a schematic diagram of the morphological reference features of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0058] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0059] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0060] Existing PCBA circuit board processing quality inspection technologies typically require excessive manual intervention and focus too much on the morphological characteristics of solder joints. However, solder joints, in addition to their morphological characteristics, also exhibit color characteristics on their surface. Normal solder joint surfaces are usually smooth, implying a predictable color variation. An uneven solder joint surface indicates problems during soldering, which is essentially a processing defect, and some defects can manifest in surface color. Therefore, relying solely on morphological characteristics to inspect PCBA circuit board processing quality makes a comprehensive assessment difficult. To address these issues, this application provides a multi-source sensor fusion-based real-time PCBA circuit board processing quality inspection method. Please refer to [link to relevant documentation]. Figure 1 As shown, it includes the following steps:

[0061] Step S1 involves acquiring processed images of the PCBA circuit board using multi-source sensors, and simultaneously performing image preprocessing on the processed images. Step S1 includes the following sub-steps:

[0062] Step S101: Install multi-source sensors on the PCBA circuit board processing production line. The multi-source sensors include a 3D line laser profilometer and a high-definition industrial camera.

[0063] Step S102: Use a 3D line laser profilometer to scan the processed PCBA circuit board to obtain the three-dimensional profile of the PCBA circuit board, which is named 3D laser profile map.

[0064] Step S103: Use a high-definition industrial camera to photograph the processed PCBA circuit board to obtain a high-definition surface image of the PCBA circuit board. At the same time, perform grayscale processing on the high-definition surface image. The 3D laser contour image and the high-definition surface image are the processed images.

[0065] In practice, after the PCBA circuit board is soldered, a 3D line laser profilometer and a high-definition industrial camera are used to photograph the PCBA circuit board. At the same time, the processed images do not include the background and only include the surface of the PCBA circuit board. This can be accomplished by existing technologies such as contour extraction. This embodiment does not impose specific limitations. To facilitate computer processing, the high-definition surface image needs to be converted into a grayscale image.

[0066] Step S2 involves identifying the coordinates of solder joints on the PCBA circuit board using the 3D laser contour map in the processed image, and simultaneously obtaining the pixels belonging to the solder joints in the high-resolution surface image of the processed image based on the solder joint coordinates, naming them as image solder joints. Step S2 includes the following sub-steps:

[0067] Step S201: Identify the solder joint coordinates in the PCBA circuit board using the height information of different coordinates in the 3D laser contour map;

[0068] Step S201 includes the following sub-steps:

[0069] Step S2011: Obtain the highest height in the 3D laser contour map, and take the median of the highest height as the weld point identification parameter;

[0070] Step S2012: Slice the 3D laser contour map into layers to obtain different height layers. Obtain the height layer whose height is equal to the solder joint recognition parameter and name it the solder joint recognition layer.

[0071] Step S2013: In the solder joint identification layer, there are different closed regions enclosed by different curves, which are named solder joint subordinate regions;

[0072] Step S2014: Obtain the coordinates of the coordinate point on the slave area of ​​the solder joint, and name it the solder joint coordinates;

[0073] In practice, since solder joints protrude from the PCBA board surface, the highest point must be the coordinate point corresponding to the solder joint. The midpoint is used as the solder joint identification parameter because solder joint heights are not uniform; using only the highest point would ignore most solder joints. Since the height of a solder joint is much higher than the PCBA board surface, the midpoint must still be a solder joint. It's important to note that the heights at different positions in the 3D laser contour map are relative to the PCBA board height, not the conveyor belt height. That is, the PCBA board surface height is assumed to be 0. For example, if the highest height in a 3D laser contour map is 1mm, then the solder joint identification parameter is 0.5. Since the 3D laser contour map is actually a 3D model, it can be sliced ​​and layered. Assuming that the slices are layered in units of 0.1mm, 11 height layers can be obtained, corresponding to heights of 0mm, 0.1mm, 0.2mm, ..., 0.9mm and 1mm respectively. The height layer with a height of 0.5mm is taken as the solder joint identification layer. The area of ​​the model within the solder joint identification layer must belong to the solder joint, and each solder joint will form a closed area, i.e., the solder joint subordinate area. The solder joint coordinates can be extracted by placing the 3D laser contour map on a reference coordinate system where both the X-axis and Y-axis are lengths.

[0074] Step S202: Obtain the pixels belonging to the solder joints in the high-definition surface image based on the solder joint coordinates to obtain the image solder joints;

[0075] Step S202 includes the following sub-steps:

[0076] Step S2021: Place the top view of the 3D laser contour map and the high-definition surface map in the same reference coordinate system, find the pixel point corresponding to the coordinate of the weld point in the high-definition surface map, and name it the image weld point.

[0077] Step S2022: Obtain the grayscale value of the welding point in the image and name it as the welding grayscale value. At the same time, cluster the grayscale values ​​of all pixels in the high-definition surface image to obtain different grayscale clusters. Name the grayscale cluster where the welding grayscale value is located as the welding grayscale cluster. Define all grayscale values ​​in the welding grayscale cluster as welding grayscale values. Define the pixels to which the welding grayscale values ​​belong as image welding points.

[0078] In practice, the 3D laser contour map and the high-resolution surface image are placed in the same position and orientation when extracting coordinates in the reference coordinate system to ensure that their coordinates correspond. Alternatively, instead of using the weld point coordinate correspondence method, the weld point identification layer is directly overlaid with the high-resolution surface image. The image weld point is obtained by extracting the pixels that the model passes through in the weld point identification layer. The grayscale value of the image weld point is the weld grayscale value. When capturing the high-resolution surface image, a surround light source is required to ensure that there are no shadows on the weld point surface. At this time, the grayscale values ​​of all pixels in the high-resolution surface image are clustered to obtain different grayscale clusters, while the grayscale values ​​of the pixels on the weld point surface remain highly consistent. Similarly, pixels should be in the same grayscale cluster. The color of the PCBA circuit board surface is usually not similar to that of the solder joint. For example, if there are four colors on the surface of a PCBA circuit board, including dark green, light green, dark gray, and light gray, then the grayscale values ​​can be divided into four grayscale clusters through cluster analysis. At this time, pixels of different colors will be in the same grayscale cluster. The other grayscale values ​​in the grayscale cluster where the solder grayscale value is located should also belong to the solder grayscale value. Then, the pixels to which the solder grayscale value belongs are defined as solder joints in the image. This will automatically identify the pixels belonging to the solder joint in the high-definition image of the surface, without the need for manual provision of solder coordinates. It can adaptively extract solder joints of PCBA circuit boards of different models and batches.

[0079] Step S3 involves extracting the morphological features of each weld point based on its location distribution in the image, and simultaneously analyzing its color features based on the grayscale values. Step S3 includes the following sub-steps:

[0080] Step S301: Extract the morphological features of each weld point based on the region formed by the weld points in the image;

[0081] Step S301 includes the following sub-steps:

[0082] Step S3011: The region formed by consecutive adjacent image welding points is named the welding point region, and simultaneously labeled with the symbol WP. n,m This represents the image welding point located in the nth row and mth column of the high-resolution surface image, where n represents the row number and m represents the column number;

[0083] In practice, the image welding points corresponding to the same welding point must be consecutively adjacent, thus obtaining the welding point area. At the same time, the image welding points are marked for reference in subsequent analysis processes.

[0084] Please see Figures 2 to 3 As shown, in step S3012, the outermost image welding point of the welding point area is named the welding contour point, and the morphological features of each welding point are extracted based on the distribution of the welding contour points.

[0085] Please see Figures 4 to 5As shown in step S3012.a, the minimum WP is obtained by finding the minimum value of n+m. n,m As the starting point for morphological analysis, the welding contour points are sequentially numbered in a clockwise direction, using the symbol H. i This indicates that, where i is a non-zero natural number and i is the subscript of H;

[0086] In specific implementation, steps S3012.a to S3012.d are used to supplement step S3012 in detail, for example, a certain solder joint area such as Figure 2 As shown, the extracted welding contour points are as follows: Figure 3 As shown, the morphological starting analysis point obtained is WP. 264,338 The welding contour points are sequentially numbered in a clockwise direction to obtain H. i 1≤i≤414, and the starting point for morphological analysis is fixed at H1. Since the original weld contour points are used as an example, the data volume is too large, and the transitions in the morphological features obtained from subsequent analysis are too dense to distinguish, this embodiment only uses a small amount of data as an example to illustrate the morphological feature analysis process of this embodiment. For example... Figure 4 The example weld contour points are shown. When there are multiple identical weld contour points (n+m), the weld contour point with the smallest n is selected as the starting point for morphological analysis. The weld contour points are then sorted to obtain their numbers, as shown below. Figure 5 As shown.

[0087] Step S3012.b, starting with i=1, set H i WP n,m Let n and m be labeled Q1 and Q2 respectively, and H be labeled Q2. i+1 WP n,m n and m are labeled E1 and E2, respectively;

[0088] Step S3012.c, for each H i Assign a morphological orientation parameter; if E1 = Q1 and E2 > Q2, then set H... i The shape orientation parameter is assigned a value of 1; if E1 > Q1 and E2 > Q2, then H is... i The morphological orientation parameter is assigned a value of 2; if E1 > Q1 and E2 = Q2, then H... i The morphological orientation parameter is assigned a value of 3; if E1 > Q1 and E2 < Q2, then H is... i The morphological orientation parameter is assigned a value of 4; if E1 = Q1 and E2 < Q2, then H... i The morphological orientation parameter is assigned a value of 5; if E1 < Q1 and E2 < Q2, then H is... i The morphological orientation parameter is assigned a value of 6; if E1 < Q1 and E2 = Q2, then H is... iThe morphological orientation parameter is assigned a value of 7; if E1 < Q1 and E2 > Q2, then H is... i The morphological orientation parameter is assigned a value of 8, i is incremented by one, and the analysis of H is repeated. i The morphological orientation parameters;

[0089] Please see Figure 6 As shown, in step S3012.d, H i The morphological orientation parameter is labeled as G. i The subscript i represents the X-axis, and the shape direction parameter G i Establish a morphological analysis coordinate system for the Y-axis, and set G... i Enter the morphological analysis coordinate system according to i, connect adjacent coordinate points in the morphological analysis coordinate system with line segments to obtain an irregular broken line, which is named morphological feature;

[0090] In specific implementation, Figure 5 Taking H1 as an example, H1 is WP at this time. 1,3 That is, Q1 is 1, Q2 is 3, and H2 is WP. 1,4 That is, E1 is 1 and E2 is 4. At this time, E1=Q1 and E2>Q2. Therefore, the shape direction parameter G1 of H1 is assigned to 1. Similarly, the analysis yields the result from H2 to H. 12 The morphological orientation parameters G2 to G 12 The numbers are 2, 2, 3, 4, 4, 5, 6, 6, 7, 8, and 8 in sequence. Then, a morphological analysis coordinate system is constructed as follows: Figure 6 As shown, Figure 6 The broken lines in the diagram represent morphological features. These morphological features reveal the direction of the weld joint's outline. Well-welded weld joints usually have a more regular outline, while weld joints with morphological defects usually have a more chaotic outline.

[0091] Step S302: Analyze the color characteristics of the welding points based on the distribution and changes in the grayscale values ​​of the welding points in the image;

[0092] Step S302 includes the following sub-steps:

[0093] Please see Figure 7 As shown, in step S3021, find the highest point of the weld joint area in the 3D laser contour map and name it the weld center;

[0094] Step S3022, with welding center and H i Draw a line segment with the geometric center as its endpoint, name it the grayscale direction line, and use the symbol L. i H represents i The corresponding grayscale direction line;

[0095] In practice, the highest point of the weld joint area in the 3D laser contour map is used to obtain the welding center. This welding center is the center of the weld joint. The height of the weld joint gradually decreases from the welding center outwards, and its surface color also exhibits a certain gradient change. Figure 5 For example, the welding center can be located as follows: Figure 7 As shown, Figure 7 The dark gray dot in the image represents the welding center, and Figure 7 The example shows the grayscale trend line L1 between the welding center and H1.

[0096] Step S3023: Extract the color features of the welding points based on the gray values ​​of the welding points in the image through which the gray-scale trend line passes.

[0097] Step S3023.a: Obtain the image welding points through which the grayscale direction line passes, name them grayscale direction points, and obtain the grayscale values ​​of the grayscale direction points, name them gradient change grayscale.

[0098] Step S3023.b: Sort the gradient change grayscale values ​​in order of increasing distance from the grayscale trend point to the welding center, using the symbol F. j This indicates that, where j is a non-zero natural number and j is the subscript of F;

[0099] Please see Figure 8 As shown, in step S3023.c, with F j The subscript j in the figure represents the X-axis, and the grayscale F is changed by gradient. j Establish a grayscale gradient analysis coordinate system for the Y-axis, and set F j Enter the grayscale gradient analysis coordinate system according to j;

[0100] Step S3023.d: Perform function fitting on the gray-level gradient analysis coordinate system to obtain the gray-level gradient change curve, which is the color feature;

[0101] In specific implementation, steps S3023.a to S3023.d are used to supplement step S3023 in detail. Figure 7 In the middle, L1 passed through WP 1，3 WP 2，3 and WP 3，3 , and WP 3，3 WP is closest to the welding center. 1，3 The WP is furthest from the welding center. 3，3 WP 2，3 and WP 1，3 The grayscale values ​​are labeled F1 to F3 sequentially. It should be noted that this is a small example provided for ease of understanding the analysis process in this embodiment. In actual analysis, there are numerous gradient changes in grayscale values. The resulting grayscale gradient analysis coordinate system is as follows: Figure 8As shown, Figure 8 The curve shown is the grayscale gradient curve, i.e., the color feature, revealing the trend of grayscale value changes between the center and edge of the solder joint. Here, only the color feature of H1 is analyzed; similarly, each H... i Its color characteristics.

[0102] Step S4 involves real-time detection of PCBA circuit board processing quality issues based on the morphological and color characteristics of the solder joints. Step S4 includes the following sub-steps:

[0103] Step S401: Obtain PCBA circuit boards with no processing defects as quality inspection samples, analyze the morphological and color characteristics of the quality inspection samples, and name them as normal morphological characteristics and normal color characteristics, respectively.

[0104] In practice, the quality inspection samples are provided by manual screening during the deep learning process, and no manual intervention is required after the deep learning is completed.

[0105] Step S402: Perform deep learning on the normal morphological features and normal color features to obtain morphological reference features and color reference features.

[0106] Please see Figure 9 As shown, in step S402.a, normal morphological features and normal color features are collectively referred to as features to be learned. Each feature to be learned is analyzed independently, and the features to be learned are placed in the same coordinate system, which is named the feature learning coordinate system.

[0107] Please see Figure 10 As shown, in step S402.b, the region between any two features to be learned in the feature learning coordinate system is named the feature reference region. The feature reference regions of normal morphological features and normal color features are analyzed to obtain morphological reference features and color reference features.

[0108] In specific implementation, steps S402.a and S402.b are used to supplement step S402 in detail. Since the analysis process for morphological reference features and color reference features is the same, this embodiment only uses morphological reference features as the features to be learned as an example for illustration. Due to the large amount of original data, it is not conducive to observation in this embodiment. Therefore, this embodiment only uses three morphological reference features as an example for illustration. The three morphological reference features are placed in the same morphological analysis coordinate system to obtain the feature learning coordinate system as follows. Figure 9 As shown, Figure 9The three lines of different colors represent three morphological reference features. When X=2, for any value of Y within the range [1,2], there is always a morphological reference feature above and below it. This is considered as being above or below the morphological reference feature. Therefore, the range of Y for the morphological reference feature at X=2 is [1,2]. Similarly, analyzing the range of Y for each value of X, we finally obtain the morphological reference features as follows: Figure 10 As shown, Figure 10 The gray area in the diagram is the morphological reference area; similarly, the color reference features can be analyzed.

[0109] Step S403: Use morphological reference features and color reference features as references to analyze whether there are any problems with the PCBA circuit board processing quality;

[0110] Step S403 includes the following sub-steps:

[0111] Step S4031: Name the range of the morphological reference feature on the X-axis as the morphological range, and name the range of the color reference feature on the X-axis as the color range.

[0112] Step S4032: If the morphological features of the PCBA circuit board are within the morphological reference features within the morphological range, and the color features are within the color reference features within the color range, then the PCBA circuit board is marked as having good processing quality; otherwise, the PCBA circuit board is marked as having defects in processing quality.

[0113] In practice Figure 10 The morphological reference feature shown has a morphological range of [1, 13], representing that the morphological features of most solder joints are formed by a maximum of 13 weld contour points. The 14th weld contour point cannot form a merged interval and is therefore not considered. If the morphological features of the solder joint are all within the range of X [1, 13]... Figure 10 If there are no line segments extending beyond the gray area shown, it indicates that the morphological characteristics of the solder joint are good and there are no abnormalities. The same applies to the judgment of color characteristics. If there are abnormalities in the shape of the solder joint, its morphological characteristics will definitely show abnormal transitions. These abnormalities are usually outside the morphological reference features.

[0114] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0115] Memory, communication interfaces, and processor are coupled together. For example, memory, communication interfaces, and processor can be coupled together via a bus.

[0116] The communication interface is used for data transmission with other devices. Computer program code is stored in the memory. The computer program code includes computer instructions, which, when executed by a processor, cause the electronic device to perform the relevant method steps described in this application.

[0117] A processor can be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. A processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0118] The bus can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0119] This application also provides a server, which includes a processor, a memory, and a communication interface; the memory is used to store a computer-executable program; when the computer-executable program is invoked by the processor, the processor is used to execute the relevant method steps in the above method embodiments through the communication interface.

[0120] This application also provides a communication device, which includes a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the communication device performs the relevant method steps in the above method embodiments.

[0121] This application also provides a computer-readable storage medium storing computer program code. When the processor executes the computer program code, the electronic device executes the relevant method steps in the above method embodiments.

[0122] This application also provides a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the relevant method steps as described in the above method embodiments.

[0123] The electronic devices, computer storage media, or computer program products provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0125] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for real-time detection of PCBA circuit board processing quality based on multi-source sensor fusion, characterized in that, include: Images of PCBA circuit boards after processing are acquired using multi-source sensors, and image preprocessing is performed on the processed images. The coordinates of solder joints in the PCBA circuit board are identified by the 3D laser contour map in the processed image. At the same time, the pixels belonging to the solder joints in the high-definition surface map of the processed image are obtained based on the solder joint coordinates and named as image solder joints. The morphological features of each weld point are extracted based on the location distribution of the weld points in the image, and the color features of the weld points are analyzed based on the grayscale values ​​of the weld points in the image. Real-time detection of PCBA circuit board processing quality issues based on the morphological and color characteristics of solder joints.

2. The method according to claim 1, characterized in that, The step of acquiring processed images of PCBA circuit boards using multi-source sensors and simultaneously performing image preprocessing on the processed images further includes the following sub-steps: Multi-source sensors are installed on the PCBA circuit board processing production line, including a 3D line laser profilometer and a high-definition industrial camera; The processed PCBA circuit board was scanned using a 3D line laser profilometer to obtain the three-dimensional profile of the PCBA circuit board, which was named 3D laser profiling image. A high-definition industrial camera is used to photograph the processed PCBA circuit board to obtain a high-definition surface image of the PCBA circuit board. At the same time, the high-definition surface image is processed into grayscale. The 3D laser contour image and the high-definition surface image are the processed images.

3. The method according to claim 2, characterized in that, The step of identifying solder joint coordinates in a PCBA circuit board using a 3D laser contour map from a processed image, and simultaneously obtaining pixels belonging to solder joints from a high-resolution surface image of the processed image based on the solder joint coordinates, further includes the following sub-steps: Identify solder joint coordinates in PCBA circuit boards by using height information at different coordinates in 3D laser contour maps; The image of the weld point is obtained by acquiring the pixels belonging to the weld point in the high-definition surface image based on the weld point coordinates.

4. The method according to claim 3, characterized in that, The step of identifying solder joint coordinates in a PCBA circuit board using height information from different coordinates in a 3D laser contour map further includes the following sub-steps: Obtain the highest height in the 3D laser contour map, and take the median of the highest height as the weld point identification parameter; The 3D laser contour map is sliced ​​and layered to obtain different height layers. The height layer with a height equal to the solder joint recognition parameter is obtained and named the solder joint recognition layer. The solder joint identification layer contains different closed regions enclosed by different curves, which are named solder joint subordinate regions. Obtain the coordinates of the point located on the slave area of ​​the solder joint, and name it the solder joint coordinates.

5. The method according to claim 4, characterized in that, The step of obtaining the pixels belonging to the solder joints in the high-definition surface image based on the solder joint coordinates to obtain the image solder joint also includes the following sub-steps: Place the top view of the 3D laser contour map and the high-definition surface map in the same reference coordinate system, find the pixel point corresponding to the coordinate of the weld point in the high-definition surface map, and name it the image weld point. The grayscale value of the welding point in the image is obtained and named as the welding grayscale value. At the same time, the grayscale values ​​of all pixels in the high-definition surface image are clustered to obtain different grayscale clusters. The grayscale cluster in which the welding grayscale value is located is named as the welding grayscale cluster. All grayscale values ​​in the welding grayscale cluster are defined as welding grayscale values, and the pixels to which the welding grayscale values ​​belong are defined as image welding points.

6. The method according to claim 5, characterized in that, The step of extracting the morphological features of each weld point based on its positional distribution in the image, and analyzing its color features based on its grayscale values, further includes the following sub-steps: The morphological features of each weld point are extracted from the region formed by the weld points in the image. The color characteristics of the weld points are analyzed based on the distribution and changes in the grayscale values ​​of the weld points in the image.

7. The method according to claim 6, characterized in that, The step of extracting the morphological features of each weld point from the region formed by the weld points in the image further includes the following sub-steps: The region consisting of consecutive adjacent image welding points is named the welding point region, and is also labeled with the symbol WP. n,m This represents the image welding point located in the nth row and mth column of the high-resolution surface image, where n represents the row number and m represents the column number; The outermost image welding point in the welding point area is named the welding contour point, and the morphological features of each welding point are extracted based on the distribution of the welding contour points.

8. The method according to claim 7, characterized in that, The step of analyzing the color features of the weld joints based on the distribution and changes in the grayscale values ​​of the weld joints in the image also includes the following sub-steps: Find the highest point of the weld joint area in the 3D laser contour map and name it the weld center. With welding center and H i Draw a line segment with the geometric center as its endpoint, name it the grayscale direction line, and use the symbol L. i H represents i The corresponding grayscale direction line; The color features of the weld points are extracted based on the gray values ​​of the weld points along the gray-scale trend line in the image.

9. The method according to claim 8, characterized in that, The step of real-time detection of PCBA circuit board processing quality issues based on the morphological and color characteristics of solder joints also includes the following sub-steps: PCBA circuit boards with no processing defects were obtained as quality inspection samples. The morphological and color characteristics of the quality inspection samples were analyzed and named as normal morphological characteristics and normal color characteristics, respectively. Deep learning is performed on normal morphological features and normal color features to obtain morphological reference features and color reference features; We use morphological and color reference features as references to analyze whether there are problems with the processing quality of PCBA circuit boards.

10. The method according to claim 9, characterized in that, The step of analyzing whether there are problems with the PCBA circuit board processing quality using morphological reference features and color reference features as references also includes the following sub-steps: The range of the morphological reference feature on the X-axis is named the morphological range, and the range of the color reference feature on the X-axis is named the color range. If the morphological features of the PCBA circuit board are within the morphological reference features within the morphological range, and the color features are within the color reference features within the color range, then the PCBA circuit board is marked as having good processing quality; otherwise, the PCBA circuit board is marked as having defects in processing quality.

Images