Multi-station PCBA board detection method based on machine vision

Abstract

The invention relates to the technical field of automatic detection, in particular to a multi-station PCBA (printed circuit board assembly) detection method based on machine vision, which comprises the following steps: acquiring a multi-station reference pulse period, dividing a time window, comparing starting time difference, controlling image acquisition to synchronize, locally imaging a PCBA based on a synchronizing signal, extracting temperature response of a welding spot, and constructing a heat conduction path. Mapping to a visible light image, evaluating welding spot symmetry, extracting welding spot hue change, marking abnormal frames, recording defect path nodes, and generating a defect path structure. According to the method, the temperature response coordinate points in the welding spot area are extracted and the heat conduction direction vector path is constructed, so that the welding quality is evaluated more carefully and accurately, the thermal grid chart is mapped to the visible light image frame, the image analysis process is further optimized, the defect detection accuracy is enhanced, and the potential defect area is effectively identified; the efficiency and reliability of PCBA board detection are significantly improved, and powerful technical support is provided for high-quality production.

Description

Technical Field

[0001] The present invention relates to the technical field of automated detection, and particularly to a multi-station PCBA board detection method based on machine vision. Background Art

[0002] The technical field of automated detection encompasses a series of methods and devices for performing quality control and inspection tasks without manual intervention. The core content is to utilize automated tools and systems to detect, identify, and evaluate product quality or process parameters. Automated detection technology is widely applied in manufacturing, online monitoring, security inspections, and many other industries, with quality control on production lines being a common application scenario. In this technical field, various sensors, cameras, and computing devices work together to achieve automatic detection of production items, thereby improving efficiency, reducing errors, and ensuring the consistency of product quality.

[0003] Among them, the multi-station PCBA board detection method based on machine vision refers to a method of using machine vision to perform quality detection during the production process of PCBA boards. The technical matters targeted by this technical theme cover using a specific vision system to capture images on PCBA boards and detecting the component positions, solder joint quality, and potential defects on the circuit boards through image analysis technology. This detection method obtains high-resolution images of the circuit boards through integrated vision devices and then uses image processing technology to analyze the images in detail to identify manufacturing defects or assembly errors.

[0004] Existing automated detection relies on traditional image processing techniques and fixed detection parameters, which are not flexible and accurate enough when dealing with complex or changing production environments. On the production line of PCBA boards, due to the high component density and complex distribution, traditional methods cannot accurately distinguish between minor defects or solder joint problems. The lack of an effective synchronization mechanism results in inconsistent image capture timing during high-speed production, affecting the overall detection efficiency and accuracy. Due to technical limitations, defects on the production line are missed, leading to a decrease in the yield rate, increasing the economic burden and market risks of enterprises. Traditional automated detection technology is difficult to meet the strict standards of modern electronics manufacturing in the face of high-precision and high-complexity detection requirements. Summary of the Invention

[0005] In order to solve the problems that the existing technology is not flexible and accurate enough when dealing with complex or changing production environments. On the PCBA board production line, due to the high component density and complex distribution, traditional methods cannot accurately distinguish minor defects or solder joint problems. The lack of an effective synchronization mechanism leads to inconsistent image capture timing during high-speed production, affecting the overall detection efficiency and accuracy. Due to technical limitations, defects on the production line are missed, resulting in a decrease in the yield rate, increasing the economic burden and market risk of the enterprise. Facing the detection requirements of high precision and high complexity, traditional automated detection technologies are difficult to meet the strict standards of modern electronic manufacturing. The present invention provides a multi-station PCBA board detection method based on machine vision. The technical solution is as follows:

[0006] On the one hand, a multi-station PCBA board detection method based on machine vision is provided, and the method includes:

[0007] S1: Obtain the multi-station reference pulse period, divide the period into multiple time window sections, extract the starting point of the corresponding allocated time window as the light source pulse start reference point for this station, compare the current light source start time with the real-time capture start time of the previous image frame. If the time difference exceeds the specified tolerance range, control the image acquisition to trigger with a delay, and generate a multi-station synchronization control signal;

[0008] S2: Based on the multi-station synchronization control signal, perform local imaging on the soldered PCBA board, extract the temperature response coordinate points within the pin solder joint area, sort them serially according to the included angle in the fan-shaped expansion direction, and generate a thermal trajectory grid map;

[0009] S3: Use the thermal trajectory grid map to map the thermal grid map onto the corresponding visible light image frame, identify the overlapping boundary area between the blurred area of the visible image and the thermal path, and use the overlapping area as the complement area to generate a solder joint boundary complement area;

[0010] S4: Utilize the solder joint boundary complement area to extract the hue change path of the solder joint area in multiple consecutive frames of the image, separate the pixel energies of the RGB three channels in each frame of the image, and compare the direction offsets of the hue main axis change vectors in each frame to generate a defect determination node set.

[0011] As a further solution of the present invention, the multi-station synchronization control signal includes a reference trigger point, a tolerance deviation value, and a delay trigger amount. The thermal trajectory grid map includes thermal response coordinate points, a heat conduction direction vector, and component package pairing coordinates. The solder joint boundary complement area includes an overlapping area range, a solder joint symmetry parameter, and a solder joint integrity index. The defect determination node set includes an abnormal frame number, a hue change offset value, and an abnormal determination threshold.

[0012] As a further solution of the present invention, the steps for obtaining the multi-station synchronous control signal are specifically as follows:

[0013] S101: Obtain the multi-station reference pulse period, divide the pulse period into multiple time window sections, and set the corresponding start time point and end time point for the time window. Locate the allocated time window section according to the number of the real-time station, and generate a time window start reference value by recording the start time point of the time window as the reference point for starting the light source pulse of this station.

[0014] S102: Call the time window start reference value and the real-time capture start time value of the previous image frame, perform a subtraction operation on the two to calculate the time difference, and judge whether the time difference falls within the set range based on the light source trigger offset tolerance interval. If it exceeds the interval, mark the light source trigger state as delayed and generate a trigger delay mark value.

[0015] S103: Set the delay parameter for the real-time frame image acquisition operation according to the trigger delay mark value. When there is a delay mark, place the delay parameter value in the offset correction interval and postpone the trigger time. After the time difference re-enters the light source trigger offset tolerance interval, generate a multi-station synchronous control signal.

[0016] As a further solution of the present invention, the steps for obtaining the thermal trajectory grid map are specifically as follows:

[0017] S201: Based on the multi-station synchronous control signal, call the image positioning parameters and station information of the welded PCBA board, perform a local thermal imaging task on the specified area, capture the thermal image frame, and locate multiple temperature response coordinate points within the pin solder joint area to generate a solder joint temperature coordinate set.

[0018] S202: Call the temperature response values of the coordinate points in the solder joint temperature coordinate set, detect the temperature gradient change trend in the surrounding neighborhood grid points, and merge the vectors of the coordinate points into the main cooling direction with reference to the temperature difference between the real-time coordinate points and the neighborhood grid points to generate a solder joint cooling direction vector group.

[0019] S203: According to the direction angle value of each vector in the solder joint cooling direction vector group, set a fixed included angle sector division interval, classify the vectors according to the direction angle into the corresponding sequence, calculate the direction difference offset value, and perform number sorting according to the angle sequence of the sector interval to generate a thermal direction sequence path set.

[0020] S204: Call the path end coordinate points of the thermal direction sequence path set, and match and pair them with the component package geometric position model to verify the component package positions associated with the end points of each heat conduction path. Generate a thermal trajectory grid map with reference to the path connection relationship and the component position points.

[0021] As a further solution of the present invention, the formula for calculating the direction difference offset value is as follows:

[0022]

[0023] where θ i represents the direction difference offset value of the i-th vector, α i represents the direction angle value of the i-th vector, n i represents the total number of vectors in the same sector interval as α i α j represents the direction angle value of the j-th vector, w j represents the heat intensity weight of the j-th vector, d j represents the distance from the solder joint corresponding to the j-th vector to the heat source center, represents the average value of the vector direction angle values in the entire sector interval, j represents the vector index number in the same sector interval as i, and i represents the vector index number.

[0024] As a further solution of the present invention, the steps for obtaining the solder joint boundary completion region are specifically as follows:

[0025] S301: Call the coordinates of the heat path nodes marked in the heat trajectory grid map, perform projection mapping on the heat path coordinates according to the image frame coordinate system, map them to the homologous coordinate region of the corresponding visible light image frame, evaluate the correspondence between the heat map nodes and the visible image pixel points, and generate the heat map mapping pixel distribution;

[0026] S302: Based on the pixel intensity gradient in the area covered by the heat map mapping pixel distribution, detect the intensity change in the image blur area, then perform overlapping coordinate judgment on the boundary of the blur area and the heat map path mapping boundary, extract the intersection area of the two, and generate the solder joint overlapping boundary coordinate set;

[0027] S303: Call the position of the boundary vertex in the solder joint overlapping boundary coordinate set, calculate the difference value with the standard symmetric boundary range of the corresponding solder joint in the PCBA board solder joint structure model, adjust the contour of the solder joint area in the real-time image frame, perform contour interpolation and completion on the missing boundary area and close and reconstruct it to generate the solder joint boundary completion region.

[0028] As a further solution of the present invention, the steps for obtaining the defective determination node set are specifically as follows:

[0029] S401: Call the spatial boundary coordinates marked in the solder joint boundary completion region, extract the hue data in the corresponding region in multiple consecutive frames of images, calculate the RGB pixel energy value of each frame of image, perform channel separation on the RGB pixels in the region of each frame of image, and generate the solder joint channel energy sequence set;

[0030] S402: Based on the RGB energy distribution values of each frame in the solder joint channel energy sequence set, calculate the unit vector direction of the main hue axis of each frame, and perform an interval judgment on the offset angle value of the frame with the offset threshold. If the offset angle is greater than the offset threshold, mark the corresponding frame as an abnormal state frame, record the corresponding number of the abnormal state frame image, and generate a defect determination node set.

[0031] As a further solution of the present invention, the formula for calculating the RGB pixel energy value of each frame image is as follows:

[0032]

[0033] Where, E RGB (t) represents the pixel energy value of the RGB channel in each frame image, and R a (t), G a (t), B a (t) respectively represent the RGB values of the red, green, and blue channels of the a-th pixel point in the t-th frame image, respectively represent the average RGB values of each channel, and N represents the number of pixel points in the image.

[0034] As a further solution of the present invention, the method further includes step S5:

[0035] S5: Through the defect determination node set, when a branch segment in the path is triggered by a defect determination node in image recognition, record the response frequency of the path node and the front and rear path structures, record the information of the abnormal path segment, and replace the light source trigger sequence, thermal mapping reference frame number, and hue direction vector information in the corresponding node of the abnormal segment to generate a PCBA board defect path structure;

[0036] The PCBA board defect path structure includes abnormal node response frequency, path structure map, trigger order correction parameter, thermal mapping reference frame serial number, and hue direction correction vector.

[0037] As a further solution of the present invention, the obtaining steps of the PCBA board defect path structure are specifically as follows:

[0038] S501: Call the number information of the abnormal state nodes in the defect determination node set, trace the position of the node in the path segment belonging to the image recognition path. When there are multiple path branch segments at the node trigger position, record the node sequence in the path segment and the connection relationship with adjacent nodes, evaluate the relationship between the response frequency of the path branch segment and the structure, and form an abnormal path structure mark;

[0039] S502: Based on the path branch segment position marked in the abnormal path structure mark, the light source trigger sequence value, thermal mapping reference frame number value and hue direction vector value recorded in the original path node are searched accordingly, and the parameters of the adjacent non-abnormal path segment are used as the replacement source. The information is reset in the node index order to generate the PCBA board defect path structure.

[0040] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0041] By using synchronous control signals to synchronize multiple stations, the precise alignment of the operation time windows of each station is ensured, which significantly reduces image capture errors caused by time deviation and improves detection accuracy and repeatability. By extracting the temperature response coordinate points in the solder joint area and constructing the heat conduction direction vector path, the evaluation of welding quality is more detailed and accurate. Mapping the thermal grid map to the visible light image frame further optimizes the image analysis process and enhances the accuracy of defect detection. Using hue change path analysis, the sensitivity and responsiveness to solder joint anomalies are enhanced, potential defect areas are effectively identified, and product quality assurance is improved. Through precise synchronization, temperature response analysis, and hue change path analysis, the efficiency and reliability of PCBA board detection are significantly improved, providing strong technical support for high-quality production. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the workflow of the present invention; DETAILED DESCRIPTION

[0043] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0044] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0045] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0046] See also Figure 1 The embodiment of the present invention provides a multi-station PCBA board detection method based on machine vision. The processing flow of the method may include the following steps:

[0047] S1: Obtain the reference pulse period of multiple workstations, divide the period into multiple time window segments, extract the starting points of the corresponding allocated time windows as the light source pulse start reference points for this workstation, compare the current light source start time with the real-time capture start time of the previous image frame. If the time difference exceeds the specified tolerance range, control the image acquisition to trigger with a delay until the synchronization deviation falls within the target window, and generate a multi-workstation synchronization control signal;

[0048] S2: Based on the multi-workstation synchronization control signal, perform local imaging on the soldered PCBA board, extract the temperature response coordinate points within the pin solder joint area, extract the temperature response change information around the coordinate points, perform serialization sorting according to the included angle in the fan-shaped expansion direction, construct a heat conduction direction vector path, extend the path and pair it with the corresponding positions of the component package to generate a heat trace grid map;

[0049] S3: Use the heat trace grid map, map the heat grid map onto the corresponding visible light image frame, identify the overlapping boundary area between the blurred area of the visible image and the heat path, use the overlapping area as the filling area, evaluate the solder joint symmetry and integrity of the PCBA board, and generate a solder joint boundary filling area;

[0050] S4: Utilize the solder joint boundary filling area, extract the hue change path of the solder joint area in multiple consecutive frames of the image, separate the pixel energies of the RGB three channels in each frame of the image, compare the direction offset of the hue main axis change vector in each frame. If an abnormal offset occurs, mark that frame as an abnormal frame, and generate a set of defect determination nodes;

[0051] S5: Through the set of defect determination nodes, when a branch segment in the defect determination node trigger path in image recognition is triggered, record the response frequency of the path nodes and the front and rear path structures, record the information of the abnormal path segment, and replace the light source trigger sequence, heat mapping reference frame number, and hue direction vector information in the corresponding nodes of the abnormal segment to generate a PCBA board defect path structure;

[0052] The multi-workstation synchronization control signal includes a reference trigger point, a tolerance deviation value, and a delay trigger amount. The heat trace grid map includes heat response coordinate points, heat conduction direction vectors, and component package pairing coordinates. The solder joint boundary filling area includes the overlapping area range, solder joint symmetry parameters, and solder joint integrity indicators. The set of defect determination nodes includes abnormal frame numbers, hue change offset values, and abnormal determination thresholds. The PCBA board defect path structure includes abnormal node response frequencies, path structure maps, trigger sequence correction parameters, heat mapping reference frame numbers, and hue direction correction vectors.

[0053] The specific steps for obtaining the multi-workstation synchronization control signal are as follows:

[0054] S101: Obtain the reference pulse period for multiple workstations, evenly divide the pulse period into multiple time window segments, set the corresponding start time point and end time point for the time window, locate the allocated time window segment according to the number of the real-time workstation, and generate the time window start reference value by recording the start time point of the time window as the reference point for the start of the light source pulse of this workstation;

[0055] In multiple workstations, it is necessary to coordinate the light source control of each workstation on a unified time basis. Therefore, the central controller reads the main pulse signal sequence, which is a continuous pulse stream generated by a high-frequency clock module. The complete pulse is equally divided according to the set period. When the pulse period is set to 100 milliseconds per round and there are five workstations, the period is divided into five equal time windows, each with a length of 20 milliseconds. The control program registers the start time and end time of each segment in the time window control table respectively, corresponding to the unique number of each workstation. For example, the time window of workstation 2 is from 20 milliseconds to 40 milliseconds, and the time window of workstation 3 is from 40 milliseconds to 60 milliseconds, and so on. During operation, according to the number of the currently activated workstation, quickly index the time window segment of this workstation, and use the start time of this segment as the basic reference time for the next trigger of the light source of this workstation. If the current workstation is workstation 4, its reference time is 60 milliseconds. Record this time in the form of a data structure for subsequent scheduling judgment processes, construct a mapping list containing the workstation number and the start reference time, and call this value in real time as the pulse start reference. In this way, ensure that the light source of each workstation starts within its own time window and generate the time window start reference value.

[0056] S102: Call the time window start reference value and the real-time capture start time value of the previous image frame, perform a subtraction operation on the two to calculate the time difference, and judge whether the time difference falls within the set range based on the light source trigger offset tolerance interval. If it exceeds the interval, mark the light source trigger state as delayed and generate a trigger delay mark value;

[0057] To ensure the timeliness and stability of image acquisition at each workstation, it is necessary to compare in real time the difference between the reference benchmark time of the workstation light source and the actual start time of image capture. Read the timestamp information at the moment of capturing the current frame image from the image acquisition. This time point is when the camera starts recording image data after detecting the light source. Referring to the benchmark time that should be started in the current cycle of this workstation, compare the difference between the two. If the difference is outside the allowable error range, either too early or too late, mark the status of this workstation as a trigger delay status. To implement this judgment, a judgment criterion is set internally, that is, the allowable time offset tolerance range for light source triggering. This range is a preset allowable error value, used to evaluate whether the current actual image acquisition time is close to the theoretically set time. Through a simple comparison operation, generate a marking parameter according to the judgment result, indicating whether this workstation needs to adjust the subsequent image acquisition process due to trigger deviation. If the image acquisition time is 2 milliseconds later than the benchmark time and the tolerance is 1 millisecond, then it exceeds the tolerance. At this time, the control program marks this workstation as a delayed status and generates a trigger delay marking value.

[0058] S103: Set the delay parameter for the real-time frame image acquisition operation according to the trigger delay marking value. When there is a delay marking, place the delay parameter value in the offset correction interval and postpone the trigger time. After the time difference re-enters the light source trigger offset tolerance interval, generate a multi-workstation synchronization control signal;

[0059] After determining that there is a trigger delay status for a certain workstation, it is necessary to immediately adjust the image acquisition strategy of this workstation to correct the trigger time. Generate a delay parameter according to the difference in the previous stage. This parameter represents the specific value by which the current frame acquisition time deviates from the preset benchmark. Set an adjustment value for this workstation in the control parameter table and introduce this adjustment amount before the next light source trigger control signal is issued, so that the new trigger time approaches the target benchmark. When recording that the current deviation is 2 milliseconds, set the trigger time to be postponed by 2 milliseconds for this workstation, that is, postpone the execution of the next frame image acquisition moment by 2 milliseconds. In the next execution, re-compare the difference between the actual acquisition time and the reference time. If the offset still exists and exceeds the tolerance, continue to adjust the delay parameter and iterate repeatedly until the difference returns within the allowable range. Once it is detected that the acquisition time of this workstation is close to the reference time and meets the set error standard, it is determined that the time synchronization of this workstation is completed, and a synchronization control signal is sent to notify the master to include the current workstation in the synchronization list. This operation process ensures that multiple workstations maintain timing coordination during continuous operation cycles, uniformly send synchronization trigger signals to execute the image acquisition task for all workstations, and generate a multi-workstation synchronization control signal.

[0060] The specific steps for obtaining the thermal trajectory grid map are as follows:

[0061] S201: Based on the multi-station synchronous control signal, call the image positioning parameters and station information of the welded PCBA board, perform a local thermal imaging task on the specified area, capture the thermal image frame, and locate multiple temperature response coordinate points within the pin solder joint area to generate a solder joint temperature coordinate set.

[0062] Triggered by the multi-station synchronous control signal, schedule the specified infrared thermal imaging to enter the corresponding station. The PCBA board that has undergone the soldering process is placed at this station. Call out the image positioning parameters corresponding to the board model, including the preset coordinate frame of the solder joint area, the pin layout structure, the main control chip area mark, etc., and load the station number to bind the subsequent image file naming rule. The imaging is performed with a local scan within the specified area, and the focal length and sensitivity parameters are controlled to ensure that the thermal image is focused on the area containing the pin solder joints. During the imaging process, continuously capture multiple frame image data. In each frame, call the edge recognition and thermal contour recognition algorithms to lock the area blocks with significant thermal reactions, extract and record the center coordinates as the solder joint temperature response points. Throughout the process, the multiple response points of the same solder joint are superimposed in the time dimension, and the coordinate positions between frames are fine-tuned and aligned to construct a set containing all the valid solder joint temperature response coordinates. Each coordinate point includes the spatial position (such as x, y) and the temperature value under the current frame, generating a solder joint temperature coordinate set.

[0063] S202: Call the temperature response values of the coordinate points in the solder joint temperature coordinate set, detect the temperature gradient change trend in the surrounding neighborhood grid points, and refer to the temperature difference between the real-time coordinate points and the neighborhood grid points to merge the vectors of the coordinate points into the main cooling direction, generating a solder joint cooling direction vector group.

[0064] For each coordinate point with a temperature response, call its temperature value and compare it with the temperature values of the surrounding neighborhood grid points. The neighborhood grid points are the adjacent pixel grids in the image where the point is located or the area point set set according to the coordinate radius. Set up a five-row and five-column neighborhood window centered on each point, with a total of 25 grid points. By retrieving the temperature change trend of the point, judge in which direction there is an obvious temperature drop. Calculate the temperature value difference between each neighborhood grid point and the center point, and find the direction with the largest difference. This direction is defined as the cooling direction vector of the point. The direction vector extends from the center point towards the adjacent point with the most significant temperature decrease. Repeat this process for all temperature response points in the image to obtain a set of cooling direction vectors. To maintain the consistency of the data structure, each vector is recorded in a single description format of the starting point coordinates and the pointing direction. For example, if the main cooling direction of a point is towards the upper left direction, it is recorded as the starting point coordinates and the unit vector description of the upper left direction. At the same time, classify and integrate the direction vectors, aggregate the cooling trends of multiple response points into a representative main direction set, forming a solder joint cooling direction vector group for heat flow analysis.

[0065] S203: Set the fixed - angle sector division intervals according to the direction - angle values of each vector in the solder - joint cooling - direction vector group, classify the vectors into corresponding sequences according to the direction angles, calculate the direction - difference offset values, and number and sort them according to the angle sequence of the sector intervals to generate the heat - direction sequence path set;

[0066] The formula for calculating the direction - difference offset value is as follows:

[0067]

[0068] Among them, θ i represents the direction - difference offset value of the i - th vector, α i represents the direction - angle value of the i - th vector, n i represents the total number of vectors in the same sector interval as α i , α j represents the direction - angle value of the j - th vector, w j represents the heat - intensity weight of the j - th vector, d j represents the distance from the solder joint corresponding to the j - th vector to the heat - source center, α represents the average value of the vector direction - angle values within the entire sector interval, j represents the vector index number in the same sector interval as i, and i represents the vector index number;

[0069] Meaning of parameters and derivation process of formula calculation:

[0070] The direction - angle value α i is obtained from the vector - direction monitoring of the solder - joint temperature field by an infrared thermal imager. The measurement angle is based on the heat - flow gradient direction and is measured in degrees. Taking the direction - angle value of solder joint i as α i = 58.4°, and the acquisition time is the stable stage in the high - temperature area;

[0071] The total number of vectors n i is the number of vectors belonging to the same angle - sector interval as this vector. The interval span is 45°, and in the acquisition sample, it is set as n i = 4,

[0072] The average direction - angle value is the arithmetic mean of the direction - angle values of all vectors in this sector interval, which is calculated by extracting the direction angles of each vector through an infrared - thermal - image analysis system. Suppose the direction angles of the 4 vectors in this interval are respectively:

[0073] α1 = 52.0°, α2 = 55.3°, α3 = 60.8°, α4 = 64.1°;

[0074] Then the average direction - angle value is:

[0075]

[0076] The weight w of each vector jis the normalized value of the vector heat intensity, which is obtained by converting the heat flux density per unit area of the temperature gradient collected by the infrared thermal imager. The standard conversion formula is as follows:

[0077]

[0078] where q j is the heat flux density per unit area of the solder joint at the j-th vector, with the unit of W / m 2 ; it is extracted from the thermal field experimental data and measured as follows:

[0079] q1 = 3400, q2 = 2800, q3 = 5100, q4 = 4300;

[0080] Normalization calculation:

[0081] ∑q j = 3400 + 2800 + 5100 + 4300 = 15600;

[0082]

[0083] The distance d j is the physical distance from the solder joint to the center of the heat source, which is obtained by converting the image pixel scale and is measured in millimeters. The distances are collected as follows:

[0084] d1 = 7.3, d2 = 6.8, d3 = 5.5, d4 = 5.9;

[0085] The expansion of each sub-expression is as follows:

[0086] For the first term where j = 1:

[0087]

[0088] The corrected direction angle α1 + the above formula:

[0089] 52.0 + 0.5995 = 52.5995;

[0090] Calculate the remaining three terms in turn:

[0091] The second term:

[0092]

[0093] The third term:

[0094]

[0095] The fourth term:

[0096]

[0097] Substitute the corrected direction angles into the summation:

[0098] 52.5995 + 55.9305 + 61.7288 + 64.713 = 234.972;

[0099] Take the average value:

[0100]

[0101] Substitute into the original formula:

[0102] θ i = |58.4 - 58.743| = 0.343;

[0103] This result indicates that the corrected value of the difference degree between the current vector direction angle and the directions of the remaining vectors in the same interval under the weight, distance, and average offset correction is 0.343. This value is an important direction difference offset parameter for judging the consistency of the thermal direction distribution and plays a key role in the sequence sorting logic of generating the thermal direction sequence path set.

[0104] S204: Call the end coordinate points of the paths in the thermal direction sequence path set, match and pair them with the component package geometric position model, verify the component package positions associated with the endpoints of each heat conduction path, and generate a thermal trajectory grid map with reference to the path connectivity relationship and component position points;

[0105] After obtaining the thermal direction sequence path set, focus on the end coordinate points of each path, that is, the termination points where heat conduction reaches equilibrium or the low-temperature region. Call the coordinate points and perform spatial matching with the known component package position model. This model includes the standard geometric positions, external dimensions, and package type identifiers of each electronic component on the PCBA board. Call the geometric mapping module to convert the end coordinate points in the thermal map to the PCB board layout coordinate system, compare the package contour position information in the component database, find adjacent or overlapping component areas, and realize the pairing and recognition of the path endpoints with the actual components. If the path endpoint is located near the center of components such as capacitors, diodes, or main chips, the path can be bound to the component. After the path pairing is completed, establish an overall heat transfer conduction structure diagram based on the spatial connectivity, overlapping areas, and heat conduction sequence between the paths, and perform topological reconstruction in combination with the station image information. Use the coordinate points as nodes and the heat conduction paths as edges to display the heat conduction process and distribution structure on the entire PCBA board and generate a thermal trajectory grid map.

[0106] The specific steps for obtaining the solder joint boundary complement area are as follows:

[0107] S301: Call the thermal path node coordinates marked in the thermal trajectory grid map, perform projection mapping on the thermal path coordinates according to the image frame coordinate system, map them to the homologous coordinate area of the corresponding visible light image frame, evaluate the correspondence between the thermal map nodes and the visible image pixel points, and generate the thermal map mapping pixel distribution;

[0108] Extract the thermal path coordinate information of each node as the input data for subsequent image projection. The thermal path node coordinates are the positions marked in the thermal imaging image frame, so its coordinate system is the internal coordinate system of the thermal map frame. To achieve synchronous processing with the visible light image, the coordinates need to be mapped and projected according to the calibration parameters during the image acquisition process, and converted into the coordinate area of the visible light image frame. This mapping process calls the dual-mode imaging calibration parameter table of the camera, combines information such as the relative installation angle, focal length parameter, and imaging resolution ratio between the thermal imaging module and the visible light module, and maps each thermal node coordinate to the corresponding position in the visible light image frame. Centered on the projected node position in the visible light image, evaluate the gray scale and color characteristics of the pixel points and surrounding pixels where it is located, analyze the coverage relationship of the thermal node in the image pixel grid, provide data support for subsequent multi-modal image fusion, and ensure that the thermal path node can be accurately located in the visible image, generating the pixel distribution of the thermal map mapping.

[0109] S302: Based on the pixel intensity gradient within the covered area in the pixel distribution of the thermal map mapping, detect the intensity change in the blurred area of the image, then perform overlapping coordinate judgment on the boundary of the blurred area and the boundary of the thermal map path mapping, extract the intersection area of the two, and generate the solder joint overlapping boundary coordinate set;

[0110] After completing the pixel mapping distribution of the thermal map in the visible light image, perform image sharpness analysis on the covered area, extract the gray scale values of the pixel points within the thermal map mapping area, and then calculate the gray scale change gradient of its surrounding pixels. By analyzing the gradient distribution, judge the clarity of the image in this area. When the amplitude of pixel intensity change is small and continuous, this area can be determined as a blurred area. Scan the mapping area with a blurred detection template and mark the blurred boundary area. At the same time, compare the blurred boundary with the thermal map mapping boundary, perform overlapping coordinate judgment on each pair of boundaries, that is, check whether there is a spatially coincident or intersecting area between the two boundaries. If there is an intersection area, extract the overlapping coordinate points, record the actual overlapping pixel range between the area where the thermal path is located and the blurred area. This data is very crucial for image completion processing, can effectively identify the solder joint contour missing area caused by imaging blur, provide accurate boundary input for subsequent completion operations, ensure that the image contour repair has a spatial basis, and generate the solder joint overlapping boundary coordinate set.

[0111] S303: Call the boundary vertex positions in the solder joint overlapping boundary coordinate set, calculate the alignment difference value with the standard symmetric boundary range of the corresponding solder joint in the solder joint structure model of the PCBA board, adjust the contour of the solder joint area in the real-time image frame, perform contour interpolation and completion and closed reconstruction on the missing boundary area, and generate the solder joint boundary completion area;

[0112] After obtaining the set of solder joint overlap boundary coordinates, each boundary vertex coordinate therein needs to be compared with the solder joint structure model of the PCBA board. The standard geometric boundaries of each solder joint are recorded in this model, especially the boundary range coordinate values of the symmetric contour. Suppose a standard solder joint is defined as a left-right symmetric structure with a clear rectangular or elliptical boundary. Through the image coordinate alignment mechanism, the overlapping boundary points in the real-time image frame are aligned with the model boundary, and the boundary position difference between the two is calculated. The comparison result is used to determine whether there is a missing or offset in the solder joint boundary in the current image frame. If a discontinuous or broken boundary area is detected, it enters the complementation processing stage, and the image reconstruction module is called to perform edge interpolation processing on the missing area. The interpolation logic is based on the trend of the existing boundary points, and the contour curve of the missing area is inferred according to the shape constraint of the standard contour structure. During the interpolation process, a reasonable curve is automatically filled between the breakpoints and the two ends are smoothly connected, and a closed contour reconstruction operation is performed on this area. This area can be used for subsequent dimension measurement, defect analysis or automatic detection comparison to generate a solder joint boundary complementation area.

[0113] The specific steps for obtaining the defect determination node set are as follows:

[0114] S401: Call the spatial boundary coordinates marked in the solder joint boundary complementation area, extract the hue data in the corresponding area in multiple consecutive frames of images, calculate the RGB pixel energy value of each frame of image, separate the RGB pixels of the area in each frame of image, and generate a solder joint channel energy sequence set;

[0115] The formula for calculating the RGB pixel energy value of each frame of image is as follows:

[0116]

[0117] Among them, E RGB (t) represents the pixel energy value of the RGB channels in each frame of image, and R a (t), G a (t), B a (t) respectively represent the RGB values of the red, green, and blue channels of the a-th pixel point in the t-th frame of image, respectively represent the average RGB values of each channel, and N represents the number of pixel points in the image;

[0118] Parameter meaning and formula calculation derivation process:

[0119] RGB value R a (t), G a (t), B a (t):

[0120] Obtain the pixel values of the image through an image processing software or programming library (such as OpenCV). For each pixel point, the R of the a-th pixela (t), G a (t) and B a (t) represent the intensity values of the pixel in the red, green, and blue channels in the image respectively. The RGB values range from 0 to 255, and the resolution of the image is related to the number N of pixels to be processed;

[0121] Average RGB value

[0122] The average value of each channel can be calculated by summing the channel values of the pixels and then dividing by the number N of pixels. The formula is:

[0123]

[0124] Suppose the resolution of an image is 4×4 (i.e., there are 16 pixels), and the RGB values of each pixel are as follows (simplified to only one RGB value per pixel):

[0125] For the first-frame image, the RGB values of some pixels are:

[0126] (R1(1), G1(1), B1(1)) = (120, 130, 140);

[0127] (R2(1), G2(1), B2(1)) = (125, 135, 145);

[0128] (R3(1), G3(1), B3(1)) = (130, 140, 150);

[0129] For the 16 pixels in this frame of image, calculate the average values of their red, green, and blue channels (assuming the calculation has been obtained):

[0130]

[0131] Substitute each parameter into the formula:

[0132]

[0133] For the first pixel, the calculation is:

[0134] (R1(1) - 130) 2 = (120 - 130) 2 = (-10) 2 = 100;

[0135] (G1(1) - 140) 2 = (130 - 140) 2 = (-10) 2 = 100;

[0136] (B1(1)-150) 2 =(140 - 150) 2 =(-10) 2 = 100;

[0137] Therefore, the contribution of the first pixel is:

[0138] 100 + 100 + 100 = 300;

[0139] Similarly, calculate for each pixel point in turn,

[0140] Sum and take the square root to obtain the RGB pixel energy value E of the entire image RGB (1), Set the value obtained by summation to 4800, then:

[0141]

[0142] The result indicates that the RGB pixel energy value of the first frame image is 69.28. The energy distribution of the color channels in the image can be further analyzed. This result is an important indicator for measuring the change of color channels in the image and can provide a basis for tasks such as color recognition of solder joint areas and welding quality assessment.

[0143] S402: Based on the RGB energy distribution values of each frame in the solder joint channel energy sequence set, calculate the unit vector direction of the main hue axis of each frame, and perform an interval judgment on the offset angle value of the frame with respect to the offset threshold. If the offset angle is greater than the offset threshold, mark the corresponding frame as an abnormal state frame, record the corresponding number of the abnormal state frame image, and generate a set of defect determination nodes;

[0144] It is necessary to further analyze the changing trend of the dominant hue of the RGB energy distribution in each frame of the image. For each frame of the image, compare the relative proportion relationship of the energy values of the three channels to determine the main hue direction of this frame, that is, which channel dominates the color performance of the current image. In order to quantify this main hue direction, generate a unit vector to describe the color direction according to the channel combination relationship of the RGB channel energy values of each frame, and record the offset angle of this unit vector relative to the reference hue direction. Under normal circumstances, the solder joint area should mainly show silver-white or bright metallic color, and its main hue direction should stably point to a specific interval of channel intensity ratio. Extract the angle difference between the main hue direction of each frame and the reference direction, and perform an interval comparison and judgment with the preset offset angle threshold. If the angle offset exceeds this threshold, it indicates that the color in the solder joint area of this frame of the image has changed abnormally. At this time, mark this frame as an abnormal state frame, add its frame number to the abnormal image record table, summarize the corresponding numbers of the frame images determined to be in the abnormal state, which are used as defect determination nodes for subsequent image review and defect confirmation, and are used to support the automatic image screening and abnormal data prompt functions, and generate a set of defect determination nodes.

[0145] The steps for obtaining the defective path structure of the PCBA board are specifically as follows:

[0146] S501: Invoke the number information of the abnormal state nodes in the defective judgment node set, track the position of the node in the path segment belonging to the image recognition path. When there are multiple path branch segments at the node trigger position, record the node sequence within the path segment and the connection relationship with adjacent nodes, evaluate the relationship between the response frequency of the path branch segment and the structure, and form an abnormal path structure mark;

[0147] It is necessary to further perform a structured path analysis on each abnormal state node, invoke the frame number information of each abnormal node in the set, and map the numbers back to the path recognition record table in the image recognition module. In this table, each node has a unique path segment belonging and sequence position in the image path structure. According to the node number, track its complete path segment in the hot path, find out which segment of the heat flow propagation path the node belongs to, and determine whether there is a branch structure in this segment of the path, that is, whether it is connected to path nodes in multiple directions. If it is detected that there are two or more subsequent connection points for this node, mark the path segment where the node is located as a path branch segment. After identifying the path branch structure, extract the node sequence within this path segment, and record the connection relationship between them in the order of the spatial coordinates of the nodes in the image. At the same time, analyze the frequency of repeated occurrences of the connection nodes in multiple image frames. By statistically analyzing the response frequency of the branch segment nodes, evaluate its stability. If most nodes in the path branch segment frequently appear in an abnormal state or the connection direction changes violently, mark this path segment as an abnormal path structure, and record the structure position, start and end node numbers, branch quantity, and connection relationship, providing a basis for spatial structure and parameter indexing for subsequent data repair operations, and forming an abnormal path structure mark.

[0148] S502: Based on the position of the path branch segment marked in the abnormal path structure mark, correspondingly search for the light source trigger sequence value, heat mapping reference frame number value, and hue direction vector value recorded in the original path nodes, use the parameters of adjacent non-abnormal path segments as the replacement source, and complete the information reset in the node index order to generate the defective path structure of the PCBA board;

[0149] Structural repair processing needs to be performed on the abnormal path segments. For each path segment marked as abnormal, extract the corresponding parameter items in the original hot path node record, including the light source trigger sequence value of each node in the path segment, the reference frame number value of the thermal image frame, and the recorded hue direction vector value. After extracting the three items of information corresponding to the abnormal node, compare them with the path segments that are spatially adjacent but not marked as abnormal. For each abnormal node, find nodes with similar structures and stable channel energy changes in the non-abnormal path segments adjacent to its left and right as replacement reference sources, extract the three original parameters of the reference node and use them as the information supplement source for the current abnormal node. Complete the parameter replacement operation one by one according to the index order of the nodes in the abnormal path segment, that is, sequentially overwrite the light source trigger sequence value, replace the thermal map reference frame number, and update the hue direction vector value, so that this segment of the path is restored to a state of complete structure and continuous data. After all abnormal node information is replaced and it is verified that there is no cross-node disconnection problem, incorporate the repaired path segment back into the hot path network, mark it as the repaired state, and merge it into the overall path data of the PCBA board for subsequent evaluation of the thermal flow integrity and defect correlation in the solder joint area, and generate the defective path structure of the PCBA board.

[0150] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or replacements, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A multi-station PCBA board detection method based on machine vision, characterized in that It includes the following steps: S1: Obtain the multi-station reference pulse period, divide the period into multiple time window segments, extract the starting point of the corresponding allocated time window as the starting reference point of the light source pulse for this station, compare the current light source start time with the real-time capture start time of the previous image frame. If the time difference exceeds the specified tolerance range, control the image acquisition to trigger with a delay and generate a multi-station synchronization control signal; S2: Based on the multi-station synchronization control signal, perform local imaging on the welded PCBA board, extract the temperature response coordinate points within the pin solder joint area, sort them serially according to the included angle in the fan-shaped expansion direction, and generate a thermal trajectory grid map; S3: Use the thermal trajectory grid map to map the thermal grid map onto the corresponding visible light image frame, identify the overlapping boundary area between the blurred area of the visible image and the thermal path, and use the overlapping area as the completion area to generate a solder joint boundary completion area; S4: Utilize the solder joint boundary completion area to extract the hue change path of the solder joint area in multiple consecutive frames of the image, separate the pixel energies of the RGB three channels in each frame of the image, and compare the direction offsets of the hue main axis change vectors in each frame to generate a set of defect determination nodes.

2. The multi-station PCBA board detection method based on machine vision according to claim 1, wherein The multi-station synchronization control signal includes a reference trigger point, a tolerance deviation value, and a delay trigger amount. The thermal trajectory grid map includes thermal response coordinate points, a heat conduction direction vector, and component package pairing coordinates. The solder joint boundary completion area includes an overlapping area range, solder joint symmetry parameters, and solder joint integrity indicators. The set of defect determination nodes includes an abnormal frame number, a hue change offset value, and an abnormal determination threshold.

3. The multi-station PCBA board detection method based on machine vision according to claim 1, wherein, The specific steps for obtaining the multi-station synchronization control signal are as follows: S101: Obtain the multi-station reference pulse period, evenly divide the pulse period into multiple time window segments, set corresponding start time points and end time points for the time windows, locate the allocated time window segment according to the number of the real-time station, and record the start time point of the time window as the reference point for starting the light source pulse for this station to generate a time window start reference value; S102: Call the time window start reference value and the real-time capture start time value of the previous image frame, perform a subtraction operation on the two to calculate the time difference, and determine whether the time difference falls within the set range based on the light source trigger offset tolerance interval. If it exceeds the interval, mark the light source trigger status as delayed and generate a trigger delay mark value; S103: Set the delay parameter for the real-time frame image acquisition operation according to the trigger delay mark value. When there is a delay mark, place the delay parameter value in the offset correction interval and postpone the trigger time. After the time difference re-enters the light source trigger offset tolerance interval, generate a multi-station synchronization control signal.

4. The multi-station PCBA board detection method based on machine vision according to claim 3, characterized in that, The specific steps for obtaining the thermal trajectory grid map are as follows: S201: Based on the multi-station synchronization control signal, call the image positioning parameters and station information of the welded PCBA board, perform a local thermal imaging task on the specified area, capture the thermal image frame and locate multiple temperature response coordinate points within the pin solder joint area to generate a solder joint temperature coordinate set; S202: Call the temperature response value of the coordinate points in the solder joint temperature coordinate set, detect the temperature gradient change trend in the surrounding neighborhood grid points, and refer to the temperature difference between the real-time coordinate points and the neighborhood grid points to merge the vectors of the coordinate points into the main cooling direction, generating a solder joint cooling direction vector group; S203: According to the direction angle value of each vector in the solder joint cooling direction vector group, set a fixed-angle sector division interval, classify the vectors according to the direction angle into the corresponding sequences, calculate the direction difference offset value, and perform number sorting according to the angle sequence of the sector interval, generating a heat direction sequence path set; S204: Call the path end coordinate points of the heat direction sequence path set and perform matching and pairing with the component package geometric position model, verify the component package positions associated with the end points of each heat conduction path, and refer to the path connection relationship and the component position points to generate a heat trajectory grid map.

5. The multi-station PCBA board detection method based on machine vision according to claim 4, wherein, The formula for calculating the direction difference offset value is as follows: Among them, θ i represents the direction difference offset value of the i-th vector, α i represents the direction angle value of the i-th vector, n i represents the total number of vectors in the same sector interval as α i , α j represents the direction angle value of the j-th vector, w j represents the heat intensity weight of the j-th vector, d j represents the distance from the welding point corresponding to the j-th vector to the heat source center, represents the average value of the vector direction angle values within the entire sector interval, j represents the vector index number in the same sector interval as i, and i represents the vector index number.

6. The multi-station PCBA board detection method based on machine vision according to claim 4, wherein, The specific steps for obtaining the solder joint boundary complemented area are as follows: S301: Call the heat path node coordinates marked in the heat trajectory grid map, perform projection mapping on the heat path coordinates according to the image frame coordinate system, map them to the homologous coordinate area of the corresponding visible light image frame, evaluate the corresponding relationship between the heat map nodes and the visible image pixel points, and generate a heat map mapping pixel distribution; S302: Based on the pixel intensity gradient in the area covered by the heat map mapping pixel distribution, detect the intensity change in the image blur area, and then perform an overlapping coordinate judgment on the boundary of the blur area and the heat map path mapping boundary, extract the intersection area of the two, and generate a solder joint overlapping boundary coordinate set; S303: Call the boundary vertex positions in the solder joint overlapping boundary coordinate set, calculate the alignment difference value with the standard symmetric boundary range of the corresponding solder joint in the PCBA board solder joint structure model, adjust the solder joint area contour in the real-time image frame, perform contour interpolation and complement for the missing boundary area and close and reconstruct it, generating a solder joint boundary complemented area.

7. The method for detecting multi-station PCBA boards based on machine vision according to claim 6, characterized in that, The specific steps for obtaining the defective judgment node set are as follows: S401: Call the spatial boundary coordinates marked in the solder joint boundary complemented area, extract the hue data in the corresponding area in consecutive multiple frames of images, calculate the RGB pixel energy value of each frame of image, perform channel separation on the RGB pixels in the area of each frame of image, and generate a solder joint channel energy sequence set; S402: Based on the RGB energy distribution value of each frame in the solder joint channel energy sequence set, calculate the unit vector direction of the main hue axis of each frame, perform an interval judgment on the offset angle value of the frame with the offset threshold. If the offset angle is greater than the offset threshold, mark the corresponding frame as an abnormal state frame, and record the corresponding number of the abnormal state frame image, generating a defective judgment node set.

8. The multi-station PCBA board detection method based on machine vision according to claim 7, wherein The formula for calculating the RGB pixel energy value of each frame of image is as follows: Among them, E RGB (t) represents the pixel energy value of the RGB channels in each frame of the image, R a (t), G a (t), B a (t) respectively represent the RGB values of the red, green, and blue channels of the a-th pixel in the t-th frame of the image, respectively represent the average RGB values of each channel, and N represents the number of pixels in the image.

9. The method for detecting multi-station PCBA boards based on machine vision according to claim 1, wherein, The method further includes step S5: S5: Through the set of defect determination nodes, when a branch segment in the path of a defect determination node in image recognition is triggered, record the response frequency of the path nodes and the front and back path structures, record the information of the abnormal path segment, replace the light source trigger order, thermal mapping reference frame number, and hue direction vector information in the nodes corresponding to the abnormal segment, and generate the PCBA board defect path structure; The PCBA board defect path structure includes the abnormal node response frequency, path structure map, trigger order correction parameter, thermal mapping reference frame serial number, and hue direction correction vector.

10. The multi-station PCBA board detection method based on machine vision according to claim 9, wherein, The specific steps for obtaining the PCBA board defect path structure are as follows: S501: Call the number information of the abnormal state nodes in the set of defect determination nodes, track the position of the nodes in the path segment to which they belong in the image recognition path. When there are multiple path branch segments at the node trigger position, record the node sequence within the path segment and the connection relationship between adjacent nodes, and evaluate the relationship between the response frequency and structure of the path branch segment to form an abnormal path structure mark; S502: Based on the position of the path branch segment marked in the abnormal path structure mark, correspondingly search for the light source trigger order value, thermal mapping reference frame number value, and hue direction vector value recorded in the original path nodes, use the parameters of adjacent non-abnormal path segments as the replacement source, and complete the information reset in the node index order to generate the PCBA board defect path structure.

Images