A system and method for detecting and repairing a placement

Abstract

The application provides a mounting detection repair system and method, which adopts a digital signal processing method to amplify and identify two-dimensional image signals. A small vibration is given to a to-be-detected circuit board in a to-be-detected area, video acquisition and transmission are performed by a high-speed camera, two-dimensional images are preprocessed, aligned and cancelled, corresponding points on residual features are tracked, and the tracking is converted into one-dimensional signals for coding and filtering, so that target components are identified. Finally, a multi-degree-of-freedom mechanical arm is used for re-mounting and welding of the components. The application does not need manual participation, can accurately detect whether the mounting of the components is qualified, reduces the missed detection rate and labor cost, improves the production test efficiency, and reduces waste.

Description

Technical Field

[0001] This invention relates to a mounting inspection and repair system and method. Background Technology

[0002] During surface mount technology (SMT) manufacturing, components may not be properly mounted on the circuit board after assembly due to stencil data discrepancies or solder paste quality issues. This can lead to components falling off during handling after reflow. However, ordinary visual inspection cannot detect this problem, and repairs typically require manual resoldering after the circuit board has cooled in the reflow oven. This method not only easily results in component loss but also wastes labor costs. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a mounting inspection and repair system and method to address the shortcomings of the prior art.

[0004] The system includes a transmission module, a vibration detection board module, an image acquisition module, a calculation and processing module, and a repair module;

[0005] The transmission module includes a conveyor belt for transporting the circuit boards to be inspected and repaired, which have been cooled in the furnace, to the inspection board vibration module for further processing.

[0006] The detection board vibration module includes a vibrating table and a displacement sensor, which are used to fix the board and monitor the displacement distance generated by the vibration of the vibrating table.

[0007] The image acquisition module is used to acquire images of the board moving at high speed during vibration, and sends the acquired image data to the calculation and processing module for further processing.

[0008] The calculation and processing module is used to identify poorly mounted components to be repaired and control the repair module to perform the repair.

[0009] The repair module includes a robotic arm with a welding function, used for re-mounting and welding defective components.

[0010] The vibration table is equipped with a structural mold customized according to the shape of the board for fixing the board. The vibration table can apply a small vibration with 2 degrees of freedom in the X direction to the board (the coordinate system here adopts the coordinate system of the corresponding vibration table).

[0011] The displacement sensor is installed on the outer wall of the vibration table in the X direction to monitor the displacement distance generated by the vibration of the vibration table and send it to the calculation and processing module for analysis.

[0012] The image acquisition module includes a high-speed camera and a matching data transmission cable.

[0013] The computing and processing module includes two or more high-performance algorithm processors.

[0014] The present invention also provides a mounting inspection and repair method, comprising the following steps:

[0015] Step 1: The test plate, after being cooled in the furnace, is transported by conveyor belt to a vibration table that can generate a specific vibration frequency, and the test plate is fixed by a structural mold.

[0016] Step 2: Set the vibration frequency in different directions on the vibration table so that the components of the test plate can respond to the vibration.

[0017] Step 3: The vibration table applies vibrations at specific frequencies in the X and Y directions to the test plate, and a high-speed camera takes real-time pictures within the vibration time T.

[0018] Step 4: Within the vibration time T, acquire N consecutive images at different times using a high-speed camera and send them to the high-performance algorithm processor. Simultaneously send the vibration displacement distance generated by the vibration table of the load, as monitored by the displacement sensor within the vibration time T, to the high-performance algorithm processor in real time.

[0019] Step 5: The high-performance algorithm processor preprocesses all the acquired images and uses the preprocessed images as the original images for displacement analysis of the test plate.

[0020] Step 6: Align the original image, shift the second frame of the two adjacent frames in the opposite direction by the inter-frame shift distance and superimpose it with the first frame, cancel the overlapping part, and extract the remaining image features.

[0021] Step 7: Based on the corresponding points on the remaining image features extracted in Step 6 (the extracted features are the parts that are inconsistent with the amplitude and frequency of the board, i.e., the image features of defective components containing noise. Ideally, the obvious outline features of the defective components can be seen, but at this time there are many noise points, so it is necessary to reduce noise by canceling the vibration frequency measured by the sensor, and then identify the final result; the concept of a corresponding point is that there is a set of images, i.e., the same point on the image features under different time images, which is the corresponding point of the image feature. For example, the southwest corner point of a defective component. This point is in a different position on different images, but the point on each image is the corresponding point of this point), an amplitude-time (AT) signal graph is generated and canceled with the vibration frequency measured by the displacement sensor to identify the defective components.

[0022] Step 8: Mark the identified defective components and re-attach and solder them using a multi-degree-of-freedom welding robotic arm.

[0023] In step 2, the vibration frequency in the X direction is set for the vibration table, and the vibration equation in the X direction is:

[0024]

[0025] Where A is the vibration amplitude, x is the vibration amplitude in the X direction at time t, and ω is the vibration frequency. For phase.

[0026] In step 3, the full-frame acquisition speed of the high-speed camera 1020 is F. full Vibration time Where L represents the board length and W represents the board width.

[0027] In step 5, the high-performance algorithm processor first converts the color image to grayscale, retaining only the image brightness information, and then performs image filtering.

[0028] Step 6 includes: calculating the image gradient by the difference between each pixel in its neighborhood; obtaining the board frame edge information by high-pass filtering; aligning the top left corner points of the board frames in adjacent frames to achieve coarse alignment between the two frames; and obtaining the coordinates of the remaining three corner points of the previous frame as (x, y, y). f1 ,y f1 ), (x f2 ,y f2 ), (x f3 ,y f3 The coordinates of the remaining three corner points in the next frame are (x) and (x) respectively. p1 ,y p1 ), (x p2 ,y p2 ), (x p3 ,y p3 ), calculate the Euclidean distances between corresponding corner points in the X and Y directions respectively, and find the maximum value L of the Euclidean distance in the X direction. x The maximum value L of the Euclidean distance in the Y direction y Set the existing coordinates of the next frame image as the center point, 2L x ×2L y The size is a neighborhood. The pixels of the entire image are shifted by the number of any pixel in any direction within the neighborhood. The gray values ​​of each corresponding pixel in the shifted image are canceled out with the gray values ​​of the corresponding pixels in the previous frame. The gray values ​​of each pixel in the resulting new image are summed. The result of each summation is recorded. The displacement of the smallest summation result is the fine alignment of the image. The overlapping parts of the fine alignment are canceled out to obtain the remaining image features.

[0029] Step 7 includes: based on the corresponding points on the remaining image features extracted in Step 6, during the entire vibration time period, using the displacement distance of the corresponding points on the remaining image features as the vertical axis and time as the horizontal axis, to create a CA-T signal map (C stands for contrast, meaning to be compared, i.e., the amplitude-time (AT) signal map to be compared), the number of the corresponding point is used as the encoding information and put into the CA-T signal map of the corresponding point for frequency modulation encoding, and the signals of the CA-T signal maps of all points that have completed frequency modulation encoding are superimposed;

[0030] Using the vibration displacement distance of the test plate measured by the displacement sensor in step 4 as the vertical axis and time as the horizontal axis, a BA-T signal diagram of the test plate is made (B is base, meaning reference, i.e., reference amplitude-time (AT) signal diagram). The amplitude of the BA-T signal diagram is reversed, and frequency modulation encoding is performed according to the corresponding point number.

[0031] By superimposing the CA-T signal diagram and the BA-T signal diagram, signals with the same frequency and phase but opposite amplitudes cancel each other out. For the remaining signal in the superimposed signal diagram, the signal frequency is decoded to obtain the key points that are inconsistent with the vibration frequency of the board under test. Combined with the original image, the poorly mounted components are identified.

[0032] This invention employs digital signal processing and image processing methods. By applying a small vibration at a specific frequency to the circuit board in the area to be detected, video is captured and transmitted via a high-speed camera. This process enables preprocessing, alignment, and cancellation of the two-dimensional image. The corresponding points on the remaining features are then tracked and converted into one-dimensional signals for encoding and filtering, thereby identifying the target component.

[0033] Beneficial effects: Existing solutions typically require manually resoldering components after the circuit board has cooled in a reflow oven for repair. This method not only easily leads to component loss but also wastes labor costs. This invention eliminates the need for manual intervention, accurately detects component placement defects, reduces the rate of missed detections and labor costs, improves production testing efficiency, and minimizes waste. Attached Figure Description

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0035] Figure 1 Block diagram of the mounting inspection and repair system of the present invention.

[0036] Figure 2 A flowchart of the mounting, inspection, and repair method of this invention. Detailed Implementation

[0037] like Figure 1 As shown, this invention discloses a mounting inspection and repair system 10, including a transmission module 100, a detection board vibration module 101, an image acquisition module 102, a calculation and processing module 103, and a repair module 104. This invention selects the mounting inspection and repair of artificial intelligence boards used in the field of edge computing as an embodiment of the invention to illustrate the application process of the device.

[0038] The transmission module 100 includes a conveyor belt 1000, which is used to transport the AI ​​board to be tested and repaired, which has been cooled in the furnace, to the test board vibration module 101 for subsequent processing.

[0039] The vibration detection module 101 includes a vibration table 1010 (preferably an aerospace-grade Hill inductive electric vibration table) and a displacement sensor 1011 (preferably an OMEGA GP901). The vibration table 1010 is equipped with a structural mold customized according to the board's shape for fixing the board. The vibration table can apply minute vibrations in the X direction with two degrees of freedom to the board. The displacement sensor 1011 is installed on the outer wall of the vibration table in the X direction to monitor the displacement distance generated by the vibration of the vibration table and send it to the calculation and processing module 103 for analysis.

[0040] The image acquisition module 102 includes a high-speed camera 1020 (preferably Senyun SG808BAF) and a matching data transmission cable, used to acquire high-speed motion images of the board during vibration, and send the acquired image data to the calculation and processing module 103 for subsequent processing.

[0041] The aforementioned computing and processing module 103 includes multiple high-performance algorithm processors 1030 (preferably Ascend Atlas 200). In one embodiment of the present invention, the computing and processing module 103 receives target images acquired by the high-speed camera 1020 for running image processing and signal processing algorithms, identifies defective components to be repaired, and controls the repair module 104 to perform repairs.

[0042] The repair module 104 includes a welding-functional robotic arm 1040 (preferably a modified AUBO-i5 version) for re-mounting and welding defective components.

[0043] The transmission module 100 uses a conveyor belt 1000 to move the circuit board to be inspected and repaired to the acquisition range of the image acquisition module 102. The circuit board is fixed by a structural mold customized according to the shape of the board, installed on the vibration table 1010 of the board vibration detection module 101. The vibration table 1010 applies a small vibration at a specific frequency to the board. The high-speed camera 1020 in the image acquisition module 102 takes frame-by-frame pictures of the circuit board and transmits the images to the high-performance processor 1030 of the calculation and processing module 103 for image processing. During the vibration, the board displacement data recorded by the displacement sensor 1011 is also transmitted synchronously to the high-performance processor 1030 for calculation. The selected high-performance processor 1030 has the computing power of complex algorithm models and can realize image processing and digital signal processing. The location coordinates of the identified defective components are sent to the welding function robotic arm 1040 in the repair module 104 for re-mounting and welding.

[0044] This invention also provides a method for mounting inspection and repair. Combined with... Figure 2 The method includes the following steps:

[0045] Step 1: In an embodiment of the present invention, the AI ​​board for edge computing after being mounted is passed through a reflow oven and cooled, and then transported by conveyor belt 1000 to a vibrating table 1010 that can generate a specific vibration mode, and the board is fixed by a structural mold.

[0046] Step 2: Set the vibration frequency in the X direction for the vibration table 1010. The vibration equation in the X direction is as follows: In an embodiment of the present invention, the plate length L and the plate weight m are taken as... The amplitude of the vibration is ω, which is 900 × m. for To ensure that poorly mounted components do not fall off due to excessive vibration amplitude or frequency, the vibration equation is determined according to the parameters mentioned above. This ensures that the test board fixed in step 1 and the custom mold on the vibration table 1010 are completely fitted, allowing the test board to respond to vibration promptly and maintain consistency with the vibration frequency.

[0047] Step 3: Based on the vibration frequency set in Step 2, the vibration table 1010 applies vibration in the X direction to the AI ​​board under test, and the high-speed camera 1020 captures real-time images during the vibration time; in this embodiment of the invention, the full-frame acquisition speed of the high-speed camera 1020 is F. full Plate length L, plate width W, vibration time To ensure that the vibration time is sufficient to obtain the number of image frames required for image processing, and to maximize detection efficiency, the vibration time is determined according to the parameters mentioned above.

[0048] Step 4: In this embodiment of the invention, during the vibration time T in step 3, high-definition images of the board at different consecutive moments within this time are acquired by a high-speed camera 1020 and sent to a high-performance algorithm processor 1030 (preferably Ascend Atlas 200). The vibration displacement distance of the vibration table 1010 monitored during the vibration time T is synchronously and in real-time sent to the high-performance algorithm processor 1030 via a displacement sensor 1011 installed on the outer wall of the vibration table 1010.

[0049] Step 5: The high-performance algorithm processor 1030 preprocesses all the images acquired in Step 4. First, the color images are converted to grayscale, retaining only the image brightness information. Image filtering is then applied. In this embodiment of the invention, the image filtering method is as follows: within a 3x3 neighborhood, the nine grayscale values ​​are sorted in ascending order, the maximum and minimum values ​​are removed, and the remaining values ​​are weighted and averaged. The weight of the middle point is set to 0.4, and the weight of the other points is set to 0.1. The formula is as follows, where the nine grayscale values ​​are represented by a matrix within the 3x3 neighborhood. Arrange the 9 grayscale values ​​in ascending order to obtain x. (1) ,x (2) ,...,x (9) Discarding the maximum and minimum values, the final weighted average yields the grayscale value. This is the grayscale value of the pixel after filtering. The filtering method used in this invention can retain the original image information to the greatest extent and eliminate the interference of high-frequency noise on the image. The preprocessed image is used as the original image for displacement analysis of the test plate;

[0050] Step 6: By aligning the preprocessed images from Step 5, the image of the following frame in two adjacent frames is shifted in the opposite direction by the inter-frame shift distance and superimposed on the image of the preceding frame. The overlapping parts are canceled out, and the remaining features are extracted. In this embodiment of the invention, the image gradient is obtained by calculating the difference between each point in its neighborhood. High-pass filtering is used to obtain the board frame edge information. The upper left corner points of the board frames in two adjacent frames are aligned accordingly, resulting in coarse alignment of the two frames. The coordinates of the remaining three corner points of the preceding frame are obtained as (x...). f1 ,y f1 ), (x f2 ,y f2 ), (x f3 ,y f3 The coordinates of the remaining three corner points in the next frame are (x) and (x) respectively. p1 ,y p1 ), (x p2 ,y p2 ), (x p3 ,y p3), calculate the Euclidean distances between corresponding corner points in the X and Y directions respectively, and find the maximum value L of the Euclidean distance in the X direction. x The maximum value L of the Euclidean distance in the Y direction y Set the existing coordinates of the next frame image as the center point, 2L x ×2L y The size is a neighborhood. The pixels of the entire image are shifted by the number of pixels in any direction within the neighborhood. The grayscale values ​​of each corresponding pixel in the shifted image are canceled out by the grayscale values ​​of the corresponding pixels in the previous frame. The grayscale values ​​of each pixel in the resulting new image are summed, and the result of each summation is recorded. The displacement of the smallest summation result is the fine alignment of the image. The overlapping parts of the fine alignment are canceled out to obtain the remaining image features. The remaining image features include the partial contour information of unmounted components on the AI ​​board and noise information introduced during acquisition and processing.

[0051] Step 7: In an embodiment of the present invention, based on the corresponding points on the remaining image features extracted in Step 6, a CA-T signal graph is generated throughout the entire vibration time period, using the displacement distance of the corresponding points on the remaining image features as the ordinate and time as the abscissa, to obtain f. c (t), the corresponding points are numbered according to spatial information to obtain the number k. i,j i and j are the numbers k i,j The coordinates of the corresponding point are represented by the number, which contains the location information of the corresponding point. The modulation signal is s. c (t)=A c cos[2πk i, j f n +p∫f c [(τ)dτ],A c f c The amplitude, f nThe reference frequency is determined based on the range and resolution of the spectrum analyzer used for measurement; in this embodiment, it is 10MHz, which is also the interval frequency of the frequency modulation signal. p is the modulation coefficient, which is 3 in this embodiment. The corresponding point number is used as encoding information and placed into the CA-T signal diagram of the corresponding point for frequency modulation encoding. The signals in the CA-T signal diagrams of all points that have completed frequency modulation encoding are superimposed to generate a detection spectrum diagram containing the number information. The vibration displacement distance of the board under test measured by the displacement sensor 1011 in step 4 is used as the vertical axis, and time as the horizontal axis to create a BA-T signal diagram of the board under test. The amplitude of the BA-T signal diagram is reversed, and frequency modulation encoding is performed again based on the corresponding point number in the CA-T signal diagram to generate a reference spectrum diagram containing the number information. The detection spectrum diagram and the reference spectrum diagram are superimposed and filtered. Signals with the same frequency and phase but opposite amplitudes can cancel each other out. The generated new spectrum diagram can be decoded according to the signal frequency to obtain the key point number information that is inconsistent with the vibration frequency of the board under test. The key points are then restored using spatial information to form an image containing the key points. Combined with the original image, faulty components on the artificial intelligence board are identified.

[0052] Step 8: Mark the defective components identified on the AI ​​board, generate defective component detection results with spatial information, and use the multi-degree-of-freedom welding function robotic arm 1040 to re-attach and weld the defective components at the corresponding locations.

[0053] This invention provides a mounting inspection and repair system and method. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for inspection and repair of mounting components, characterized in that, Includes the following steps: Step 1: The test plate, after being cooled in the furnace, is transported by conveyor belt to a vibration table that can generate a specific vibration frequency, and the test plate is fixed by a structural mold. Step 2: Set the vibration frequency in different directions on the vibration table so that the components of the test plate can respond to the vibration. Step 3: The vibration table applies vibrations at specific frequencies in the X and Y directions to the test plate, and a high-speed camera takes real-time pictures within the vibration time T. Step 4: Within the vibration time T, acquire N consecutive images at different times using a high-speed camera and send them to the algorithm processor. The vibration displacement distance generated by the vibrating table, monitored by the internal displacement sensor, is synchronously sent to the algorithm processor in real time. Step 5: The algorithm processor preprocesses all the acquired images and uses the preprocessed images as the original images for displacement analysis of the plate under test. Step 6: Align the original image, shift the second frame of the two adjacent frames in the opposite direction by the inter-frame shift distance and superimpose it with the first frame, cancel the overlapping part, and extract the remaining image features. Step 7: Based on the corresponding points on the remaining image features extracted in Step 6, create an amplitude-time signal graph and cancel it out with the vibration frequency measured by the displacement sensor to identify poorly mounted components. Step 7 includes: based on the corresponding points on the remaining image features extracted in Step 6, using the displacement distance of the corresponding points on the remaining image features as the vertical axis and time as the horizontal axis throughout the entire vibration time period, a CA-T signal map is created; the number of the corresponding point is used as the encoding information and placed into the CA-T signal map of the corresponding point for frequency modulation encoding; and the signals in the CA-T signal maps of all points that have completed frequency modulation encoding are superimposed. Using the vibration displacement distance of the test plate measured by the displacement sensor in step 4 as the vertical axis and time as the horizontal axis, a BA-T signal diagram of the test plate is created. The amplitude of the BA-T signal diagram is reversed, and frequency modulation encoding is performed according to the corresponding point number. By superimposing the CA-T signal diagram and the BA-T signal diagram, signals with the same frequency and phase but opposite amplitudes cancel each other out. For the remaining signal in the superimposed signal diagram, the signal frequency is decoded to obtain the key points that are inconsistent with the vibration frequency of the board under test. Combined with the original image, the poorly mounted components are identified. Step 8: Mark the identified defective components and re-attach and solder them using a multi-degree-of-freedom welding robotic arm.

2. The method according to claim 1, characterized in that, In step 2, the vibration frequency in the X direction is set for the vibration table, and the vibration equation in the X direction is: ， Where A is the vibration amplitude. Let be the amplitude of the vibration in the X direction at time t. The vibration frequency, For phase.

3. The method according to claim 2, characterized in that, In step 3, the full-frame acquisition speed of the high-speed camera 1020 is: Vibration time Where L represents the board length and W represents the board width.

4. The method according to claim 3, characterized in that, In step 5, the algorithm processor first converts the color image to grayscale, retaining only the image brightness information, and then performs image filtering.

5. The method according to claim 4, characterized in that, Step 6 includes: calculating the image gradient by the difference between each pixel in its neighborhood; obtaining the board frame edge information by high-pass filtering; aligning the top left corner points of the board frames in adjacent frames to achieve coarse alignment between the two frames; and obtaining the coordinates of the remaining three corner points of the previous frame. , , The coordinates of the remaining three corner points in the next frame are respectively , , Calculate the Euclidean distances between corresponding corner points in the X and Y directions, and find the maximum value of the Euclidean distance in the X direction. Maximum value of the Euclidean distance in the Y direction Set the existing coordinates of the next frame image as the center point. The size is a neighborhood. The pixels of the entire image are shifted by the number of any pixel in any direction within the neighborhood. The gray values ​​of each corresponding pixel in the shifted image are canceled out with the gray values ​​of the corresponding pixels in the previous frame. The gray values ​​of each pixel in the resulting new image are summed. The result of each summation is recorded. The displacement of the smallest summation result is the fine alignment of the image. The overlapping parts of the fine alignment are canceled out to obtain the remaining image features.

6. A mounting inspection and repair system for implementing the method as described in any one of claims 1 to 5, characterized in that, It includes a transmission module, a vibration detection board module, an image acquisition module, a calculation and processing module, and a repair module; The transmission module includes a conveyor belt for transporting the circuit boards to be inspected and repaired, which have been cooled in the furnace, to the inspection board vibration module for further processing. The detection board vibration module includes a vibrating table and a displacement sensor, which are used to fix the board and monitor the displacement distance generated by the vibration of the vibrating table. The image acquisition module is used to acquire images of the high-speed movement of the board during vibration, and sends the acquired image data to the calculation and processing module for further processing; The calculation and processing module is used to identify poorly mounted components to be repaired and control the repair module to perform the repair. The repair module includes a robotic arm with a welding function, used for re-mounting and welding defective components.

7. The system according to claim 6, characterized in that, The vibration table is equipped with a structural mold customized according to the shape of the board for fixing the board. The vibration table can apply a small vibration with 2 degrees of freedom in the X direction to the board. The displacement sensor is installed on the outer wall of the vibration table in the X direction to monitor the displacement distance generated by the vibration of the vibration table and send it to the calculation and processing module for analysis.

8. The system according to claim 7, characterized in that, The image acquisition module includes a high-speed camera and a matching data transmission cable.

9. The system according to claim 8, characterized in that, The computation and processing module includes two or more algorithm processors.

Images