Structured light imaging method and system suitable for pin needle detection in new energy vehicles

Abstract

The application discloses a structured light imaging method and system suitable for Pin needle detection in a new energy automobile. The method focuses on the surface of a circuit board and the top of a Pin needle in turn by driving a structured light imaging system, and projects a group of stripe images for three-dimensional reconstruction and a group of positive and negative stripe images with opposite gray scales at each focusing position respectively; a gray scale difference image at each focusing position is calculated based on the positive and negative stripe images; the gray scale difference images at different focusing positions are compared pixel by pixel, corresponding height map data is adaptively selected and fused, and finally a high-precision fused height map is generated. The application uses gray scale difference comparison in the time domain to replace the traditional spatial domain definition calculation, fundamentally overcomes the edge noise and optical dispersion circle interference problems, and significantly improves the precision and reliability of Pin needle height and inclination measurement.

Description

Technical Field

[0001] This invention relates to the field of pin height detection technology in IGBT modules and domain controllers of new energy vehicles, and particularly to a structured light imaging method and system suitable for pin detection in new energy vehicles. Background Technology

[0002] Pins in new energy domain controllers and IGBT modules are common electrical connection devices, usually used to connect two circuit boards. This method is widely used in industry because it is convenient; you only need to align one circuit board with the hole on the other and insert it backwards.

[0003] A simplified diagram of pins on a circuit board is shown in Figures 1(a) and 1(b). The circuit board has components with lower heights and pins with higher heights. However, the insertion process of the pins themselves needs to be controlled. Inconsistent heights can lead to insufficient electrical connections, and if the pins are tilted, it can cause problems with misalignment with the holes. Therefore, major new energy vehicle manufacturers are looking for a device and method to detect the height and tilt of the pins.

[0004] Figure 2(a) shows the side and top views of a normal pin. Figure 2(b) shows a pin with an abnormal tilt. This tilt has two adverse effects: it makes it impossible to align and insert the pin into subsequent holes, and it also poses a risk of short circuits between pins. Figure 2(c) shows a pin with an abnormal height, which is generally caused by a force control error when the pin is inserted into the circuit board. This situation poses a risk of open circuits after the pin is inserted into subsequent holes.

[0005] Structured light technology is a triangulation technique based on moiré fringes. Its advantages of a large field of view and high speed make it widely used in the testing of domain controllers and IGBT modules in new energy vehicles. Common structured light systems include... Figure 3 As shown, it typically consists of a camera, a lens, and multiple DLPs. The test relies on the DLPs to project encoded patterns onto the object, usually as stripe signals. The shape of the stripes deforms depending on the object's height. The camera captures the deformed image, and the algorithm analyzes it to obtain the object's true height. However, the core of this work is that the stripes must be relatively clear, meaning the object must be within the lens's depth of field (the height range the lens can clearly see).

[0006] In new energy IGBTs or domain controllers, the pin height typically ranges from 15-30 mm, far exceeding the depth of field of a lens. Lenses used in the new energy industry usually have a depth of field of around 2 mm. During testing, the lens's focal plane is usually placed on the surface of the circuit board to better visualize components such as resistors and capacitors. However, this causes the top of the pin to become out of focus, preventing structured light from imaging and measuring its height.

[0007] According to publicly available information, a common solution to address insufficient depth of field is to raise the imaging component a second time, focusing it onto the surface of the pin to obtain a clear image of the pin. The two images, before and after the raising, are then fused based on their sharpness. The specific heights before and after the raising are shown in Figures 4(a) and 4(b), where Figure 4(a) shows the image focused on the circuit board surface before raising, and Figure 4(b) shows the image focused on the pin surface after raising.

[0008] However, this approach has the following drawbacks:

[0009] 1. Sharpness calculations can introduce edge noise. Sharpness calculated from an image is achieved using a convolution kernel of a certain size, typically 5x5 or larger. The top area of ​​a pin is usually 5x5 or 10x10 in size, and a 5x5 convolution kernel will result in significant edge errors. For example... Figure 5 As shown, noise will exist at the edges.

[0010] 2. Interference between circles of confusion leads to noise between pins. Interference between optical circles of confusion results in significant blurring in the area between pins. As shown in Figures 6(a) and 6(b), Figure 6(a) shows the area between pins with low blurring when focused on the top of the pin; Figure 6(b) shows the area between pins with high blurring when the pin is out of focus, leading to misjudgment. Summary of the Invention

[0011] The technical problem to be solved by this invention is to design a structured light imaging method and system suitable for pin detection in new energy vehicles. Based on the structured light system being stretched multiple times and imaging fusion according to the projection of positive and negative stripes, the problem of pin imaging is solved, thereby solving the existing technical problems.

[0012] To address the aforementioned technical problems, this invention provides a structured light imaging method suitable for pin detection in new energy vehicles, employing a structured light system for image acquisition, specifically including:

[0013] Adjust the focal plane of the structured light system so that it covers the surface of the circuit board and the top of the pin respectively, and obtain the first focus state and the second focus state accordingly.

[0014] In the first focusing state, a three-dimensional reconstructed stripe pattern is projected through the DLP projection module to generate a first height pattern H1; at the same time, a set of stripes with completely opposite gray levels are sequentially projected through the DLP projection module to obtain stripe patterns F1 and F2 respectively, and the gray level difference between stripe patterns F1 and F2 is calculated.

[0015] In the second focusing state, a three-dimensional reconstructed fringe pattern is projected through the DLP projection module to generate a second height pattern H2. At the same time, another set of stripes with completely opposite gray levels is sequentially projected through the DLP projection module, and fringe patterns F1H and F2H are collected respectively. The gray level difference between fringe patterns F1H and F2H is calculated.

[0016] The first height map H1 and the second height map H2 are merged, and the height is selected from the locations with large differences in grayscale values.

[0017] Furthermore, the formula for calculating grayscale difference in this invention is as follows:

[0018] In the first focusing state, the grayscale difference between stripe pattern F1 and stripe pattern F2 is:

[0019] ;

[0020] In the second focusing state, the grayscale difference between stripe pattern F1H and stripe pattern F2H is:

[0021] ;

[0022] in, Represents the coordinates of a point. This indicates taking the absolute value.

[0023] Furthermore, in this invention, the first height map H1 and the second height map H2 are fused to obtain... Specifically:

[0024] .

[0025] Furthermore, in the first focusing state, the structured light system focuses on the surface of the circuit board, so that the circuit board is within the depth of field of the lens.

[0026] Furthermore, in the second focusing state, the structured light system focuses on the pin surface, so that the top of the pin is within the depth of field of the lens.

[0027] Furthermore, in this invention, the stripes with completely opposite gray levels are black and white stripe patterns. When the same spatial point is in focus in the positive and negative stripe patterns, the gray level difference is significant, and when it is out of focus, the gray level difference tends to be consistent.

[0028] Furthermore, in this invention, the focal plane adjustment method of the structured light system is as follows: first, the focal plane is adjusted to cover the surface of the circuit board (first focusing state), and then the structured light imaging system is pulled up so that the focal plane covers the top of the pin (second focusing state).

[0029] Furthermore, in this invention, the focal plane adjustment method of the structured light system is as follows: first, the focal plane is adjusted to cover the top of the pin (second focusing state), and then the focal plane is lowered to cover the surface of the circuit board by lowering the structured light imaging system (first focusing state).

[0030] Furthermore, in this invention, the structured light system includes a camera, a lens, and multiple DLPs.

[0031] This invention also provides a structured light imaging system suitable for pin detection in new energy vehicles, employing the aforementioned structured light imaging method for pin detection in new energy vehicles, specifically including the following modules:

[0032] The structured light imaging module includes a camera, a lens, and a DLP projection unit. The lens is used to achieve focusing and imaging, and the DLP projection unit is used to project a three-dimensional reconstructed fringe pattern and two sets of positive and negative fringe patterns with completely opposite gray levels.

[0033] The focus adjustment module is used to adjust the height of the structured light imaging module so that the focal plane of the lens covers the surface of the circuit board and the top of the pin respectively, forming a first focus state and a second focus state.

[0034] Data processing module: Receives image data captured by the camera at different focal planes, generates a height map based on the 3D reconstructed fringe map, and calculates the grayscale difference value of different focal planes based on the positive and negative fringe maps;

[0035] Data fusion module: Based on the grayscale differences of different focal planes, the height map is fused to obtain fused height map data.

[0036] The beneficial effects of this invention are:

[0037] This invention presents a structured light imaging method for pin detection in new energy vehicles. It replaces traditional sharpness calculations with projected positive and negative stripes, using temporal grayscale differences to replace spatial domain sharpness calculations. By comparing the absolute grayscale differences of the same location in two consecutive frames of positive and negative stripe images, this invention addresses the issue. This difference is solely related to the defocusing degree of the pixel itself and is completely unaffected by surrounding pixels, thus fundamentally eliminating edge noise and achieving pixel-level precise fusion decision. Furthermore, this invention utilizes the characteristic that the grayscale of positive and negative stripes tends to be consistent in the defocused state. In defocused areas (whether it's the pin itself or the background interfered with by a blur circle), the grayscale difference between the positive and negative stripes becomes very small. Therefore, the system can intelligently identify and eliminate blur circle interference, always selecting the sharpest height map data, greatly improving the purity of the background data between pins. Attached Figure Description

[0038] The specific embodiments of the present invention will be further explained below with reference to the accompanying drawings.

[0039] Figure 1(a) is a schematic diagram of the pins on the circuit board.

[0040] Figure 1(b) is a top view of Figure 1(a).

[0041] Figure 2(a) shows the side and top views of a normal pin.

[0042] Figure 2(b) shows the side and top views of a pin with an abnormal tilt.

[0043] Figure 2(c) shows the side and top views of a pin with an abnormal height.

[0044] Figure 3 This is a schematic diagram of a structured light system.

[0045] Figure 4(a) is a schematic diagram of the height of the structured light system focused on the surface of the circuit board.

[0046] Figure 4(b) is a schematic diagram of the height of the structured light system focused on the surface of the pin.

[0047] Figure 5 This is a schematic diagram of the invalid edge regions caused by the convolution kernel.

[0048] Figure 6(a) is a schematic diagram of focusing the image onto the top of the pin.

[0049] Figure 6(b) is a schematic diagram of the blur circle when the pin needle is out of focus.

[0050] Figure 7(a) shows the positive fringes of the DLP projection (in focus mode).

[0051] Figure 7(b) shows the inverted stripes of the DLP projection (in focus mode).

[0052] Figure 8(a) shows the positive fringes of the DLP projection (in defocus state).

[0053] Figure 8(b) shows the inverted fringes of the DLP projection (in defocus).

[0054] Figure 9 This is a flowchart of the structured light imaging method for pin detection in new energy vehicles, as described in Example 1.

[0055] Figure 10 This is a flowchart of the structured light imaging method for pin detection in new energy vehicles, as described in Example 2.

[0056] Figure 11 This is a block diagram of the structured light imaging system for pin detection in new energy vehicles according to the present invention. Detailed Implementation

[0057] The structured light imaging method for pin detection in new energy vehicles in this embodiment uses a structured light system for image acquisition, specifically including:

[0058] Adjust the focal plane of the structured light system so that it covers the surface of the circuit board and the top of the pin respectively, and obtain the first focus state and the second focus state accordingly.

[0059] In the first focusing state, a three-dimensional reconstructed stripe pattern is projected through the DLP projection module to generate a first height map H1. In this height map, the pin data will have very large noise. At the same time, a set of stripes with completely opposite gray levels are sequentially projected through the DLP projection module, and stripe patterns F1 and F2 are collected respectively. The gray level difference between stripe patterns F1 and F2 is calculated.

[0060] In the second focusing state, a three-dimensional reconstructed fringe pattern is projected through the DLP projection module to generate a second height pattern H2. At the same time, another set of stripes with completely opposite gray levels is sequentially projected through the DLP projection module, and fringe patterns F1H and F2H are acquired respectively. The gray level difference between fringe patterns F1H and F2H is calculated.

[0061] Stripes are a type of structured light, featuring alternating black and white patterns that exhibit strong contrast information. As shown in Figures 7(a) and 7(b), the same point on the positive and negative stripes, let's call it point A1 and point A2, is the same spatial point. And when located at the focal plane, the difference is very obvious. When the positive and negative stripes are out of focus, as shown in Figures 8(a) and 8(b), the gray levels of A1 and A2 are very close. This is because the optical imaging system itself is a low-pass filter; the greater the defocusing, the lower its cutoff frequency, meaning the more blurred the image.

[0062] The first height map H1 and the second height map H2 are merged, and the height is selected from the locations with large differences in grayscale values.

[0063] In this embodiment, the preferred formula for calculating grayscale difference is:

[0064] In the first focusing state, the grayscale difference between stripe pattern F1 and stripe pattern F2 is:

[0065] ;

[0066] In the second focusing state, the grayscale difference between stripe pattern F1H and stripe pattern F2H is:

[0067] ;

[0068] in, Represents the coordinates of a point. This indicates taking the absolute value.

[0069] In this embodiment, preferably, the first height map H1 and the second height map H2 are fused together to obtain... Specifically:

[0070] .

[0071] In this embodiment, preferably, in the first focusing state, the structured light system focuses on the surface of the circuit board, so that the circuit board is within the depth of field of the lens.

[0072] In this embodiment, preferably, in the second focusing state, the structured light system focuses on the pin surface, so that the top of the pin is within the depth of field of the lens.

[0073] In this embodiment, preferably, the stripes with completely opposite gray levels are black and white stripe patterns. When the same spatial point is in focus in the positive and negative stripe patterns, the gray level difference is significant, and when it is out of focus, the gray level difference tends to be consistent.

[0074] In this embodiment, preferably, the focal plane adjustment method of the structured light system is as follows: first, the focal plane is adjusted to cover the surface of the circuit board (first focusing state), and then the structured light imaging system is pulled up so that the focal plane covers the top of the pin (second focusing state).

[0075] In this embodiment, preferably, the focal plane adjustment method of the structured light system is as follows: first, the focal plane is adjusted to cover the top of the pin (second focusing state), and then the structured light imaging system is lowered to cover the surface of the circuit board (first focusing state).

[0076] In this embodiment, preferably, the structured light system includes a camera, a lens, and multiple DLPs.

[0077] Example 1

[0078] Combination Figure 9 As shown, the structured light imaging method for pin detection in new energy vehicles in this embodiment specifically includes the following steps:

[0079] Step 1: Focus the structured light system onto the surface of the circuit board so that the circuit board is within the depth of field of the lens.

[0080] Step 2: Perform DLP projection to obtain the stripe pattern for 3D reconstruction, and calculate the height map H1.

[0081] Step 3: Continue to use DLP to project the front and back stripe patterns F1 and F2.

[0082] Step 4: Raise the structured light system so that its depth of field covers the surface of the pin.

[0083] Step 5: Repeat steps 2 and 3 to obtain height map H2, positive stripe F1H, and negative stripe F2H.

[0084] Step 6: Calculate the grayscale difference images D and DH for different focal planes.

[0085] Step 7: Compare D and DH pixel by pixel. If D is greater than DH, then merge the height maps. Choose H1, or H2 if necessary.

[0086] Example 2

[0087] Combination Figure 10 As shown, the structured light imaging method for pin detection in new energy vehicles in this embodiment specifically includes the following steps:

[0088] Step 1: Focus the structured light system onto the pin surface so that the top of the pin is within the depth of field of the lens.

[0089] Step 2: Perform DLP projection to obtain the stripe pattern for 3D reconstruction, and calculate the height map H2.

[0090] Step 3: Continue to use DLP to project the front and back stripe patterns F1H and F2H.

[0091] Step 4: Lower the height of the structured light system so that its depth of field covers the surface of the circuit board.

[0092] Step 5: Repeat steps 2 and 3 to obtain height map H1, positive stripe F1, and negative stripe F2.

[0093] Step 6: Calculate the grayscale difference images D and DH for different focal planes.

[0094] Step 7: Compare D and DH pixel by pixel. If D is greater than DH, then select H1 for the merged height map Hf; otherwise, select H2.

[0095] Example 3

[0096] Combination Figure 11 As shown, the structured light imaging system for pin detection in new energy vehicles in this embodiment adopts the structured light imaging method for pin detection in new energy vehicles in Embodiment 1 or Embodiment 2, and specifically includes the following modules:

[0097] The structured light imaging module includes a camera, a lens, and a DLP projection unit. The lens is used to achieve focusing and imaging, and the DLP projection unit is used to project a three-dimensional reconstructed fringe pattern and two sets of positive and negative fringe patterns with completely opposite gray levels.

[0098] The focus adjustment module is used to adjust the height of the structured light imaging module so that the focal plane of the lens covers the surface of the circuit board and the top of the pin, respectively, forming a first focus state and a second focus state.

[0099] Data processing module: Receives image data captured by the camera at different focal planes, generates a height map based on the 3D reconstructed fringe map, and calculates the grayscale difference value of different focal planes based on the positive and negative fringe maps.

[0100] Data fusion module: Based on the grayscale differences of different focal planes, the height map is fused to obtain fused height map data.

[0101] Many specific details have been set forth in the foregoing description to provide a thorough understanding of the present invention. However, the above description is merely a preferred embodiment of the present invention, and the present invention can be implemented in many other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed above. Furthermore, any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention, or modify them into equivalent embodiments, using the methods and techniques disclosed above, without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention, without departing from the content of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A structured light imaging method suitable for pin detection in new energy vehicles, characterized in that: Image acquisition is performed using a structured light system, specifically including: Adjust the focal plane of the structured light system so that it covers the surface of the circuit board and the top of the pin respectively, and obtain the first focus state and the second focus state accordingly. In the first focusing state, a three-dimensional reconstructed stripe pattern is projected through the DLP projection module to generate a first height pattern H1; at the same time, a set of stripes with completely opposite gray levels are sequentially projected through the DLP projection module to obtain stripe patterns F1 and F2 respectively, and the gray level difference between stripe patterns F1 and F2 is calculated. In the second focusing state, a three-dimensional reconstructed fringe pattern is projected through the DLP projection module to generate a second height pattern H2. At the same time, another set of stripes with completely opposite gray levels is sequentially projected through the DLP projection module, and fringe patterns F1H and F2H are collected respectively. The gray level difference between fringe patterns F1H and F2H is calculated. The first height map H1 and the second height map H2 are merged, and the height is selected from the locations with large differences in grayscale values. The formula for calculating grayscale difference is: In the first focusing state, the grayscale difference between stripe pattern F1 and stripe pattern F2 is: ； In the second focusing state, the grayscale difference between stripe pattern F1H and stripe pattern F2H is: ； in, Represents the coordinates of a point. Indicates taking the absolute value; The result obtained by fusing the first elevation map H1 and the second elevation map H2 is Specifically: ； In the first focusing state, the structured light system focuses on the surface of the circuit board, so that the circuit board is within the depth of field of the lens; In the second focusing state, the structured light system focuses on the pin surface, so that the top of the pin is within the depth of field of the lens. The stripes with completely opposite gray levels are black and white stripe patterns. When the same point in the same space is in focus in the positive and negative stripe patterns, the gray level difference is significant, and when it is out of focus, the gray level difference tends to be consistent.

2. The structured light imaging method for pin detection in new energy vehicles according to claim 1, characterized in that: The focal plane adjustment method of the structured light system is as follows: first, adjust the focal plane to cover the surface of the circuit board, and then pull up the structured light system so that the focal plane covers the top of the pin.

3. The structured light imaging method for pin detection in new energy vehicles according to claim 1, characterized in that: The focal plane adjustment method of the structured light system is as follows: first, adjust the focal plane to cover the top of the pin, and then lower the structured light system so that the focal plane covers the surface of the circuit board.

4. The structured light imaging method for pin detection in new energy vehicles according to claim 1, characterized in that: The structured light system includes a camera, a lens, and multiple DLPs.

5. A structured light imaging system suitable for pin detection in new energy vehicles, characterized in that: The structured light imaging method for pin detection in new energy vehicles, as described in any one of claims 1-4, specifically includes the following modules: The structured light imaging module includes a camera, a lens, and a DLP projection unit. The lens is used to achieve focusing and imaging, and the DLP projection unit is used to project a three-dimensional reconstructed fringe pattern and two sets of positive and negative fringe patterns with completely opposite gray levels. The focus adjustment module is used to adjust the height of the structured light imaging module so that the focal plane of the lens covers the surface of the circuit board and the top of the pin respectively, forming a first focus state and a second focus state. Data processing module: Receives image data captured by the camera at different focal planes, generates a height map based on the 3D reconstructed fringe map, and calculates the grayscale difference value of different focal planes based on the positive and negative fringe maps; Data fusion module: Based on the grayscale differences of different focal planes, the height map is fused to obtain fused height map data.

Images