Area array structured light projection height information detection method and device

Abstract

The application discloses a kind of plane array structured light projection height information detection methods, comprising the following steps: the object to be measured is placed in the field of view of camera, and the measured surface has groove structure;First, second projection light source is set, the line of first projection light source and camera, the line of second projection light source and camera is 45-135 ° angle;Start first projection light source, and its projection reconstructs first height information matrix, and the multiple reflection phenomenon exists for region one, and the rest is region two;Close first projection light source, and start second projection light source, and its projection reconstructs second height information matrix, and the multiple reflection phenomenon exists for region two, and the rest is region one;Set target height information matrix as third height information matrix, and the height information in region one is obtained from second height information matrix, and the height information in region two is obtained from first height information matrix.The application further discloses a kind of plane array structured light projection height information detection device.The device of the application is simple, and detection speed is fast.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional scanning technology, and in particular relates to a method and device for detecting height information of structured light projection in a planar array. Background Technology

[0002] 3D scanning technology is widely used in fields such as human-computer interaction, object recognition, facial recognition, product quality inspection, and 3D modeling. Classified by the basic physical principles of the measurement methods, they mainly include triangulation, time-of-flight measurement, focusing height measurement, confocal measurement, and interferometry.

[0003] Structured light technology can be categorized into point structured light, line structured light, and area structured light based on the dimension of its projected light. Since point and line structured light often utilize laser light sources in practice, they are commonly referred to as point laser and line laser sensors, respectively, both of which are widely used in industrial online inspection. Point laser sensors are frequently used to measure the height of a single point. Line laser sensors, on the other hand, are often used to measure the contours of an object's cross-section, and are therefore also known as two-dimensional contour sensors.

[0004] Structured light technology using area arrays is a popular 3D scanning technology in recent years. Compared with point structured light and line structured light, its advantage is that it can obtain the height information of a surface without the need for mechanical motion scanning, and the system integration requirements are low.

[0005] Most existing structured light sensors use a monocular approach, consisting of a light source emitter and a receiver, with the receiver and emitter on the same horizontal axis, as shown in the schematic diagram. Figure 1 , Figure 2 As shown.

[0006] During projection, the projection light source projects a coded pattern onto the surface being measured. After the camera receives the coded image modulated by the height of the measured surface, it decodes the image to obtain the height of the object's surface, such as... Figure 3 As shown in the diagram. This architecture allows the receiver to receive information returned from the measured surface and reconstruct the height information for most diffuse reflective surfaces, such as... Figure 4 As shown. However, because projection is used in scenarios with specular reflection, blocked light paths, and multiple reflections, the receiver cannot effectively obtain the grayscale information returned by the measured surface for decoding, and the height reconstruction will fail.

[0007] like Figure 5-1 As shown, when specular reflection exists, most of the incident light from the projector is not reflected to the camera. In this case, the grayscale value of the pixel is too low, resulting in decoding errors and an incorrect height value being output. When most of the incident light from the projector is directly reflected to the camera, the grayscale value of the pixel is 255, leading to decoding errors and an incorrect height value being output.

[0008] like Figure 5-2 As shown, when the light path is blocked, the light emitted by the projector is reflected off the object's surface and cannot enter the camera due to the obstruction of the step, resulting in no valid information and no height information at this location. Alternatively, the light emitted by the projector cannot enter the recess, or the light entering the recess is reflected off the surface and cannot enter the camera, resulting in no valid information and no height information at this location.

[0009] like Figure 5-3 As shown, when multiple reflections exist, the light rays that are directly emitted by the projection light to point A of the groove and the light rays that are emitted by the projection light to point B of the groove reach point A again after one or more reflections. Both of these light rays are received by the camera after being reflected by point A and are judged to be the modulation information of point A, which causes the camera to be unable to correctly demodulate the correct height of the point.

[0010] For objects with grooved structures exhibiting multiple reflections, the only solution is to adjust the projection angle to prevent or reduce these reflections. Otherwise, if multiple beams overlap, it becomes impossible to demodulate the correct height. Currently, this problem is typically solved using multiple projectors, usually four. These four projectors project from different angles, ensuring that at least two projectors in each area will not produce multiple reflections, which is determined using the proximity method. However, more projectors mean higher costs and more time spent taking pictures; each additional projector doubles the number of pictures and the time required. Summary of the Invention

[0011] To overcome the shortcomings of existing technologies, this invention provides a method and apparatus for detecting height information of area array structured light projection, which is simple in structure, fast in detection speed, and effective for scanning objects with multiple reflection phenomena.

[0012] The technical solution adopted by this invention to solve its technical problem is: a method for detecting the projection height information of a structured light array, comprising the following steps:

[0013] The object to be tested is placed within the field of view of the camera, and the surface of the object to be tested has a groove structure;

[0014] A first projection light source and a second projection light source are set at the horizontal height of the camera, and the line connecting the first projection light source and the camera and the line connecting the second projection light source and the camera form an angle of 45-135°.

[0015] The first projection light source is activated, and the projection of the first projection light source reconstructs the first height information matrix. At this time, the area with multiple reflections is region one, and the area without multiple reflections is region two.

[0016] The first projection light source is turned off, and the second projection light source is turned on. The projection of the second projection light source reconstructs the second height information matrix. At this time, the region with multiple reflections is region two, and the region without multiple reflections is region one.

[0017] Furthermore, Region 1 and Region 2 constitute a complete region;

[0018] The target altitude information matrix is ​​set as the third altitude information matrix. Its altitude information in region one is obtained from the second altitude information matrix, and its altitude information in region two is obtained from the first altitude information matrix.

[0019] Furthermore, there is a transition region between region one and region two, whose height information is obtained by a weighted average of the first height information matrix and the second height information matrix.

[0020] Furthermore, the diagonal lines of the camera's field of view form an X-shaped dividing line, with two regions on the same axis as the first projection light source forming region one, and two regions on the same axis as the second projection light source forming region two.

[0021] Furthermore, the two diagonal lines rotate 0-20° clockwise and counterclockwise around the center of the field of view to form the aforementioned transition region.

[0022] Furthermore, the included angle is 90°.

[0023] Furthermore, the groove structure can be a closed annular groove or an open annular groove.

[0024] The present invention also discloses a device for detecting the height information of a structured light projection array, comprising:

[0025] A camera is positioned above the surface of the object to be measured, which has a groove structure and is divided into at least region one and region two.

[0026] The first projection light source is located at the same horizontal level as the camera.

[0027] The second projection light source is located at the same horizontal height as the camera, and the line connecting the first projection light source and the camera, and the line connecting the second projection light source and the camera form an angle of 45-135°.

[0028] The control unit controls the activation and deactivation of the first and second projection light sources.

[0029] The output unit is used to output the first altitude information matrix, the second altitude information matrix, and the target third altitude information matrix;

[0030] When the first projection light source is turned on and the second projection light source is turned off, the output unit outputs the first height information matrix reconstructed from the projection of the first projection light source. At this time, the region with multiple reflection phenomena is region one.

[0031] When the second projection light source is turned on and the first projection light source is turned off, the output unit outputs the second height information matrix reconstructed by the projection of the second projection light source. At this time, the region with multiple reflections is region two.

[0032] The target's third altitude information matrix obtains its altitude information in region one from the second altitude information matrix and its altitude information in region two from the first altitude information matrix.

[0033] Furthermore, the included angle is 90°.

[0034] Furthermore, a transition region is formed between region one and region two, and its height information is obtained by a weighted average of the first height information matrix and the second height information matrix.

[0035] Furthermore, the lines connecting the first projection light source and the camera, and the lines connecting the second projection light source and the camera, form an angle of 75-105°.

[0036] This invention proposes a novel method for fusing a structured light source architecture and algorithm called "dual projection," which can achieve three-dimensional reconstruction of the measured surface based on the original monocular array structured light, and has better technical indicators and a wider range of application scenarios.

[0037] The beneficial effects of this invention are: 1) The device is simple. For the structure of the annular groove, four projectors are usually required to ensure that at least two projectors in the multiple reflection area do not have multiple reflections, that is, to ensure that the height information they obtain is correct. However, the height information in the multiple reflection area varies greatly. Therefore, the height value can be judged by two height values ​​that are close to each other as the correct value, and the height value that is not close to each other as the abnormal value (i.e., multiple reflections exist). However, the multiple reflection areas of the two projectors at 180° are overlapping, that is, neither of them has the correct height value in the area, and the correct height of the area cannot be detected. This invention solves the above problem by using two mutually perpendicular projectors; 2) Compared with the existing methods that need to calculate each pixel, the detection speed of this invention is fast; 3) The device cost is reduced. Attached Figure Description

[0038] Figure 1 This is a top view of a prior art system where there is one light source emitter and the receiver and light source emitter are on the same horizontal axis.

[0039] Figure 2 This is a front view of a prior art system where there is one light source emitter and the receiver and light source emitter are on the same horizontal axis.

[0040] Figure 3This is a schematic diagram of image processing in the prior art, where there is one light source emitter and the receiver and light source emitter are on the same horizontal axis.

[0041] Figure 4 This is a schematic diagram of a scenario in the prior art where there is one light source emitter and the receiver and light source emitter are on the same horizontal axis.

[0042] Figure 5-1 This is a schematic diagram of the existing technology when specular reflection exists.

[0043] Figure 5-2 This is a schematic diagram of the situation where the optical path is blocked in the prior art.

[0044] Figure 5-3 This is a schematic diagram of the situation where multiple reflections exist in the prior art.

[0045] Figure 6 This is a schematic diagram of the actual surface being measured in this invention.

[0046] Figure 7 This is a three-dimensional structural diagram of the detection device of the present invention.

[0047] Figure 8 This is a front view of the detection device of the present invention.

[0048] Figure 9 This is a top view of the detection device of the present invention.

[0049] Figure 10 This is a schematic diagram of the main view structure of the detection device of the present invention when the first projection light source is turned on.

[0050] Figure 11 This is a three-dimensional structural diagram of the detection device of the present invention when the first projection light source is turned on.

[0051] Figure 12 This is a simplified optical path diagram of the detection device of the present invention when the first projection light source is turned on.

[0052] Figure 13 This is a diagram of the actual state of the detection device of the present invention when the first projection light source is turned on and the second projection light source is turned off.

[0053] Figure 14 This is a schematic diagram of the imaging of the test surface when the first projection light source of the detection device of the present invention is turned on and the second projection light source is turned off.

[0054] Figure 15 This is a schematic diagram of the concentrated area of ​​multiple reflections in the groove structure of the surface to be tested in this invention. Figure 1 .

[0055] Figure 16 This is a schematic diagram of the main view structure of the detection device of the present invention when the second projection light source is turned on.

[0056] Figure 17 This is a three-dimensional structural diagram of the detection device of the present invention when the second projection light source is turned on.

[0057] Figure 18 This is a simplified optical path diagram of the detection device of the present invention when the second projection light source is turned on.

[0058] Figure 19 This is a diagram of the actual state of the detection device of the present invention when the first projection light source is off and the second projection light source is on.

[0059] Figure 20 This is a schematic diagram of the imaging of the test surface when the first projection light source of the detection device of the present invention is turned off and the second projection light source is turned on.

[0060] Figure 21 This is a schematic diagram of the concentrated area of ​​multiple reflections in the groove structure of the surface to be tested in this invention. Figure 2 .

[0061] Figure 22 This is a schematic diagram of the region division for obtaining height information in the third height information matrix M3 of this invention.

[0062] Figure 23 This is a schematic diagram of the height information acquisition area division of the third height information matrix M3 in this invention, where there is a transition area.

[0063] Wherein, 1-camera, 2-surface under test, 21-region one, 22-region two, 23-groove structure, 24-transition region, 3-first projection light source, 4-second projection light source. Detailed Implementation

[0064] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0065] like Figure 7-9 As shown, a device for detecting the height information of a structured light projection array includes:

[0066] Camera 1 is positioned above the surface 2 of the object to be measured. The surface 2 has a groove structure 23. In other words, the surface 2 has multiple reflections and is divided into at least region 1 21 and region 2 22.

[0067] The first projection light source 3 is set at the same horizontal height as the camera 1;

[0068] The second projection light source 4 is set at the same horizontal height as the camera 1, and the line connecting the first projection light source 3 and the camera 1 and the line connecting the second projection light source 4 and the camera 1 form an angle of 45-135°, preferably 75-105°.

[0069] The control unit controls the opening and closing of the first projection light source 3 and the second projection light source 4;

[0070] The output unit is used to output the first altitude information matrix M1, the second altitude information matrix M2, and the target third altitude information matrix M3.

[0071] When the first projection light source 3 is turned on and the second projection light source 4 is turned off, the output unit outputs the first height information matrix M1 reconstructed from the projection of the first projection light source 3. The region with multiple reflections is defined as region 1 21 and the region without multiple reflections is defined as region 22. Region 1 21 and region 22 constitute a complete region.

[0072] When the second projection light source 4 is turned on and the first projection light source 3 is turned off, the output unit outputs the second height information matrix M2 reconstructed from the projection of the second projection light source 4. Region 22 is defined as the area with multiple reflections and region 21 is defined as the area without multiple reflections. Region 21 and region 22 constitute a complete region. The target third height information matrix M3 obtains its height information in region 21 from the second height information matrix M2 and its height information in region 22 from the first height information matrix M1. Thus, the output unit can output the target third height information matrix M3.

[0073] In this invention, the surface to be measured 2 has a groove structure 23, which can be a closed annular groove or an open annular groove. In this embodiment, a closed annular groove is used as an example for explanation. Figure 6 As shown, it is a closed annular groove, therefore multiple reflections may exist between the two opposite walls of the groove. Preferably, the second projection light source 4 is perpendicular to the first projection light source 3; more specifically, the line connecting the first projection light source 3 and the camera 1, and the line connecting the second projection light source 4 and the camera 1, form a 90° angle. Figure 9 In the middle, α is 90°.

[0074] A method for detecting the height information of a structured light projection array, using the aforementioned detection device, includes the following steps:

[0075] The object to be tested is placed within the field of view of the camera 1. The surface 2 of the object to be tested has a groove structure 23. In other words, the surface 2 of the object to be tested has multiple reflections.

[0076] A first projection light source 3 and a second projection light source 4 are set at the horizontal height of the camera 1, and the line connecting the first projection light source 3 and the camera 1 and the line connecting the second projection light source 4 and the camera 1 form an angle of 45-135°, preferably 75-105°.

[0077] In this embodiment, the included angle is 90°, that is Figure 9 In the middle, α is 90°.

[0078] The first projection light source 3 is activated, and the projection of the first projection light source 3 reconstructs the first height information matrix M1. At this time, the region with multiple reflections is region 1 21, and the region without multiple reflections is region 22. Region 1 21 and region 2 22 constitute a complete region.

[0079] Specifically, such as Figure 10 , Figure 11 As shown, the first projection light source 3 is turned on, and the second projection light source 4 is turned off; the actual object is as follows. Figure 13 As shown, the first projection light source 3 is on, and the second projection light source 4 is off. The positions of the first projection light source 3 and the second projection light source 4 are relative to... Figure 10 , Figure 11 It is not a perfect match, but it is clear that the area facing the first projection light source 3 has a higher brightness.

[0080] The image of the surface to be tested, 2, is as follows: Figure 14 As shown, the positional relationship and height information matrix of the first projection light source 3 and the second projection light source 4 at this time are as follows: Figure 12 As shown, Figure 15 As shown, the multiple reflection phenomenon is concentrated in the groove structure 23 within region 21.

[0081] The second projection light source 4 is activated, and the projection of the second projection light source 4 reconstructs the second height information matrix M2. At this time, the region with multiple reflections is region 22, and the region without multiple reflections is region 1 21. Region 1 21 and region 2 22 constitute a complete region.

[0082] Specifically, such as Figure 16 , Figure 17 As shown, the second projection light source 4 is turned on, and the first projection light source 3 is turned off; the actual object condition at this time is as follows. Figure 19 As shown, the first projection light source 3 is off, and the second projection light source 4 is on. The positions of the first projection light source 3 and the second projection light source 4 are relative to... Figure 16 , Figure 17 It is not a perfect match, but it is clear that the area facing the second projection light source 4 is brighter.

[0083] The image of the surface to be tested, 2, is as follows: Figure 20 As shown, the positional relationship and height information matrix of the first projection light source 3 and the second projection light source 4 at this time are as follows: Figure 18 As shown, Figure 21 As shown, the multiple reflection phenomenon is concentrated in the groove structure 23 within region 22.

[0084] If the target height information matrix is ​​set to the third height information matrix M3, then its height information in region 1 (21) is obtained from the second height information matrix M2, and its height information in region 2 (22) is obtained from the first height information matrix M1.

[0085] Specifically, such as Figure 22 As shown, the final target height information matrix is ​​set as the third height information matrix M3. Referring to the area division of the annular groove structure 23, four rays are drawn from the center point of the matrix to divide the matrix into four areas, namely area 02, area 04, area 06, and area 08.

[0086] The four rays mentioned above also form an X-shaped dividing line along the diagonal lines of the camera's field of view. The two regions located on the same axis as the first projection light source 3 are region 1 21, and the two regions located on the same axis as the second projection light source 4 are region 22.

[0087] Therefore, regions 02 and 06 are the aforementioned region 1 21, and regions 04 and 08 are the aforementioned region 2 22. Thus, the values ​​of the third altitude information matrix M3 in regions 02 and 06 are given by the values ​​of the second altitude information matrix M2; the values ​​of the third altitude information matrix M3 in regions 04 and 08 are given by the values ​​of the first altitude information matrix M1.

[0088] To ensure more accurate values ​​between adjacent regions, a transition region 24 is provided between region 1 (21) and region 2 (22). The height information of this transition region 24 is obtained by weighted averaging of the first height information matrix M1 and the second height information matrix M2.

[0089] Specifically, such as Figure 23 As shown, the two diagonal lines rotate 0-20° clockwise and counterclockwise around the center of the field of view, respectively. Figure 23 The mid-β angle is 0-20°, thus forming the aforementioned transition region 24.

[0090] Therefore, regions 01, 03, 05, and 07 constitute the aforementioned transition region 24. Consequently, the value of the third altitude information matrix M3 within transition region 24 is given by the weighted average of the values ​​of the first altitude information matrix M1 and the second altitude information matrix M2. This ultimately yields a complete regional third altitude information matrix M3.

[0091] The above specific embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for detecting height information by projecting structured light on an area array, characterized in that, The method comprises the following steps: placing a test object in the field of view of a camera, the test surface of the test object having a groove structure; setting a first projection light source and a second projection light source at the horizontal level of the camera, and the line connecting the first projection light source and the camera and the line connecting the second projection light source and the camera form an angle of 90°; two diagonal lines of the field of view of the camera form X-shaped boundary lines, which divide the field of view of the camera into four regions, two regions on the same axis as the first projection light source are region one, and two regions on the same axis as the second projection light source are region two, and region one and region two constitute a complete region; the two diagonal lines are rotated clockwise and counterclockwise around the center of the field of view by 0-20°, forming a transition region; starting the first projection light source, the projection of the first projection light source reconstructs a first height information matrix, at this time, multiple reflection phenomena exist in region one, and multiple reflection phenomena do not exist in region two; turning off the first projection light source and starting the second projection light source, the projection of the second projection light source reconstructs a second height information matrix, at this time, multiple reflection phenomena exist in region two, and multiple reflection phenomena do not exist in region one; setting the target height information matrix as a third height information matrix, the height information in region one is obtained from the second height information matrix, and the height information in region two is obtained from the first height information matrix; the height information of the transition region is obtained by weighted average of the first height information matrix and the second height information matrix.

2. The area structured light projection height information detection method according to claim 1, characterized in that: The groove structure is a closed ring-shaped groove or an open ring-shaped groove.

3. An area array structured light projection height information detection apparatus applying the area array structured light projection height information detection method according to claim 1, characterized by The method comprises: a camera arranged above the test surface of a test object, the test surface having a groove structure; a first projection light source arranged at the horizontal level of the camera; a second projection light source arranged at the horizontal level of the camera, the line connecting the first projection light source and the camera and the line connecting the second projection light source and the camera form an angle of 90°; a control unit for controlling the opening and closing of the first projection light source and the second projection light source; an output unit for outputting a first height information matrix, a second height information matrix, and a target third height information matrix; two diagonal lines of the field of view of the camera form X-shaped boundary lines, which divide the field of view of the camera into four regions, two regions on the same axis as the first projection light source are region one, and two regions on the same axis as the second projection light source are region two, and region one and region two constitute a complete region; the two diagonal lines are rotated clockwise and counterclockwise around the center of the field of view by 0-20°, forming a transition region; when the first projection light source is turned on and the second projection light source is turned off, the output unit outputs the first height information matrix reconstructed by the projection of the first projection light source, at this time, multiple reflection phenomena exist in region one, and multiple reflection phenomena do not exist in region two; when the second projection light source is turned on and the first projection light source is turned off, the output unit outputs the second height information matrix reconstructed by the projection of the second projection light source, at this time, multiple reflection phenomena exist in region two, and multiple reflection phenomena do not exist in region one; the target third height information matrix obtains the height information in region one from the second height information matrix and the height information in region two from the first height information matrix; The height information of the transition region is obtained by weighted average of the first height information matrix and the second height information matrix.

Images