3D structured light recognition device, system and method

Abstract

The present application discloses a 3D structured light recognition device, system and method, wherein a 3D structured light recognition device comprises: a transmitting module configured to transmit light to the surface of the object to be measured; a receiving module configured to receive at least part of the light reflected back from the surface of the object to be measured and obtain a recognition image related to the object to be measured; wherein the recognition image comprises a plurality of light spot blocks arranged in an array, and the light spot blocks are staggered in the first direction and staggered in the second direction. The 3D structured light recognition device provided by the embodiment of the present application adjusts the angle setting of the transmitting module and the receiving module so that the arrangement of the light spots in the first direction and the second direction in the obtained recognition image is more random, so that the repeatability of searching for light spots in the first direction and the second direction is reduced. In this way, because the probability of repeatability of the light spots inside the search box is reduced, the probability of the occurrence of the black hole problem in the depth map is also greatly reduced.

Description

Technical Field

[0001] The present application generally relates to the field of 3D technology, and specifically to a 3D structured light recognition device, system and method. Background Art

[0002] 3D modules are increasingly being used in new payment industries. Currently, in the new face-scanning payment industry, 3D structured light modules are widely used due to their high precision, low cost, and low power consumption, becoming the mainstream of current equipment.

[0003] However, in the prior art, when searching images to obtain 3D depth information of the object being measured, the algorithm may encounter repeatability problems when searching for frames. This algorithm cannot clearly distinguish and identify during search calculations, so confusion may occur, resulting in a large difference between the searched position and the reference image, and a black hole problem in the output depth map. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a 3D structured light recognition device, system and method, which can avoid the black hole problem when calculating the 3D depth information of the object under test.

[0005] In a first aspect, the present application provides a 3D structured light recognition device, comprising:

[0006] The transmitting module is configured to transmit light to the surface of the object to be measured;

[0007] A receiving module, configured to receive at least a portion of the light reflected from the surface of the object to be measured and obtain an identification image related to the object to be measured;

[0008] The recognition image includes a plurality of light spot blocks arranged in an array, wherein the light spot blocks are staggered in a first direction and the light spot blocks are staggered in a second direction.

[0009] Optionally, by adjusting the emission light angle of the emission module, the light spot blocks in the recognition image are arranged in an array along a third direction and a fourth direction, the third direction is set at an angle to the first direction, and the fourth direction is set at an angle to the second direction.

[0010] Optionally, the light spot blocks in the recognition image are staggered in the third direction and / or the light spot blocks are staggered in the fourth direction.

[0011] Optionally, by adjusting the shooting angle of the receiving module, the light spot blocks in the recognition image are arranged in an array along a fifth direction and a sixth direction, the fifth direction is set at an angle to the first direction, and the sixth direction is set at an angle to the second direction.

[0012] Optionally, by adjusting the shooting angle of the receiving module and by adjusting the emission light angle of the transmitting module, the light spot blocks in the recognition image are arranged in an array along a seventh direction and an eighth direction, the seventh direction is set at an angle to the first direction, and the eighth direction is set at an angle to the second direction.

[0013] Optionally, an angle between the first direction and the second direction is 90°.

[0014] Optionally, the light spot block includes a plurality of randomly arranged light spots.

[0015] Optionally, the transmitting module includes:

[0016] a light source configured to emit a structured light beam having a random speckle pattern;

[0017] a diffractive optical element, configured to receive the structured light beam and expand the structured light beam into a patterned light beam, and configured to project the patterned light beam onto a surface of an object to be measured;

[0018] The patterned light beam is configured to form a plurality of light spot blocks arranged in an array on the surface of the object to be measured, wherein the light spot blocks are staggered in a first direction and the light spot blocks are staggered in a second direction.

[0019] In a second aspect, the present application provides a 3D structured light recognition device as described in any one of the above and a data processing unit, wherein the data processing unit is configured to obtain the recognition image and construct 3D depth information of the object under test based on the recognition image.

[0020] In a third aspect, the present application provides a 3D structured light recognition method, comprising:

[0021] The transmitting module transmits light to the surface of the object to be measured;

[0022] The receiving module receives the light reflected from the surface of the object to be measured and obtains an identification image related to the object to be measured;

[0023] The data processing unit acquires the recognition image and presets a picture frame of a square structure, wherein the picture frame includes a first side extending along a first direction and a second side extending along a second direction;

[0024] The data processing unit searches the recognition image through the picture frame to obtain 3D depth information of the object under test.

[0025] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0026] The 3D structured light recognition device provided in the embodiment of the present application adjusts the angle settings of the transmitting module and the receiving module so that the arrangement of the light spots in the obtained recognition image along the first direction and the second direction is more random, thereby reducing the repeatability of searching for the light spots in the first direction and the second direction. In this way, since the probability of repeatability of the light spots within the search box is reduced, the probability of the occurrence of black holes in the depth map is also greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0028] Figure 1 A schematic diagram of the principle of 3D structured light recognition provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the principle of picture frame search provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the principle of the search light spot duplication provided in the embodiment of the present application;

[0031] Figure 4 A schematic diagram of the structure of a 3D structured light recognition device provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the structure of another 3D structured light recognition device provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of a light spot emitted by an emission module according to an embodiment of the present application;

[0034] Figure 7 A schematic diagram of a recognition image received by a receiving module provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a shooting angle of a receiving module provided in an embodiment of the present application;

[0036] Fig. 9 A flowchart of a 3D structured light recognition method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] The projection collection of light beams with known spatial directions is called structured light. Figure 1 As shown: a specific wavelength, invisible infrared laser is used as the light source. The emitted light passes through an optical diffraction element (DOE) to form an image with certain coding rules and is projected onto the object. At the same time, the receiving module receives the image with coding rules on the surface of the object, and then calculates the distortion of the returned coding pattern based on the optical triangulation measurement principle to obtain the object's position and depth information.

[0040] That is, the 3D structured light module will burn a reference image on a reference surface at a specific distance, and the reference image will contain some specific information. The image of the actual object surface will be compared with the previously determined reference image. After confirming the difference between the two, it will be converted into a depth map for display after algorithm calculation.

[0041] When comparing the actual captured image with the reference image, a frame is used for comparison within a certain range. After confirming similar coding patterns, a comparison calculation is performed and then a depth map is output. Figure 2 When two pictures are compared, the range of each frame is like the thin-line frame, and the thick-line frame is the range limited by the algorithm. The thin-line frame searches from left to right and from top to bottom inside the thick-line frame, and calculates after finding the one that is most similar to the reference picture.

[0042] There are repetitive problems when searching for frames, such as Figure 3 As shown, in this case, there is no way to clearly distinguish and identify during the search calculation, so there will be confusion in the search, resulting in a large difference between the searched position and the reference image.

[0043] In response to this technical problem, this application makes improvements from the hardware aspect to avoid loopholes in the algorithm search, which may lead to black hole problems in 3D depth information.

[0044] Please see Figure 4 and 5 , the present application provides a 3D structured light recognition device, comprising:

[0045] The emission module 10 is configured to emit light toward the surface of the object to be measured;

[0046] The receiving module 20 is configured to receive at least part of the light reflected from the surface of the object to be measured and obtain an identification image related to the object to be measured;

[0047] The recognition image includes a plurality of light spot blocks arranged in an array, wherein the light spot blocks are staggered in a first direction X1 and the light spot blocks are staggered in a second direction X2.

[0048] The present application achieves a stronger randomness of the light spot in the final identification image related to the object under test by adjusting the transmitting module or the receiving module, thereby avoiding the problem of light spot repeatability.

[0049] It should be noted that in the embodiment of the present application, the first direction X1 is defined as a horizontal direction, the second direction X2 is defined as a vertical direction, and the angle between the first direction X1 and the second direction X2 is 90°. In some other embodiments, the first direction X1 and the second direction X2 can be interchangeable.

[0050] The transmitting module comprises:

[0051] The light source is configured to emit a structured light beam, wherein the structured light beam is a random speckle pattern.

[0052] A diffraction optical element (DOE, full name in English: Diffraction Optical Elements) is configured to receive the structured light beam and expand the structured light beam into a patterned light beam to achieve multi-light path output. The diffraction optical element is also configured to project the patterned light beam onto the surface of the object to be measured.

[0053] The patterned light beam is configured to form a plurality of light spot blocks arranged in an array on the surface of the object to be measured, wherein the light spot blocks are staggered in a first direction X1 and the light spot blocks are staggered in a second direction X2.

[0054] The light source may be an infrared beam, or a VCSEL (Vertical Cavity Surface Emitting Laser), which emits infrared structured light and / or flood light as an object recognition signal through a diffractive optical element.

[0055] In the embodiment of the present application, the light source includes one or more of a high contrast vertical cavity surface emitting laser, a single hole wide face vertical cavity surface emitting laser, an array vertical cavity surface emitting laser, a laser diode, and an LED light source. The emission module may also include optical elements with other functions. When specifically set, the light source may be appropriately adjusted according to different application scenarios, such as a microlens array, a grating, etc.

[0056] In some embodiments, the present application adjusts the transmitting module 10 to rotate it to a certain angle, such as Figure 4As shown, the light emitted by the emission module to the surface of the object to be measured will also rotate at a certain angle. Exemplarily, by adjusting the emission light angle of the emission module, the light spot blocks in the recognition image are arranged in an array along the third direction X3 and the fourth direction X4, the third direction X3 is set at an angle with the first direction X1, and the fourth direction X4 is set at an angle with the second direction X2.

[0057] It should be noted that the angles between the third direction X3 and the fourth direction X4 and the first direction X1 and the second direction X2 are related to the angle of adjustment of the emission module. At the same time, the third direction X3 and the fourth direction X4 are also related to the light source in the emission module and the direction of the light path in the diffractive optical element. The present application does not limit the angles between the third direction X3 and the fourth direction X4 and the first direction X1 and the second direction X2. In specific settings, different adjustments can be made according to different application scenarios.

[0058] In some embodiments, the present application adjusts the specific structure inside the emission module, and the light spot blocks in the recognition image are staggered in the third direction and / or the light spot blocks are staggered in the fourth direction.

[0059] Since the third direction X3 and the fourth direction X4 are also related to the light source in the emission module and the direction of the light path in the diffractive optical element, in this embodiment, the light spot irradiated on the surface of the object to be measured can be directional by adjusting the structure or light path of the optical element. In the embodiment of the present application, this is achieved by adjusting the internal optical structure of the emission module.

[0060] like Figure 6 As shown, in the embodiment of the present application, the light spot emitted by the emitting module is adjusted by adjusting the angle of the emitting module or the structure of the emitting module. Figure 6 In the example, a staggered arrangement of light spots along the third direction and a linear arrangement of light spots along the fourth direction are given. It should be noted that the arrangement of the light spot blocks in the third direction X3 and the fourth direction X4 is not limited in the embodiment of the present application. By staggering or randomly arranging, the light spots in the image received by the receiving module along the first direction X1 and the second direction X2 can be made more random, such as Figure 7 As shown, the picture frame can be clearly distinguished when searching and calculating, thereby preventing the occurrence of confusion in the search, which may cause a large difference between the searched position and the reference image.

[0061] In some embodiments, the present application adjusts the receiving module 20, such as Figure 5As shown, the image is rotated at a certain angle, so that the shooting angle of the receiving module also rotates at a certain angle. By adjusting the shooting angle of the receiving module, the light spot blocks in the recognition image are arranged in an array along the fifth direction X5 and the sixth direction X6, the fifth direction X5 is set at an angle with the first direction X1, and the sixth direction X6 is set at an angle with the second direction X2.

[0062] It should be noted that the angles between the fifth direction X5 and the sixth direction X6 and the first direction X1 and the second direction X2 are related to the angle of adjustment of the receiving module. At the same time, the fifth direction X5 and the sixth direction X6 are also related to the light source in the transmitting module and the light path direction in the diffractive optical element. The present application does not limit the angles between the fifth direction X5 and the sixth direction X6 and the first direction X1 and the second direction X2. In specific settings, different adjustments can be made according to different application scenarios.

[0063] like Figure 8 As shown, in the embodiment of the present application, the light spot condition on the surface of the object to be measured is obtained by adjusting the shooting angle of the receiving module. Figure 8 In the example, a staggered arrangement of light spot images along the fifth direction X5 and a linear arrangement of light spot images along the sixth direction X6 are given. It should be noted that the arrangement of the light spot blocks in the fifth direction X5 and the sixth direction X6 is not limited in the embodiment of the present application. By staggering or randomly arranging, the light spots in the image received by the receiving module along the first direction X1 and the second direction X2 can be made more random, so that the picture frame can be clearly distinguished when searching and calculating, and the situation of confusion in searching can be avoided, which causes a large difference between the searched position and the reference image.

[0064] In some embodiments, by adjusting the shooting angle of the receiving module and by adjusting the emission light angle of the transmitting module, the light spot blocks in the recognition image are arranged in an array along the seventh direction and the eighth direction, and the seventh direction is set at an angle to the first direction X1, and the eighth direction is set at an angle to the second direction X2.

[0065] It should be noted that the angles between the seventh direction and the eighth direction and the first direction X1 and the second direction X2 are related to the angle of adjustment of the emission module. At the same time, the seventh direction and the eighth direction are also related to the light source in the emission module and the direction of the light path in the diffractive optical element. This application does not limit the size of the angles between the seventh direction and the eighth direction and the first direction X1 and the second direction X2. In specific settings, different adjustments can be made according to different application scenarios.

[0066] It should be noted that in the embodiment of the present application, the shooting angle refers to the angle of the device outside the receiving module, and the image portion displayed in the algorithm search portion is the image displayed along the first direction and the second direction, such as Figure 7 The specific shooting principle adopts various shooting principles in the prior art, and this application will not repeat them here.

[0067] It should also be noted that in the embodiment of the present application, the arrangement of the light spots is adjusted by adjusting the configuration of the transmitting module and the receiving module at the same time. Different angles can be adjusted as needed to obtain a better effect.

[0068] In the embodiment of the present application, the light spot block includes a number of randomly arranged light spots. The emitted light spots are designed to be random light spots. Currently, the optical diffraction element (DOE) is designed in a dislocation manner, so that each light spot block on the VCSEL is dislocated to make it more random. The shape of the light spot block is exemplified in the embodiment of the present application. In some embodiments, the light spot block is in an irregular dot matrix, grid type, stripe type, or coded type.

[0069] In a second aspect, the present application provides a 3D structured light recognition device and a data processing unit as described above, wherein:

[0070] The emission module 10 is configured to emit light toward the surface of the object to be measured;

[0071] The receiving module 20 is configured to receive at least part of the light reflected from the surface of the object to be measured and obtain an identification image related to the object to be measured;

[0072] The data processing unit is configured to acquire the recognition image and construct 3D depth information of the object under test based on the recognition image.

[0073] Thirdly, Fig. 9 As shown, the present application provides a 3D structured light recognition method, comprising:

[0074] S02, the transmitting module transmits light to the surface of the object to be measured;

[0075] S04, the receiving module receives the light reflected from the surface of the object to be measured and obtains an identification image related to the object to be measured;

[0076] S06, the data processing unit acquires the recognition image, and presets a picture frame of a square structure, wherein the picture frame includes a first side extending along a first direction and a second side extending along a second direction;

[0077] S08. The data processing unit searches the recognition image through the picture frame to obtain 3D depth information of the object under test.

[0078] It should also be noted that in the embodiments of the present application, other processing methods in the prior art are used to post-process the standard image to obtain an image depth map, which will not be described in detail in the present application. Regardless of the processing method, it belongs to the protection scope of the present application without violating the inventive concept of the present application.

[0079] For the field of image processing, image transformation: such as Fourier transform, Walsh transform, discrete cosine transform and other indirect processing technologies, converting the processing in the spatial domain into the processing in the transform domain, can not only reduce the amount of calculation, but also obtain more efficient processing. Image coding: Compression image coding compression technology can reduce the amount of data describing the image (i.e. the number of bits) in order to save image transmission, processing time and reduce the occupied memory capacity. Image enhancement and restoration: The purpose of image enhancement and restoration is to improve the quality of the image, such as removing noise and improving the clarity of the image. Image segmentation: Image segmentation is one of the key technologies in digital image processing. Image segmentation is to extract the meaningful feature parts in the image. Its meaningful features include edges and regions in the image, which is the basis for further image recognition, analysis and understanding. Image description: Image description is a necessary prerequisite for image recognition and understanding. As the simplest binary image, its geometric characteristics can be used to describe the characteristics of the object. The general image description method uses two-dimensional shape description, which has two types of methods: boundary description and region description. Image recognition: Image recognition belongs to the category of pattern recognition. Its main content is to perform image segmentation and feature extraction after certain preprocessing (enhancement, restoration, compression) of the image, so as to make judgments and classifications.

[0080] The 3D structured light recognition method provided in the present application makes it more difficult to find the repeatability of light spots compared to the prior art. In this way, because the probability of repeatability of light spots within the search box is reduced, the probability of black hole problems in the depth map can also be greatly reduced.

[0081] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0082] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0083] Unless otherwise defined, the technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art in the technical field of the present invention. The terms used herein are only for describing specific implementation purposes and are not intended to limit the present invention. Terms such as "setting" and the like appearing in this article can mean that one component is directly attached to another component, or that one component is attached to another component through an intermediate. Features described in this article in one embodiment may be applied to another embodiment alone or in combination with other features, unless the feature is not applicable in the other embodiment or otherwise specified.

[0084] The present invention has been described through the above-mentioned embodiments, but it should be understood that the above-mentioned embodiments are only for the purpose of example and illustration, and are not intended to limit the present invention to the described embodiments. It can be understood by those skilled in the art that more variations and modifications can be made according to the teachings of the present invention, and these variations and modifications all fall within the scope of the protection claimed by the present invention.

Claims

1. A 3D structured light recognition system, characterized in that: Including 3D structured light recognition device and data processing unit, The 3D structured light recognition device comprises: The transmitting module is configured to transmit light to the surface of the object to be measured; A receiving module is configured to receive at least part of the light reflected from the surface of the object to be measured and obtain an identification image related to the object to be measured; wherein the identification image includes a plurality of light spot blocks arranged in an array, the light spot blocks are staggered in a first direction and the light spot blocks are staggered in a second direction; The 3D structured light recognition device further adjusts the shooting angle of the receiving module and / or adjusts the emission light angle of the transmitting module so that the light spot blocks in the recognition image are arranged in an array along a direction set at an angle to the first direction and a direction set at an angle to the second direction; The data processing unit is configured to acquire the recognition image and preset a picture frame of a square structure, wherein the picture frame includes a first side extending along a first direction and a second side extending along a second direction; and to search the recognition image through the picture frame to obtain 3D depth information of the object under test.

2. The 3D structured light recognition system according to claim 1, characterized in that: By adjusting the emission light angle of the emission module, the light spot blocks in the recognition image are arranged in an array along a third direction and a fourth direction, the third direction is set at an angle to the first direction, and the fourth direction is set at an angle to the second direction.

3. The 3D structured light recognition system according to claim 1, characterized in that: The light spot blocks in the recognition image are arranged in a staggered manner in the third direction and / or the light spot blocks are arranged in a staggered manner in the fourth direction.

4. The 3D structured light recognition system according to claim 1, characterized in that: By adjusting the shooting angle of the receiving module, the light spot blocks in the recognition image are arranged in an array along a fifth direction and a sixth direction, the fifth direction is set at an angle to the first direction, and the sixth direction is set at an angle to the second direction.

5. The 3D structured light recognition system according to claim 1, characterized in that: By adjusting the shooting angle of the receiving module and the emitting light angle of the transmitting module, the light spot blocks in the recognition image are arranged in an array along the seventh direction and the eighth direction, the seventh direction is set at an angle to the first direction, and the eighth direction is set at an angle to the second direction.

6. The 3D structured light recognition system according to claim 1, characterized in that: The included angle between the first direction and the second direction is 90°.

7. The 3D structured light recognition system according to claim 1, characterized in that: The light spot block includes a plurality of randomly arranged light spots.

8. The 3D structured light recognition system according to claim 1, characterized in that: The transmitting module comprises: a light source configured to emit a structured light beam having a random speckle pattern; a diffractive optical element, configured to receive the structured light beam and expand the structured light beam into a patterned light beam, and configured to project the patterned light beam onto a surface of an object to be measured; The patterned light beam is configured to form a plurality of light spot blocks arranged in an array on the surface of the object to be measured, wherein the light spot blocks are staggered in a first direction and the light spot blocks are staggered in a second direction.

9. A 3D structured light recognition method, characterized in that: A 3D structured light recognition system as claimed in any one of claims 1 to 8, comprising: The transmitting module transmits light to the surface of the object to be measured; The receiving module receives the light reflected from the surface of the object to be measured and obtains an identification image related to the object to be measured; The data processing unit acquires the recognition image and presets a picture frame of a square structure, wherein the picture frame includes a first side extending along a first direction and a second side extending along a second direction; The data processing unit searches the recognition image through the picture frame to obtain 3D depth information of the object under test.

Images