Depth data measuring head, computing device and measuring method

Abstract

The depth data measuring head (600, 700) includes a structured light projection device (110, 610, 710) used to project a textured beam into the measurement target space at different projection angles driven by a driving device (114, 314) to form different textures on the detection target object, and a first and second image sensor (620, 630; 720, 730) arranged on both sides of the structured light projection device (110, 610, 710), having a predetermined relative spatial positional relationship, and used to perform imaging on the measurement target space at least twice, thereby obtaining at least two sets of images with different texture distributions, and the at least two sets of images are used to obtain one measurement depth data of the detection target object. The structured light projection device (110, 610, 710) is used to reflect the structured light generated by the light source module (I) at different angles, thereby realizing multi-pattern projection that is faster, more economical, and has a lower failure rate. Furthermore, the structured light projection device (110, 610, 710) is combined with multiple pairs of twin-lens sensors (723, 724; 733, 734) that share an optical path, thereby further shortening the frame interval and improving the quality of the depth fusion data. Further, a depth data calculation device and measurement method are disclosed.

Description

[Technical field]

[0001] The present invention relates to the field of three-dimensional detection technology, and in particular to a depth data measuring head, a computing device and a measuring method. [Background technology]

[0002] In recent years, the development of three-dimensional imaging technology has been remarkable. At present, the twin-lens detection means based on structured light can perform real-time three-dimensional measurement on the surface of an object. In brief, the means first projects a two-dimensional laser texture pattern with code information, for example a discretized speckle pattern, on the surface of the target object, and continuously collects the laser texture by two image collecting devices with relatively fixed positions; the processing unit uses a sampling window to sample the two images simultaneously collected by the two image collecting devices, identifies the matching laser texture pattern in the sampling window, and calculates the depth distance of each laser texture sequence segment projected on the surface of the natural object based on the difference between the matching texture patterns, and further measures to obtain the three-dimensional data of the surface of the measurement object.

[0003] In the matching process, the larger the sampling window, the more pattern information is contained in one sampling, making it easier to perform matching, but the graininess of the resulting depth image is coarser. Correspondingly, the smaller the sampling window, the finer the graininess of the image, but the higher the rate of false matching. The sampling window can be reduced by continuously taking multiple sets of different images, but this makes the system more complex and reduces the frame rate.

[0004] Thus, there is a need for improved means of measuring depth data. Summary of the Invention [Problem to be solved by the invention]

[0005] The technical problem to be solved by the present disclosure is to provide a depth data measuring means, which is particularly improved by multi-angle pattern projection and uses a structured light projection device that can reflect the structured light generated by the light source module at different angles, thereby realizing faster, more economical and lower failure rate projection. Furthermore, the structured light projection device can be combined with multiple pairs of twin-lens sensors that share an optical path, thereby further shortening the frame interval and improving the quality of the depth fusion data. [Means for solving the problem]

[0006] According to a first aspect of the present disclosure, there is provided a depth data measurement head including a structured light projection device and first and second image sensors respectively arranged on either side of the structured light projection device, wherein the structured light projection device is used to project a textured beam into a measurement target space at different projection angles driven by a driving device to form different textures on a detection target object in the measurement target space, the first and second image sensors have a predetermined relative spatial positional relationship, and imaging is performed at least two times on the measurement target space during the movement process of the reflection device, thereby obtaining at least two sets of images having different texture distributions, and the at least two sets of images are used to obtain one measurement depth data of the detection target object.

[0007] According to a second aspect of the present disclosure, there is provided a depth data calculation device including a depth data measurement head according to the first aspect of the present invention and a processor, wherein the processor is used to acquire the at least two sets of images, and identify depth data of the texture in each set of images based on a predetermined relative spatial positional relationship between the first and second image sensors, and fuse the identified depth data based on the at least two sets of images to obtain new depth data as a single measurement depth data of the detected object.

[0008] According to a third aspect of the present disclosure, a depth data measurement method is provided, comprising the steps of: projecting a speckle patterned beam emitted from a light source module at different angles; imaging a measurement target space at least two times using first and second image sensors having fixed relative positions, thereby obtaining at least two sets of images, in which different speckle patterns appearing due to the projection at different angles are projected into the measurement target space in the at least two imaging times; and determining depth data from the at least two sets of images and fusing the depth data. Effect of the Invention

[0009] Therefore, the reflected structured light can be rotated to improve the versatility of the structured light projection device, which can further combine the mask light source and the coaxial twin-lens method to further improve the accuracy and imaging speed of the multi-frame fusion method. [Brief description of the drawings]

[0010] The above and other objects, features, and advantages of the present disclosure will become more apparent by describing in more detail exemplary embodiments of the present disclosure with reference to the drawings, in which like reference numerals generally refer to like elements. [Figure 1] 1 is a schematic diagram illustrating the configuration of a structured light projection device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing how the projection direction is changed according to the present invention; [Diagram 3] 1 is a perspective view of a structured light projection device according to an embodiment of the present invention; [Figure 4A] 4 is an example of the structured light projection device shown in FIG. 3 projecting structured light at different perspective angles. [Figure 4B] 4 is an example of the structured light projection device shown in FIG. 3 projecting structured light at different perspective angles. [Figure 5A] This is an example of projecting speckles using the device shown in FIG. 3 or FIG. 4. [Figure 5B]This is an example of projecting speckles using the device shown in FIG. 3 or FIG. 4. [Figure 6] FIG. 2 is a configuration block diagram of a depth data measuring head according to an embodiment of the present invention. [Figure 7] 1 is a schematic diagram showing the configuration of a depth data measurement head according to an embodiment of the present invention. [Figure 8] FIG. 13 is a comparative timing diagram of two sets of coaxial imaging and one set of imaging. [Figure 9] FIG. 13 is a timing diagram of three sets of coaxial twin-lens imaging. [Figure 10] 2 is a schematic flowchart of a method for measuring depth data according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The preferred embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms and should not be limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0012] As mentioned above, in the matching process of twin-lens imaging, the larger the sampling window, the more pattern information is contained in one sampling, making it easier to perform matching, but the graininess of the resulting depth image will be coarser. Correspondingly, the smaller the sampling window, the finer the graininess of the image, but the higher the rate of false matching. Therefore, the sampling window can be reduced by continuously taking multiple sets of different images.

[0013] For example, a driving device can be used to drive the rotation of a light source module to realize projection from different angles by the same light source module. At this time, even if the pattern projected by the light source module is the same, different patterns will still appear in the field of view of the imaging device (image sensor) due to different projection angles. However, the light source module itself is heavy and needs to be wired and powered, so the driving function of the driving device is limited. In addition, depth fusion based on multi-frame reduces the frame rate and reduces the shooting performance of dynamic objects.

[0014] Therefore, the present invention provides a depth data measurement means that preferably uses an improved structured light projection device that can reflect the structured light generated by the light source module at different angles, thereby realizing multi-pattern projection that is faster, more economical and has a lower failure rate. Furthermore, the structured light projection device can be combined with multiple pairs of twin-lens sensors that share an optical path, thereby further shortening the frame interval and improving the quality of the depth fusion data.

[0015] 1 is a schematic diagram of a structured light projection device according to an embodiment of the present invention, which can be used to project structured light in the depth data measuring head of the present invention.

[0016] As shown in the figure, the structured light projection device 110 can include a light source module I located above the dashed line and a redirecting projection module II located below the dashed line in the figure. The light source module is used to generate a beam to be projected, and the redirecting projection module is used to redirect and emit the beam.

[0017] Specifically, the light source module is used to generate and emit a textured beam. In general, the light source module does not directly project the beam emitted by the light emitting device, but performs a certain optical processing on the beam projected by the light emitting device so that it exhibits a required distribution, brightness or pattern. Therefore, different embodiments can use different speckle generating means to achieve different projection effects.

[0018] As shown in Fig. 1, the light source module I may include a laser emitter 111 and a diffractive optical element (DOE) 112. The laser emitter 111 is used to emit a laser beam (shown by one arrow from 111 in the figure). The DOE 112 arranged on the emission optical path of the laser beam can modulate the incident laser light, for example, diffract the incident laser light and modulate it into discrete spots with a certain projection rule (as shown by two arrows from 112 in the figure, the beam diffracted on the surface has a certain width, i.e., a width occupying a certain area on the plane where the DOE 112 is located).

[0019] In one embodiment, the laser emitter 111 may be a laser diode (LD), such as an edge-emitting laser diode. The laser light generated by the LD may be collimated by a collimating lens (not shown) and diffracted by the DOE 112. If the laser emitter 111 is a single-beam laser emitter such as an LD, the DOE 112 may have a relatively complex structure to diffract one incident laser spot into a pattern having multiple spots (e.g., 2,000 or even 20,000 spots) to form complex speckles of projection. Because it is collimated, it can perform fixed-point projection by diffraction, and has advantages such as a large working distance and low power consumption.

[0020] In another embodiment, the laser light emitting element 111 may be realized as a VCSEL (Vertical Cavity Surface Emitting Laser). The VCSEL itself can contain multiple light emitting particles, so that one VCSEL chip can emit a predetermined pattern by itself, for example a pattern formed by 100 light emitting particles. In this case, the DOE 112 only needs simple copying, such as direct copying, cross copying, rotation copying, etc., to realize the speckle pattern of the above-mentioned 2,000 or even 20,000 spots. Although it generally cannot be collimated, the VCSEL method can have higher energy efficiency and also has lower requirements for the complexity of the DOE.

[0021] In other embodiments, the light source module I may be realized by using other means than laser elements and diffractive elements, for example, the light source module may include a flood light source used to generate a flood light and a mask disposed on the flood light source used to convert the flood light into a spot having a specific projection code.

[0022] In the embodiment in which a mask is added to the floodlight source, the mask can be designed with higher accuracy than the DOE, so that a finer speckle pattern can be obtained, and the information ratio (area occupied by speckles in the projection area) can be higher than that of the previous DOE method, for example, reaching 50%. With respect to obtaining the decoded depth data of the space where the speckles are projected, a higher information ratio means that the depth information of a larger surface area can be obtained in one projection, that is, more information can be obtained. However, the light energy of the masked area cannot actually be utilized, so the energy conversion efficiency of the mask method is not high. In other words, the structured light generation method in which a mask is added to the floodlight source is more suitable for close-range high-precision imaging scenes.

[0023] Furthermore, the redirecting projection module is a module for reflecting and outputting the beam (e.g., speckle pattern) generated by the light source module. However, unlike a conventional reflecting module, the redirecting projection module of the present invention can drive and move the reflecting module, thereby changing the beam reflected by the reflecting module.

[0024] A reflector 113 arranged in the exit path of the beam is used to reflect the incident beam so that the beam is exited, and a driver 114 connected to the reflector is used to change the angle of the reflector with respect to the incident beam, thereby changing the exit direction of the beam.

[0025] In the embodiment of the present invention, for convenience of explanation, the direction in which the light beam leaves the measurement head (i.e., the direction in which the light beam leaves the measurement head) may be defined as the z direction, the horizontal direction of the imaging plane as the x direction, and the vertical direction as the y direction. For this reason, in FIG. 1 and the following FIGS. 2, 3, and 4A-B, an example in which structured light is projected downward (y direction) to the light source module and the direction of emission is changed to the z direction (actually, a direction with a slight angular deviation along the z direction) by the direction-changing projection module is described in detail. Note that in other embodiments, the structured light projection device of the present invention may be arranged or installed in other directions as required in the actual imaging scene.

[0026] For convenience of explanation, FIG. 2 is a schematic diagram of changing the projection direction of the present invention. The light emitted from the light source is projected downward along the y axis onto a reflecting device, such as a reflecting mirror, and is reflected by the reflecting device, so that it can be projected into the measurement target space and form a spot on an imaging plane perpendicular to the z axis. The reflecting device can be rotated along the axial direction of the x axis, for example, rotated within an angle range of A to B as shown in the figure, thereby obtaining a spot that moves within a range of A' to B' on the imaging plane. When the light source module I shown in FIG. 1 projects discrete spots having a two-dimensional distribution pattern, the pattern projected by the light source module I onto the reflecting mirror is the same (because the relative positions of the laser light emitting element 111 and the DOE 112 are fixed and are not driven by themselves), but due to the rotation of the reflecting mirror, the projected pattern will have an angle deviation, so that in the situation of using two-eye imaging in combination as follows, the patterns captured by the image sensor at different projection angles can be regarded as different patterns.

[0027] FIG. 2 shows a reflective mirror that can rotate within a large angle range A-B to clarify the principle of the present invention, but it should be understood that in actual application scenarios, the angle difference projecting the same pattern may be as small as 1°, thereby ensuring that the imaging patterns are different while at the same time ensuring that the imaging ranges are substantially consistent.

[0028] Furthermore, Fig. 3 is a perspective view of a structured light projection device according to an embodiment of the present invention. Figs. 4A and 4B are examples of the structured light projection device shown in Fig. 3 projecting structured light at different perspective angles.

[0029] As shown in the figure, the laser light emitting element 311 may be installed inside a housing (or fixed structure) 315, and the generated light source can be diffracted by a DOE 312 arranged on the output light path to obtain a diffraction pattern with a predetermined two-dimensional distribution, and the diffraction pattern propagates upward to a reflecting mirror 313, which reflects the diffraction pattern and outputs it (approximately) along the z direction, and forms a speckle pattern on a projection plane perpendicular to the z direction (as shown in Figures 4A and 4B, where Figure 4B can be regarded as a view when the device shown in Figures 3 and 4A is rotated 90° counterclockwise along the y axis). The projection plane in Figures 4A and 4B can be regarded as a three-dimensional view of the imaging plane shown in Figure 1.

[0030] Furthermore, the driving device can control the reflecting device to move it in the axial direction, and the beam emitted from the light source module enters the reflecting device in a direction perpendicular to the axial direction (x direction), and the emission direction changes based on the axial movement of the reflecting device.

[0031] Specifically, as shown in the figure, a rotation shaft extending from a driving device (e.g., a motor) 314 is fixedly connected to a reflecting mirror 313, so that when the motor operates, the rotation shaft drives the reflecting mirror 313 to move in the axial direction. This forms a projection pattern with a predetermined angular deviation on the projection plane. These projection patterns may be different patterns for an image sensor that captures the projection, so that the projection device of the present invention can easily project "different" patterns by rotating the reflected structured light (e.g., a diffraction pattern with a two-dimensional distribution).

[0032] In some embodiments, the redirecting projection module may be a vibrating mirror, and the motor may drive the reflecting mirror to move back and forth within a predetermined range, while in other embodiments, the redirecting projection module may be a rotating mirror, and the motor may only perform unidirectional motion along an axial direction.

[0033] Specifically, the redirecting projection module may be a mechanical vibration mirror that reciprocates at a predetermined frequency, so that it can project structured light onto the measurement area at a predetermined frequency, so that a two-dimensional diffraction pattern (discrete spots) moving up and down along the y-direction appears on the projection plane. Due to the controllability of the mechanical vibration mirror, the redirecting projection module may remain stationary for a predetermined window period during the continuous motion.

[0034] For example, the mechanical vibration mirror may have a range of motion of ±1° along the z-direction and a vibration frequency of up to 2k per second. When used in combination with an image sensor, for example in an application scenario where three frames of images are synthesized into one frame of image, the mechanical vibration mirror can stop moving for a predetermined time when it moves to -1°, 0°, and 1°, and for example, remain stationary for the exposure time (e.g., 1 ms) required by the photosensitive unit, thereby preventing the projected pattern from changing during the exposure time of the image sensor and improving the accuracy of imaging.

[0035] In an embodiment using a rotating mirror, the rotating mirror only rotates in one direction and it is difficult to control the speed change, but a device such as a photodiode can be used to detect the movement angle of the rotating mirror, and structured light projection and dynamic imaging can be performed within an appropriate angle, for example, within a range of ±1° along the z direction out of 360°, and projection and corresponding imaging can be performed.

[0036] 5A-B are examples of projecting speckles using the device shown in FIG. 3 or FIG. 4. FIG. 5A and FIG. 5B can be regarded as patterns projected when the mechanical vibration mirror moves to 1° and −1°, respectively, and can also be regarded as imaging of the image sensor when the structured light projection device and the image sensor are combined to form a measurement head. To facilitate understanding, the projection range of the speckles is shown by a black frame in the figure, but in actual use, the black frame in the figure is not projected. In addition, since the area occupied by the speckles within the projection range of the speckles is less than 30%, it can be considered that a projection method using a laser light emitting element and a DOE is adopted.

[0037] As shown in Figures 5A and 5B, the two projected patterns are actually "same" because the same DOE is used (more specifically, the same pattern in the same DOE is used), but because the projection angles are different, the pattern in Figure 5B is shifted downward by a certain distance along the y direction. When measuring an object at the same position, such as a gray square in the figure, the speckles projected on the object are actually different (only the projection pattern in the range indicated by the dashed frame in Figure 5B is the same as the projection pattern in Figure 5A), so it can be considered that different patterns are projected on the same measurement object. And due to the irregularity (or randomness) of the speckle pattern, in the projection at different angles, the speckles are projected on different positions on the surface of the square object, so that more surface depth information can be obtained in the subsequent depth data fusion.

[0038] 1 to 5B, the projected structured light is projected along the z direction and fluctuates in the vertical direction (y direction), but in other embodiments, the projected structured light may be projected along the z direction and fluctuates in the horizontal direction (x direction), or may be a projection in which there is an angle transformation in both the x and y directions. In other words, by changing the projection angle in different directions, the dashed frame shown in FIG. 5B can be moved up and down, left and right, and even along the diagonal line.

[0039] In addition, the number of image sets required to obtain one measurement of depth data of a detection target object can be specified based on a specific application scene. For example, when the number of image sets that can be processed per second is specified, the imaging frame rate can be improved by reducing the number of image sets required for one measurement of depth data, and the imaging accuracy can be improved by increasing the number of image sets required for one measurement of depth data.

[0040] When utilizing the projection structure shown in Figures 4A-B, if the number of image sets required for one measurement of depth data is reduced, the angle at which the motor is changed may be reduced, and if the number of image sets required for one measurement of depth data is increased, the angle at which the motor is changed may be increased accordingly.

[0041] In one embodiment, the structured light projection device of the present invention may have an adjustable imaging rate of, for example, five stages. At the highest frame rate (i.e., the highest stage), a frame rate of, for example, 100 frames / second may be realized. At this time, only one set of image frames taken at the same time is used to calculate the measured depth data each time, and the projection angle of the motor can be always kept constant. At the next highest frame rate (i.e., the next highest stage), a frame rate of, for example, 50 frames / second may be realized. At this time, two sets of image frames taken at different times are used to calculate the measured depth data each time, and the projection angle of the motor can be changed as necessary, for example, by changing back and forth between 0.5° and 0°, and taking pictures at 0.5° and 0° respectively, two sets of image frames are obtained and merged. Similarly, at an intermediate frame rate (i.e., an intermediate stage), three sets of image frames are obtained and merged by taking pictures at 0.5°, 0°, and -0.5° respectively, to realize a frame rate of 33 frames / second. At the next lower frame rate (i.e., the next lower step), a frame rate of, for example, 25 frames / second may be realized.At the lowest frame rate (i.e., the lowest step), a frame rate of, for example, 20 frames / second may be realized.

[0042] The higher the frame rate, the fewer the pixel points in each frame, but it is suitable for the situation requiring high-speed imaging, while the lower the frame rate, the more the pixel points in each frame, and the more images are used for detection, so that the size of the matching window can be reduced at the same time, and higher accuracy can be obtained, which is suitable for the situation requiring high-precision imaging but not high requirements for the imaging frame rate. In actual use, the user can select the corresponding stage according to need, and the driving mechanism, the light source, the image sensor and the processor are combined with each other to realize the required image capture, fusion and calculation.

[0043] The structured light projection device of the present invention can be used to measure depth data. Fig. 6 is a block diagram of a depth data measurement head according to an embodiment of the present invention.

[0044] The illustrated depth data measurement head 600 includes a structured light projection device 610, and a first image sensor 620 and a second image sensor 630 having a predetermined relative positional relationship.

[0045] Under the drive of the drive unit included therein, the structured light projection device 610 can project a textured beam into the measurement target space at different projection angles, thereby forming different textures (e.g., the different textures shown in Figures 5A and 5B) on the detected object in the measurement target space.

[0046] The first and second image sensors 620 and 630, respectively arranged on either side of the structured light projection device, perform imaging of the measurement target space at least twice during the movement of the reflection device, thereby obtaining at least two sets of images with different texture distributions, and the at least two sets of images are used to obtain one measurement depth data of the detection target object.

[0047] In one embodiment, the structured light projection device 610 may include a light source module and a driving device, where the light source module is used to generate and emit a textured beam, and the driving device is used to drive the light source module to project the textured beam into the measurement target space at different projection angles. In other words, the driving device can directly drive the light source module to perform angle conversion. At this time, the driving device may be, for example, a voice coil motor attached to the light source module.

[0048] In a more preferred embodiment, the structured light projection device 610 may be a structured light projection device including a reflector as described with reference to Figures 1, 3 and 4. An embodiment of a measurement head based on this structured light projection device will be described in detail below with reference to Figure 7. The example of Figure 7 also includes a preferred imaging scheme with shared optical paths.

[0049] 7 is a schematic diagram of a depth data measurement head according to an embodiment of the present invention, in which an example of a single image sensor 720 is shown in more detail for ease of explanation.

[0050] 7, a depth data measurement head 700 based on the two-eye principle includes a projection device 710, and a first image sensor 720 and a second image sensor 730 having a predetermined relative positional relationship. The projection device 710 may be the structured light projection device described above with reference to FIGS. 1 and 3-4.

[0051] Although not shown for convenience, the measurement head 700 may further include a housing surrounding the above devices, and the coupling structure 740 shown in FIG. 7 may be considered as a mechanism for fixing and coupling the above devices to the housing. In some embodiments, the coupling structure 740 may be a circuit board including a control circuit. It is noted that in other embodiments, the above devices 710-730 may be coupled to the housing in other ways and perform corresponding data transmission and command receiving operations.

[0052] Here, the projection device 710 is used to project structured light onto the shooting area, for example, the same pattern diffracted through the DOE, but the existence of a redirecting mechanism allows the same pattern to be projected along different angles, so that the first image sensor 720 and the second image sensor 530 with a predetermined relative positional relationship photograph the shooting area to obtain a set of image frame pairs with different patterns, which can be used for one depth data calculation of the shooting area.

[0053] Specifically, the first image sensor 720 and the second image sensor 730 may be disposed on both sides of the structured light projection device 710. The first and second image sensors have a predetermined relative spatial relationship, and perform imaging on the measurement target space at least twice during the movement of the reflecting device, thereby obtaining at least two sets of images with different texture distributions, and the at least two sets of images are used to obtain one measurement depth data of the detection target object.

[0054] For example, the projection device 710 can project a textured beam into the measurement target space at a constantly changing projection angle under the driving device, thereby forming different textures on the detection target object in the measurement target space. The image sensor can perform imaging multiple times during the course of turning, for example, performing imaging once each time it moves to -1°, 0°, and 1°, thereby obtaining one set of image frame pairs including three pairs (six frames). These six frame images are used together to calculate one depth data of the shooting area, i.e., one frame of depth image can be calculated.

[0055] In some embodiments, the light source module in the projection device 510 may be constantly on during operation, in which case the image sensor may perform multiple imaging at a particular angle or at any angle (or any angle within a predetermined range of motion) as the redirection device rotates.

[0056] For example, in situations where a controllable mechanical rotating mirror is utilized, corresponding imaging can be performed based on the rotation angle of the rotating mirror, e.g., one image each when moved to -1°, 0°, and 1°, and corresponding exposures can be performed based on the measured rotation angle.

[0057] In some embodiments, the rotation angle and the exposure of the image sensor may not be synchronized. For example, if the mechanical rotating mirror is set to rotate within ±1°, the image sensor may take a set of images as needed at any time and at any interval, and different images may be taken as long as the interval does not completely overlap with the rotation frequency of the rotating mirror. In other words, it is not necessary to specify which pattern to project, since in a binocular situation, it is the difference between the same set of images that is compared, not necessarily with a reference pattern.

[0058] In another embodiment, the light source module in the projection device 710 may be synchronized with the exposure of the image sensor, in which case the measurement head 700 may further include a controller that is used to control the light source module to illuminate synchronously when the first and second image sensors are exposed.

[0059] Additionally, the controller may further be used to control the actuator to remain stationary during exposure of the first and second image sensors, thereby obtaining a sharper projection pattern, e.g., discrete spots, compared to imaging while the actuator is moving.

[0060] In some embodiments of the invention, the first and second image sensors may be conventional image sensors, however in other embodiments, the first and second image sensors each include at least two sub-image sensors sharing at least a portion of an optical path, each of the at least two sub-image sensors being used to perform one of the at least two imaging operations.

[0061] In the prior art, the first image sensor and the second image sensor each include only one photosensitive unit, and each photosensitive unit performs imaging three times to obtain an image frame pair consisting of three pairs (six frames) in one set; however, in the present invention, each of the first and second image sensors includes at least two sub-image sensors that share at least a portion of the optical path, and the at least two sub-image sensors are each used to image different patterns of structured light successively projected by the projection device.

[0062] FIG. 7 is an example in which the first and second image sensors each include two sub-image sensors (photosensitive units). As shown in the figure, the first image sensor 720 includes sub-image sensors 723 and 724, and the second image sensor 730 includes sub-image sensors 733 and 734. The sub-image sensors 723 and 724 share an optical path to the beam splitter surface of the beam splitter 722 and are at equal distances from the area of ​​the beam splitter. Similarly, the sub-image sensors 733 and 734 share an optical path to the beam splitter surface of the beam splitter 732 and are at equal distances from the area of ​​the beam splitter. In other words, the present invention introduces a binocular structure with multiple sets of coaxial lenses. Here, the sub-image sensors 723 and 733 located in different image sensor housings can be regarded as a first image sensor set (first binocular set), and are used to perform imaging with respect to structured light at one projection angle. Then, the sub-image sensors 724 and 734, which can be considered as a second image sensor set (second binocular set), are used to perform imaging of the structured light at another projection angle. In other words, the sub-image sensors 724 and 734, which are coaxial with 723 and 733 at this time, respectively, can be considered to perform imaging of the next pattern structured light in place of 723 and 733 (i.e., having an equivalent optical path). This allows the imaging interval of two adjacent frames to be independent of the frame interval of each image sensor, and imaging can be performed at a smaller interval.

[0063] For this purpose, the measurement head 700 may further include a synchronization device, which is used to cause at least two sub-image sensors included in the first image sensor 720 and the second image sensor 730, respectively, to sequentially and synchronously image the at least two different patterns of structured light at a first interval shorter than the frame imaging interval of the sub-image sensors at the same time as the projection device projects the at least two different patterns of structured light at the first interval. Correspondingly, each sub-image sensor images its own next frame at a second interval equal to or greater than the frame imaging interval of the sub-image sensors (e.g., images at its own frame interval), and the imaging operation may be synchronized with the projection by the projection device under synchronization by the synchronization device.

[0064] 8 is a comparative timing diagram of two sets of coaxial imaging and one set of imaging. For convenience of explanation, the frame rate of each photosensitive unit (sub-image sensor) is set to 100 frames / s, the frame interval is set to 10 ms (for example, the minimum frame interval is set to 10 ms), and the exposure time required for each photosensitive unit is set to 1 ms.

[0065] If the first image sensor 720 and the second image sensor 730 are conventional image sensors including only one photosensitive unit, and three patterns (corresponding to six images to be acquired) are used to perform depth data calculation as shown in Fig. 1, three imaging operations are required at 0, 10, and 20 ms, as shown in the lower part of Fig. 8. Therefore, to synthesize each depth data image, the subject needs to remain still for 21 ms (hence, it is more difficult to photograph a moving subject), and the frame rate is also reduced from 100 frames / s to 33.3 frames / s.

[0066] In contrast, in the image sensor of the present invention, in which the first image sensor 720 and the second image sensor 730 each include two photosensitive units (e.g., the first image sensor 720 and the second image sensor 730 each include sub-image sensors 723 and 724, and sub-image sensors 733 and 734), when three patterns are used to calculate depth data, as shown in the upper part of FIG. 8, the first photosensitive unit set images pattern 1 (e.g., a pattern at a first projection angle) at 0 ms, followed by the second photosensitive unit set images pattern 2 (e.g., a pattern at a second projection angle) at 1 ms, and then 10 ms later, the first photosensitive unit images pattern 3 (e.g., a pattern at a third projection angle) at 10 ms, thereby completing three imagings required for one depth data image. After that, the second photosensitive unit set can start the next imaging of pattern 1 at 11 ms. At 20 ms, the first photosensitive unit set images pattern 2. At 21 ms, the second photosensitive unit set further images pattern 3. In this way, the interval between imaging by different photosensitive unit sets only needs to be the time required for imaging (e.g., 1 ms), but the interval until the same photosensitive unit next images is still based on the minimum frame interval time (e.g., 10 ms) corresponding to the frame rate. At this time, by introducing two sets of coaxial twin lenses, the target only needs to remain still for 11 ms to synthesize each depth data image (so it is easier to photograph a moving target), and the frame rate can be maintained at about 66.6 frames / s.

[0067] Although an example having two sets of coaxial (same optical axis) photosensitive units has been described with reference to Figs. 7-8, in other embodiments, each of the first and second image sensors may include more photosensitive units. Fig. 9 is a timing diagram of twin-lens imaging with three coaxial sets. In this case, each of the first and second image sensors may include three coaxial photosensitive units (sub-image sensors). As a result, as shown in Fig. 9, the first photosensitive unit set images pattern 1 at 0 ms, immediately thereafter the second photosensitive unit set images pattern 2 at 1 ms, and immediately thereafter the third photosensitive unit set images pattern 3 at 2 ms. After that, the next three sets of imaging are started at 10 ms, and the next three sets of imaging are started at 20 ms, and so on. At this time, by introducing three sets of coaxial twin lenses, the three pairs (six frames) of images required to synthesize one depth data image can be acquired in just 3 ms, and the subject only needs to remain still for 3 ms, greatly improving the shooting level for moving subjects, and the frame rate can be maintained at approximately 100 frames / s (in this example, it takes 1003 ms, or 1.003 seconds, to shoot 100 frames).

[0068] Therefore, it can be understood that by simply introducing one additional set of coaxial twin-lens structures (or monocular structures), the frame rate of depth data based on multi-frame synthesis can be doubled, and the imaging time of each frame can be reduced. In theory, it is possible to arrange the same number of coaxial twin-lens structures as the number of images projected by the projection device, so that the framing time of each depth frame and the frame interval of the sensor are related only to a multiple of the exposure time (when the frame interval is greater than the exposure time times the number of sets of coaxial structures). For example, when synthesizing depth frames based on four patterns, when using two sets of coaxial twin-lens as shown in FIG. 7, the imaging time to acquire four frames is slightly extended to 12 ms, but the frame rate drops to about 50 frames / s. However, when four sets of coaxial twin-lens are used, the imaging time to acquire four frames is only 4 ms, and the frame rate is still kept at about 100 frames / s. However, the introduction of an excessive number of coaxial structures increases the difficulty of the structure of the image sensor, so a compromise is required in cost, feasibility, and imaging speed.

[0069] It should also be understood that Fig. 8 and Fig. 9 show an example in which the first sub-image sensor 722 performs imaging for 1 ms, and then the second sub-image sensor 73 immediately performs imaging for 1 ms, to illustrate the performance of coaxial imaging. However, in practical applications, the imaging interval of the first and second sub-image sensors should also take into account the driving of the driving device. Specifically, if imaging is performed directly in the driving process and no requirement is set for the appropriate angle of projection, the imaging of the second sub-image sensor can be performed directly after the first sub-image sensor completes imaging, as shown in Fig. 8 and Fig. 9. However, if there is a requirement for the appropriate angle of projection, or if the driving device needs to remain stopped during exposure, it is necessary to wait until the driving device moves to an appropriate position (and / or to an appropriate motion state, for example, completely stationary) after the imaging of the first sub-image sensor is completed, and then perform the exposure of the second sub-image sensor. Because the moving speed of the rotating mirror and the vibrating mirror is fast, the waiting time is relatively short, for example, tens of μs.

[0070] In the situation where imaging is performed by combining multiple sets of coaxial twin-lens structures, it is also necessary to set the actual framing rate as needed, for example, provide the user with high, medium and low framing stages. In different stages, the number of image sets that need to be captured in each frame is different, for example, 2 sets for the high stage, 4 sets for the medium stage, and 6 sets for the low stage. When the user selects the corresponding stage, the motor selection angle and rhythm, the projection of the structured light and the imaging time are correspondingly set to meet the needs of different imaging. Furthermore, when the processor performs calculations, different matching windows may be set based on the number of image sets included in one generated depth data, for example, the left and right image matching windows of 6 sets in the low stage may be smaller than the matching windows of 2 sets in the high stage, thereby improving the accuracy of imaging.

[0071] In order to realize the coaxial arrangement of different light-sensitive units in the same image sensor, the optical path needs to be designed. In the example of Fig. 7, the coaxial arrangement is shown to be realized based on beam splitting. In this case, the first image sensor 720 can include a lens unit 721 for receiving the incident return structured light, a beam splitter 722 for splitting the incident return structured light into at least a first beam and a second beam, a first sub-image sensor 723 for imaging the first beam, and a second sub-image sensor 724 for imaging the second beam of return structured light corresponding to a different pattern.

[0072] In one embodiment, the beam splitter 722 is an optical prism, such as a cube prism or a triangular prism, such that the reflected infrared light of the incident light reaches the second sub-image sensor 724 and the unreflected visible light of the incident light travels straight to the first sub-image sensor 723.

[0073] As shown, the prism-form beam splitter 722 can split the incoming light into two beams with mutually orthogonal propagation directions, and correspondingly the first sub-image sensor 723 and the second sub-image sensor 724 can be arranged perpendicularly, such that the incoming visible and infrared light beams are received at perpendicular angles, respectively.

[0074] In order to eliminate parallax and realize pixel-level alignment, it is necessary that the components of the incident light have the same optical path length. Therefore, when a cube prism is used as the beam splitter 722, the first sub-image sensor 723 and the second sub-image sensor 724 can be disposed at equal distances from the beam splitting region of the beam splitter 722. When a triangular prism is used as the beam splitter 722, the distance between the two photosensitive units and the beam splitter 722, especially the beam splitting region, can be flexibly adjusted based on the ratio of the refractive index between air and the prism material.

[0075] The pixel-level alignment or near-pixel-level alignment between the first sub-image sensor 723 and the second sub-image sensor 724 is theoretically realized by making the incident light share most of the optical path and have the same optical path length. However, during the actual manufacturing process of the image sensor, the actual arrangement of the first sub-image sensor 723 and the second sub-image sensor 524 is not ideally perpendicular and equidistant, so that deviations occur between the two imaging. In this case, a forced software correction can be performed on the manufactured image sensor. For example, a calibration target is introduced and both the imaging of the first sub-image sensor 723 and the second sub-image sensor 724 are aligned with the calibration target to achieve reliable pixel-level correction. In other words, the pixel-level alignment between the first sub-image sensor 723 and the second sub-image sensor 724 may be accurate pixel-level alignment, or may have a slight pixel difference and be aligned by calibration.

[0076] As shown in the figure, the image sensor 720 of the present invention may be realized by a single module. Therefore, the image sensor 720 may further include a housing used to fix the relative positions of the lens unit, the beam splitter, and the two light-sensitive units. Preferably, the housing can be combined with the lens unit 721 to form a sealed body, thereby preventing the contained device from being contaminated by the external environment. In other embodiments, the image sensor 720 of the present invention may be a part of a larger module (e.g., a depth data measuring head), and the fixing between each element is realized by the housing of the larger module.

[0077] Preferably, the image sensor 720 may further include a cable connected to the first sub-image sensor 723 and the second sub-image sensor 724, respectively. The housing has an opening for inserting the cable. In one embodiment, the cable may be a flexible cable, such as a FPC (flexible printed circuit) cable.

[0078] In one embodiment, the beam may be passed through a filter to further filter out the influence of light of other wavelengths before it is incident on the first sub-image sensor 723 and the second sub-image sensor 724. In one embodiment, the projection device may project infrared laser light, and therefore the filter disposed in the image sensor may be a corresponding infrared light transmission unit for transmitting infrared light of a specific frequency range, for example, infrared light of wavelengths 780 to 1100 nm in the present invention. In another embodiment, the projection device may project visible light, for example, red laser light or blue laser light such as 635 nm red light or 450 nm blue light. Although the ambient light may also contain red light or blue light, imaging with a high signal-to-noise ratio can also be performed with the help of a filter that transmits the corresponding red light or blue light, since the exposure time is short and the instantaneous laser light is strong. In addition, the sensor has a higher photoelectric conversion efficiency for blue light than for red light, and the shorter the wavelength band, the higher the accuracy, so that a laser element of an appropriate wavelength can be selected by comprehensively considering the cost of the laser element (blue light laser elements are generally expensive), the photoelectric conversion efficiency, and the accuracy of imaging.

[0079] Preferably, when the beam splitter is a cube prism, one side of the filter is in direct physical contact with the cube prism and the other side is in physical contact with the photosensitive unit, allowing the photosensitive unit and the cube prism to be locked within the housing, thereby ensuring a high degree of constancy in the relative positions of each device.

[0080] In some embodiments, particularly when the first and second sub-image sensors are infrared light sensors for receiving the projected infrared light patterns, an additional visible light sensing unit (not shown) may be disposed in the image sensor to capture image information of the measurement target object, so that the image captured by the image sensor includes not only image information of the measurement target object but also depth information. The visible light sensing unit may be a grayscale sensor or a color sensor. The grayscale sensor captures only brightness information, and the color sensor is used to capture color information of the measurement target object, in which case the visible light sensing unit may be composed of a three-primary color sensing unit, and the three primary colors may be red, green, blue (RGB) or cyan, magenta, yellow (CMY). Compared with depth data, the visible light image can better resolve the edge of the target object (especially when the color and / or brightness difference between the target object and the background environment is significant), so that the three-dimensional information of the target object can be more comprehensively obtained by combining with the visible light image.

[0081] 7, the structure of the first image sensor 720 has been specifically described, but the second image sensor 730 may have the same structure. In addition, 723 and 733 may be regarded as a first twin-lens set, and 724 and 734 may be regarded as a second twin-lens set, but 723 and 734 may also be regarded as a first twin-lens set, and 724 and 733 may also be regarded as a second twin-lens set. In either case, it is sufficient that the corresponding pattern is turned on and imaged after it is incident.

[0082] In addition to the beam splitter shown in FIG. 7, in different embodiments, other embodiments of an optical path conversion device used to change the optical path of the incident return structured light to send it to the first sub-image sensor and the first sub-image sensor may further be included.

[0083] When beam splitting is used to share the optical path as shown in FIG. 7, the amount of light acquired by each photosensitive unit is reduced, so the imaging sensitivity or effective distance range can be ensured by increasing the projection brightness or enlarging the entrance aperture.

[0084] Therefore, alternatively, light path sharing can be realized based on light path conversion. In this case, the first image sensor 720 and the second image sensor 730 can respectively include a lens unit for receiving the incident return structured light, a light path conversion device for directing the incident return structured light to at least a first sub-path and a second sub-path, a first sub-image sensor for imaging the return structured light on the first sub-path, and a second sub-image sensor for imaging the return structured light corresponding to a different pattern on the second sub-path. In one embodiment, the light path conversion device can be a rotating mirror, which can, for example, reflect the incident light to the light sensing unit 723 at 0 ms and reflect the incident light to the light sensing unit 724 at 1 ms. In other embodiments, the light path conversion device can be a device for performing light path conversion based on other mechanical, chemical, or electrical principles.

[0085] If the projection device performs full pattern projection (rather than scanning projection), the image sensor can be a rolling shutter image sensor or a global shutter image sensor (i.e., all pixels are imaged simultaneously). Global shutter sensors can achieve higher frame rates, and rolling shutter sensors can have an adjustable dynamic range, so you can choose the type of sensor you want to use depending on the actual application scenario.

[0086] As mentioned above, the projection device may include a vibrating mirror, such as a MEMS mirror or a mechanical vibrating mirror, that vibrates back and forth at a predetermined frequency, and is used to scan the shooting area with a predetermined frequency and range of motion to project structured light. Since the vibrating mirror can achieve a very high vibration frequency, such as 2k per second, it is impossible to directly use the activation signal of the MEMS mirror for synchronization (because the delay is not reliable). Therefore, in scenes where synchronization is required (such as detecting the rotation angle), a measuring device may be included in the synchronization device to measure the vibration phase of the vibrating mirror in real time, taking into account the phase vibration characteristics of the micromirror element.

[0087] In one embodiment, the measurement may be based on the emitted light itself. Thus, the measurement device may be one or more photosensors (e.g., two photodiodes PD), and the two photosensors are arranged in one of the following ways: on different emission paths of the projection device, on different reflection paths within the projection device, or on emission paths and reflection paths inside and outside the projection device, respectively. The arrangement manner of the photosensors can be reasonably selected to accurately measure the phase while not affecting the normal projection of the structured light. The PD may be mounted inside the projection device, and the instantaneous vibration phase is determined by measuring the reflection angle when the laser light is emitted from the optical window. Since the vibration phase of the MEMS mirror is a sinusoidal distribution, one PD can determine the sinusoidal distribution information, but more PDs are helpful to measure the phase more accurately. In another embodiment, the PD may be mounted outside the projection device, for example, on the optical window, for example near the edge of the optical window, thereby preventing the influence on the projection in the shooting area. In another embodiment, other ways may be used to measure the phase, for example, by performing a capacitance measurement.

[0088] However, in other embodiments, the projection device may include a mechanical rotating mirror that rotates in one direction, and correspondingly, if synchronization is required, the measurement device included in the synchronization device may be an angle meter for measuring the rotation angle of the motor of the reflection device in real time.

[0089] In the above embodiment, the synchronization between projection and exposure is realized by controlling the exposure of the image sensor. This can be used in situations where the projection angle of the light source is controllable (for example, the angle and rotation speed of the mechanical vibration mirror can be controlled by voltage and current), and especially in situations where the phase and speed of the light source scan are not controllable (for example, in MEMS mirrors or mechanical rotating mirrors). Therefore, the MEMS mirror can detect the angle using PD or capacitor, and the mechanical rotating mirror can also realize the detection of the position by voltage detection or photoelectric encoding.

[0090] According to another embodiment of the present invention, a depth data calculation device is provided, comprising the depth data measurement head and a processor connected to the depth data measurement head, the processor being used to determine depth data of an object to be photographed in the photographing area based on predetermined relative positions of a first and second image sensor in a binocular manner and the set of image frame pairs obtained by imaging the structured light therewith.

[0091] The present invention further provides a method for measuring depth data. Figure 10 is a schematic flow chart of a method for measuring depth data according to an embodiment of the present invention. The method can be realized by combining a structured light projection device, a measurement head and a computing device of the present invention.

[0092] In step S1010, the speckle patterned beam emitted from the light source module is subjected to rotational reflection.

[0093] In step S1020, the first and second image sensors are used to perform imaging on the measurement target space at least twice, thereby obtaining at least two sets of images, and different speckle patterns appearing due to the rotational reflection are projected on the measurement target space in the at least two imagings. In step S1030, depth data is obtained from the at least two sets of images, and the depth data is fused.

[0094] In a situation where a coaxial twin-lens structure is used, the step of performing imaging of the measurement target space at least twice using first and second infrared light image sensors whose relative positions are fixed includes a step of performing imaging a first time using a first sub-image sensor pair having a predetermined relative positional relationship to obtain a first image frame pair, and a step of performing imaging a second time using a second sub-image sensor pair to obtain a second image frame pair, wherein one sub-image sensor of each of the first and second sub-image sensor pairs shares at least a portion of the optical path and constitutes a first image sensor, and the other sub-image sensor of each of the first and second sub-image sensor pairs shares at least a portion of the optical path and constitutes a second image sensor, and the first and second image frame pairs are used to calculate one depth data of the shooting area.

[0095] The depth data measuring head, the calculation device and the measuring method of the present invention have been described in detail above with reference to the drawings. The depth measuring means of the present invention particularly uses an improved structured light projection device that can reflect the structured light generated by the light source module at different angles, thereby realizing multi-pattern projection that is faster, more economical and has a lower failure rate. Furthermore, the structured light projection device can be combined with multiple pairs of twin-lens sensors that share an optical path, thereby further shortening the frame interval and improving the quality of the depth fusion data.

[0096] The speckle measurement means of the present invention is particularly suitable for depth measurements on continuous surfaces, for example for gripping material for feeding and unloading or for detecting weld seams in shipyards.

[0097] The flowcharts and block diagrams in the figures show possible architectures, functions and operations of the systems and methods of the embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment or part of code, which includes executable commands for implementing a specified logical function. It is to be noted that in some alternative embodiments, the functions described in the blocks may be performed in a different order than that described in the figures. For example, two consecutive blocks may actually be executed substantially simultaneously, or may sometimes be executed in a reverse order depending on the functions involved. Also, each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be realized by a dedicated system based on hardware for executing the specified functions or operations, or a combination of dedicated hardware and computer commands.

[0098] Although each embodiment of the present invention has been described, the above description is illustrative, not exhaustive, and not limited to each embodiment disclosed. Many modifications and changes will be obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification are selected to best explain the principles of each embodiment, practical applications, or improvements to technology in the market, or to enable those skilled in the art to understand each embodiment disclosed in this specification.

Claims

1. a structured light projection device; and first and second image sensors disposed on either side of the structured light projection device, The structured light projection device is used for projecting a textured beam into a measurement target space at different projection angles driven by a driving device, thereby forming different textures on a detection target object in the measurement target space; the first and second image sensors have a predetermined relative spatial relationship and are used to perform imaging of the measurement target space at least twice, thereby obtaining at least two sets of images having different texture distributions; The at least two sets of images are used to obtain one measurement depth data of the detected object; each of the first and second image sensors includes at least two sub-image sensors that share at least a portion of an optical path, and each of the at least two sub-image sensors is used to perform one of the at least two imaging operations; a controller for causing at least two sub-image sensors included in each of the first and second image sensors to perform continuous synchronous imaging at a first interval that is shorter than a minimum frame imaging interval of the sub-image sensors; Depth data measurement head.

2. The structured light projection device includes a light source module and the driving device, The light source module is used for generating and emitting a textured beam; The depth data measuring head according to claim 1 , wherein the driving device is used to drive the light source module to project textured beams at different projection angles into the measurement target space.

3. the structured light projection device includes a light source module and a redirecting projection module; The light source module is used for generating and emitting a textured beam; The depth data measurement head of claim 1, wherein the redirecting projection module includes a reflecting device and the driving device, the reflecting device being disposed in the exit path of the beam and used to reflect the incident beam so that the beam is emitted, and the driving device being connected to the reflecting device and used to change the angle of the reflecting device relative to the incident beam, thereby changing the exit direction of the beam.

4. the redirecting projection module is a mechanically oscillating mirror, and the reflector reciprocates along an axial direction; or The depth data measurement head of claim 3 , wherein the redirecting projection module is a mechanical rotating mirror, and the reflecting device moves unidirectionally along an axial direction.

5. The redirecting projection module is a mechanical vibrating mirror, and the reflecting device reciprocates along an axial direction; The depth data measuring head of claim 4 , wherein the drive of the redirecting projection module remains stationary for a predetermined window period during continuous motion of the depth data measuring head.

6. The structured light projection device includes a light source module, The light source module includes: A laser light emitting element used to emit a laser beam; A diffractive optical element is disposed on the output optical path of the laser beam, and is used to diffract the incident laser light and modulate it into discrete spots with a specific projection pattern; or A floodlight source used for generating floodlight; a mask disposed on the light source and used to convert the light into a spot having a specific projection code; The depth data measuring head of claim 1 , comprising:

7. The structured light projection device according to claim 1, further comprising: a light source module; The depth data measurement head of claim 1 , further comprising a controller adapted to control the light source modules to synchronously illuminate when the first and second image sensors are exposed.

8. 2. The depth data measuring head of claim 1, wherein the controller is adapted for each sub-image sensor to image its respective subsequent frame at a second interval that is greater than or equal to a minimum frame imaging interval of the sub-image sensor.

9. The first and second image sensors each include: a lens unit for receiving the incident return structured light; an optical path conversion device for transmitting the incident return structured light to the first sub-image sensor and the second sub-image sensor by changing an optical path; a first sub-image sensor and a second sub-image sensor for imaging different patterns at different times; The depth data measuring head of claim 1 , comprising:

10. The optical path changing device includes: a beam splitter for splitting the incident return structured light into at least a first beam and a second beam; a first sub-image sensor adapted to image the first beam; 10. The depth data measurement head of claim 9, wherein a second sub-image sensor is used to image a second beam of returning structured light corresponding to a different pattern.

11. the optical path changing device includes an optical path changing device for directing the incident return structured light to at least a first sub-path and a second sub-path; a first sub-image sensor for imaging the returning structured light on the first sub-path; The depth data measuring head of claim 9 , further comprising a second sub-image sensor adapted to image returning structured light corresponding to a different pattern on a second sub-path.

12. The depth data measuring head according to claim 9 , wherein the first sub-image sensor and the second sub-image sensor are equidistant from a beam splitting region of a beam splitter or an optical path changing region of the optical path changing device.

13. At least two sub-image sensors included in each of the first and second image sensors and sharing at least a portion of the optical path are infrared light sensors; and / or The depth data measurement head of claim 9 , wherein the first and second image sensors each include a visible light image sensor used for imaging incident structured light and sharing at least a portion of an optical path with the first and / or second image sub-sensor.

14. The depth data measuring head of claim 1 , wherein corresponding combinations of projection angles of the driving device, projection times of the beams, and imaging times of the first and second image sensors are set based on the number of different image sets required for one measurement of depth data.

15. A depth data measuring head according to any one of claims 1 to 14, and a processor, A depth data calculation device, wherein the processor is used for acquiring the at least two sets of images, and determining depth data of the texture in each set of images based on a predetermined relative spatial positional relationship between the first and second image sensors, and fusing the determined depth data based on the at least two sets of images to obtain new depth data as a single measurement depth data of the detected object.

16. projecting a speckle patterned beam emitted from a light source module at different angles; Performing imaging of the measurement target space at least twice using first and second image sensors whose relative positions are fixed, thereby obtaining at least two sets of images, in which different speckle patterns appearing due to the projection at different angles are projected into the measurement target space in the at least two imaging operations; determining depth data from the at least two sets of images and performing depth data fusion; The step of performing imaging of the measurement target space at least twice using first and second infrared light image sensors whose relative positions are fixed includes causing at least two sub-image sensors, each of which shares at least a portion of an optical path included in the first and second image sensors, to perform imaging continuously and synchronously at an interval shorter than a minimum frame imaging interval of the sub-image sensors. Depth data measurement method.

17. The step of performing imaging of the measurement target space at least twice using the first and second infrared light image sensors whose relative positions are fixed includes: performing imaging a first time using a first pair of sub-image sensors having a predetermined relative positional relationship to obtain a first pair of image frames; performing imaging a second time using a second sub-image sensor pair to obtain a second image frame pair, wherein one sub-image sensor of each of the first and second sub-image sensor pairs shares at least a portion of the optical path and constitutes a first image sensor, and the other sub-image sensor of each of the first and second sub-image sensor pairs shares at least a portion of the optical path and constitutes a second image sensor, and the first and second image frame pairs are used to calculate one-time depth data of the measurement target space; 20. The method of claim 16, comprising:

Images