Depth camera and terminal
By designing multi-density and multi-density dot matrix light sources and diffraction elements in depth cameras, the problem of poor recognition accuracy of low-reverse objects by robot depth cameras is solved, the obstacle avoidance ability and synchronous positioning accuracy are improved, and the system power consumption is reduced and computing power.
Patent Information
- Application Number
- CN202411083415.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-13
AI Technical Summary
The depth cameras of existing robots have poor accuracy in low-reverse object recognition, which affects obstacle avoidance capabilities.
Design a depth camera to emit dot matrix light of different density and density through the first sub-light source array and the second sub-light source array, and combine specific diffraction elements and driving circuits to achieve high-precision identification of low-reverse objects and large objects. and obstacle avoidance.
It improves the recognition accuracy of the depth camera for low-reverse objects and large objects, enhances the robot's obstacle avoidance ability, and reduces power consumption and computing power, achieving synchronous positioning and real-time mapping.
Smart Images
Figure CN119135866A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of camera technology, and in particular to a depth camera and a terminal. Background Art
[0002] At present, the depth cameras of robots such as sweeping robots, service robots, and lawn mowing robots usually use surface light sources for obstacle avoidance and object recognition. However, the surface light source has poor recognition accuracy for low-reflective objects, which affects the robot's obstacle avoidance ability. Summary of the invention
[0003] In view of the above, it is necessary to propose a depth camera and a terminal to improve the recognition accuracy of the depth camera for low-reflective objects and the obstacle avoidance capability of the terminal.
[0004] The present application provides a depth camera, including:
[0005] An emitting component, used for emitting a first dot matrix light and a second dot matrix light toward a target scene, wherein the first dot matrix light and the second dot matrix light are configured such that an emission distance of the first dot matrix light is greater than an emission distance of the second dot matrix light, the emitting component comprises a light source and a diffraction element, the light source comprises a first sub-light source array and a second sub-light source array, the first sub-light source array comprises a plurality of first point light sources arranged according to a first rule, the second sub-light source array comprises a plurality of second point light sources arranged according to a second rule, the density of the plurality of first point light sources of the first sub-light source array is less than the density of the plurality of second point light sources of the second sub-light source array, and the diffraction element comprises a first partition pattern a first sub-light source array and a second sub-light source array, the first sub-light source array and the second sub-light source array are arranged correspondingly, the first sub-light source array includes a plurality of first microstructures, the second sub-light source array includes a plurality of second microstructures, the first sub-light source array emits a light beam toward the first sub-light source array and emits the first dot matrix light through the plurality of first microstructures of the first sub-light source array, the second sub-light source array emits a light beam toward the second sub-light source array and emits the second dot matrix light through the plurality of second microstructures of the second sub-light source array, and the spot density of the first dot matrix light is less than the spot density of the second dot matrix light;
[0006] a receiving component, configured to receive the first dot matrix light reflected back by a target scene and the second dot matrix light reflected back by the target scene, and to generate first depth information according to the first dot matrix light reflected back by the target scene, and to generate second depth information according to the second dot matrix light reflected back by the target scene;
[0007] The driving circuit is electrically connected to the receiving component and the light source respectively, and is used to drive and light up the first sub-light source array and the second sub-light source array according to the instruction of the receiving component.
[0008] In the above-mentioned depth camera, the transmitting component emits a first dot matrix light through multiple first point light sources of the first sub-light source array and multiple first microstructures of the first partition pattern, and emits a second dot matrix light through multiple second point light sources of the second sub-light source array and multiple second microstructures of the second partition pattern. The density of the multiple first point light sources of the first sub-light source array is less than the density of the multiple second point light sources of the second sub-light source array, the emission distance of the first dot matrix light is greater than the emission distance of the second dot matrix light, the spot density of the first dot matrix light is less than the spot density of the second dot matrix light, the first dot matrix light forms a relatively long and relatively sparse light spot array, and the second dot matrix light forms a relatively short and relatively dense light spot array. The spot of the first dot matrix light has stronger light energy than the spot of the second dot matrix light. The first dot matrix light can be emitted at a longer distance and can be used for synchronous positioning and real-time mapping. The spot of the second dot matrix light has a higher spatial resolution than the spot of the first dot matrix light, and can be used to identify low-reflective objects and small objects, which is beneficial to identify and avoid obstacles for low-reflective objects and small objects, so that the depth camera can simultaneously achieve obstacle avoidance and synchronous positioning and real-time mapping. In addition, the receiving component can clearly distinguish and differentiate the optical paths according to the first dot matrix light and the second dot matrix light emitted by the transmitting component, which is helpful to eliminate the multipath effect and improve the measurement accuracy of the depth camera.
[0009] In one embodiment, the first rule is that multiple first point light sources are arranged into several layers of first sub-arrays according to a first horizontal spacing value, and several layers of the first sub-arrays are arranged according to a first vertical spacing value, and the first point light sources of two adjacent layers of the first sub-arrays are staggered; the second rule is that multiple second point light sources are arranged into several layers of second sub-arrays according to a second horizontal spacing value, and several layers of the second sub-arrays are arranged according to a second vertical spacing value, and the second point light sources of two adjacent layers of the second sub-arrays are staggered, or, the second rule is that multiple second point light sources are randomly arranged according to the second horizontal spacing value and the second vertical spacing value, wherein the horizontal spacing value is the number of point light sources within a horizontal range of 1° field of view, and the vertical spacing value is the number of point light sources within a vertical range of 1° field of view.
[0010] The above-mentioned depth camera realizes the design of the first sub-light source array and the second sub-light source array by defining the first rule and the second rule.
[0011] In one embodiment, the first horizontal interval value range is less than 1.5°, the first vertical interval value range is less than 1.0°, the second horizontal interval value range is less than 1.0°, and the second vertical interval value range is less than 1.0°.
[0012] The above-mentioned depth camera, by limiting the range of the first horizontal interval value, the first vertical interval value, the second horizontal interval value and the second vertical interval value, is designed through the above-mentioned horizontal interval value and vertical interval value, which is beneficial for the receiving component to distinguish the reflected light path to eliminate the multi-path effect of the reflected light, and is also beneficial to reducing the power consumption of the transmitting component of the depth camera, and at the same time makes the density of the multiple first point light sources of the first sub-light source array less than the density of the multiple second point light sources of the second sub-light source array.
[0013] In one embodiment, the single-point light spot divergence angle range of the first point light source and the single-point light spot divergence angle range of the second point light source are both between 0.2° and 0.4°.
[0014] The above-mentioned depth camera, by limiting the single-point light spot divergence angle range of the first point light source and the single-point light spot divergence angle range of the second point light source, is conducive to the receiving component to distinguish and identify the interval value of the first dot matrix light and the second dot matrix light to eliminate the multipath effect of the reflected light. At the same time, the single-point light spot emission angle range is within the design value range, and the energy density can be increased, thereby improving the detection distance and accuracy.
[0015] In one embodiment, the light source includes a first sub-light source and a second sub-light source, the first sub-light source and the second sub-light source are connected, the first sub-light source includes the first sub-light source array, and the second sub-light source includes the second sub-light source array; the driving circuit includes a first sub-driving circuit and a second sub-driving circuit, the first sub-driving circuit is electrically connected to the receiving component and the first sub-light source, respectively, the first sub-driving circuit is used to drive and light up the first sub-light source array according to the instruction of the receiving component, the second sub-driving circuit is electrically connected to the receiving component and the second sub-light source, respectively, the second sub-driving circuit is used to drive and light up the second sub-light source array according to the instruction of the receiving component; and / or the first sub-light source array and the second sub-light source array are configured to be driven and lit at the same time.
[0016] The above-mentioned depth camera, by limiting the light source to include a first sub-light source and a second sub-light source, the first sub-light source and the second sub-light source can be designed and manufactured separately, which is conducive to simplifying the manufacturing process of the light source and facilitating mass production; wherein the first sub-light source and the second light source can also be designed and manufactured on a common substrate; by limiting the driving circuit to include a first sub-driving circuit and a second sub-driving circuit, the first sub-driving circuit and the second sub-driving circuit can respectively drive and light up the first sub-light source and the second sub-light source, simplifying the algorithm and method for driving and lighting up the first sub-light source and the second sub-light source, which is conducive to improving the detection distance capability of the depth camera, reducing hardware costs, and reducing size; by limiting the first sub-light source array and the second sub-light source array to be driven and lit at the same time, the depth camera simultaneously outputs a frame of sparse and dense depth information, thereby achieving obstacle avoidance and synchronous positioning and real-time mapping.
[0017] In one embodiment, a density of the first microstructures in the first partition pattern is smaller than a density of the second microstructures in the second partition pattern.
[0018] The above-mentioned depth camera, by limiting the density of multiple first microstructures of the first partition pattern to be smaller than the density of multiple second microstructures of the second partition pattern, enables the first partition pattern to cooperate with the first sub-light source array to emit a relatively sparse array of light spots, i.e., a first dot matrix light, and enables the second partition pattern to cooperate with the second sub-light source array to emit a relatively dense array of light spots, i.e., a second dot matrix light.
[0019] In one embodiment, the diffraction factor of the first microstructure is smaller than the diffraction factor of the second microstructure.
[0020] The above-mentioned depth camera limits the diffraction magnification of the first microstructure to be smaller than the diffraction magnification of the second microstructure, so that the spot density of the first lattice light emitted through the first partition pattern is smaller than the spot density of the second lattice light emitted through the second partition pattern.
[0021] In one embodiment, the light intensity of the light beam emitted by the first point light source is greater than the light intensity of the light beam emitted by the second point light source, and / or the frequency of the light beam emitted by the first point light source is greater than or equal to the frequency of the light beam emitted by the second point light source.
[0022] The above-mentioned depth camera, by limiting the light intensity of the light beam emitted by the first point light source to be greater than the light intensity of the light beam emitted by the second point light source, enables the first point array light to be projected at a farther distance relative to the second point array light; by limiting the frequency of the light beam emitted by the first point light source to be greater than or equal to the frequency of the light beam emitted by the second point light source, the first point array light is more conducive to synchronous positioning and real-time mapping, and the second point array light is more conducive to obstacle avoidance.
[0023] In one embodiment, the receiving component is configured to obtain multiple pixel amplitude highest coordinate points corresponding to the first dot matrix light and the second dot matrix light according to the first rule, the second rule and the computer program algorithm, and output the corresponding multiple pixel amplitude highest coordinate points as multiple effective pixel points, and generate the first depth information and the second depth information based on the multiple effective pixel points.
[0024] The above-mentioned depth camera, by limiting the way in which the receiving component generates the first depth information details and the second depth information details, is conducive to reducing the output bandwidth of the receiving component and the back-end computing power burden, reducing the power consumption and computing power of the depth camera, and improving the frame rate.
[0025] An embodiment of the present application further provides a terminal, comprising a main body and a depth camera as described in any of the above technical solutions, wherein the depth camera is arranged on the main body.
[0026] In the above-mentioned terminal, the transmitting component of the depth camera emits a first dot matrix light through multiple first point light sources of the first sub-light source array and multiple first microstructures of the first partition pattern, and emits a second dot matrix light through multiple second point light sources of the second sub-light source array and multiple second microstructures of the second partition pattern. The density of the multiple first point light sources of the first sub-light source array is less than the density of the multiple second point light sources of the second sub-light source array, the emission distance of the first dot matrix light is greater than the emission distance of the second dot matrix light, the spot density of the first dot matrix light is less than the spot density of the second dot matrix light, the first dot matrix light forms a relatively long and relatively sparse light spot array with an emission distance, and the second dot matrix light forms a relatively close and relatively dense light spot array with an emission distance. The spot of the first dot matrix light has stronger light energy than the spot of the second dot matrix light, the first dot matrix light can be emitted at a longer distance, and can be used for synchronous positioning and real-time mapping. The spot of the second dot matrix light has a higher spatial resolution than the spot of the first dot matrix light, and can be used to identify low-reflective objects and small objects, which is beneficial to identify and avoid obstacles for low-reflective objects and small objects, so that the depth camera can simultaneously achieve obstacle avoidance and synchronous positioning and real-time mapping. In addition, the receiving component can clearly distinguish and differentiate the optical paths according to the first dot matrix light and the second dot matrix light emitted by the transmitting component, which is conducive to eliminating the multipath effect, improving the measurement accuracy of the terminal's depth camera, and reducing the power consumption and computing power of the terminal's depth camera. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 Schematic diagram of the structure of the depth camera provided in the embodiment of the present application.
[0028] Figure 2 yes Figure 1 The schematic diagram of the optical path of the emitting component of the depth camera emitting the first dot matrix light and the second dot matrix light.
[0029] Figure 3yes Figure 1 Schematic diagram of the structure of the light source of the depth camera shown.
[0030] Figure 4 yes Figure 1 Schematic diagram of the structure of the diffraction element of the depth camera shown.
[0031] Figure 5 It is a structural block diagram of a power supply and a driving circuit provided in another embodiment of the present application.
[0032] Figure 6 It is a schematic diagram of the three-dimensional structure of the terminal provided in an embodiment of the present application.
[0033] Main component symbols
[0034] Terminal 1
[0035] Depth Camera 100
[0036] Transmitter assembly 10
[0037] Light source 12
[0038] The first sub-light source array 122
[0039] The first point light source 123
[0040] First subarray 1222
[0041] The second sub-light source array 124
[0042] The second point light source 125
[0043] The second subarray 1242
[0044] The first sub-light source 126
[0045] The second sub-light source 128
[0046] Diffractive element 14
[0047] First partition pattern 142
[0048] First microstructure 143
[0049] Second partition pattern 144
[0050] The second microstructure 145
[0051] Receiving component 20
[0052] Chip 22
[0053] Shot 24
[0054] Driving circuit 30
[0055] The first sub-driving circuit 32
[0056] The second sub-driving circuit 34
[0057] First Dot Light 40
[0058] Second dot light 50
[0059] Launch tube 60
[0060] Circuit connection substrate 70
[0061] Tube 80
[0062] Ontology 200 DETAILED DESCRIPTION
[0063] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0064] In the description of the present application, it should be understood that the terms indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, it should be noted that the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0065] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or mutual communication; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0066] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0067] See also Figure 1The embodiment of the present application provides a depth camera 100, which can obtain the depth information of any object in the target scene. The arbitrary object can be a large object such as a table, a chair, a floor fan, etc., or a small object such as a power cord, a fruit shell, a small toy, etc., or a low-reflective object such as a black thin line, glass, etc. The depth camera 100 can be applied to a sweeping robot, a service robot, and a lawn mowing robot to achieve obstacle avoidance, synchronous positioning, and real-time mapping.
[0068] The depth camera 100 includes a transmitting component 10 , a receiving component 20 and a driving circuit 30 .
[0069] Please refer to Figure 2 , Figure 3 and Figure 4 The transmitting assembly 10 is used to transmit a first dot matrix light 40 and a second dot matrix light 50 toward the target scene. The first dot matrix light 40 and the second dot matrix light 50 are configured so that the emission distance of the first dot matrix light 40 is greater than the emission distance of the second dot matrix light 50. The first dot matrix light 40 is approximately located above the second dot matrix light 50. It can be understood that in actual use, the transmitting assembly 10 transmits the first dot matrix light 40 toward the front or the sky. The first dot matrix light 40 is used to identify large objects and realize synchronous positioning and real-time mapping. The transmitting assembly 10 transmits the second dot matrix light 50 toward the ground. The second dot matrix light 50 is used to identify low-reflective objects and small objects. The second dot matrix light 50 is used to avoid obstacles. The transmitting assembly 10 includes a light source 12 and a diffraction element 14. The diffraction element 14 is arranged on the optical path of the light source 12. It can be understood that in actual use, the light source 12 and the diffraction element 14 can be arranged in a transmitting tube 60, so that the transmitting assembly 10 can be modularly arranged.
[0070] The light source 12 includes a first sub-light source array 122 and a second sub-light source array 124 (wherein, Figure 2The dashed line is only used to schematically distinguish the first sub-light source array 122 and the second sub-light source array 124, and is not a definition of the boundary between the first sub-light source array 122 and the second sub-light source array 124). The first sub-light source array 122 includes a plurality of first point light sources 123 arranged according to a first rule, and the second sub-light source array 124 includes a plurality of second point light sources 125 arranged according to a second rule. The density of the plurality of first point light sources 123 of the first sub-light source array 122 is less than the density of the plurality of second point light sources 125 of the second sub-light source array 124. Each first point light source 123 and each second point light source 125 are used to emit a light beam. The light beams emitted by the first point light source 123 and the second point light source 125 may be visible light, invisible light such as infrared light, ultraviolet light, blue light, etc. It can be understood that the light source 12 formed by the first sub-light source array 122 and the second sub-light source array 124 may be a vertical cavity surface laser emitter. The first sub-light source array 122 and the second sub-light source array 124 may be packaged together. In actual use, the first sub-light source array 122 is located above the second sub-light source array 124 .
[0071] The diffraction element 14 includes a first partition pattern 142 and a second partition pattern 144 (wherein, Figure 2 The middle dotted line is only used to schematically distinguish the first partition pattern 142 from the second partition pattern 144, and is not a limitation on the boundary between the first partition pattern 142 and the second partition pattern 144). The first partition pattern 142 is arranged corresponding to the first sub-light source array 122, and the second partition pattern 144 is arranged corresponding to the second sub-light source array 124. In actual use, the first partition pattern 142 is located above the second partition pattern 144. The first partition pattern 142 includes a plurality of first microstructures 143, and the second partition pattern 144 includes a plurality of second microstructures 145, wherein the density of the plurality of first microstructures 143 of the first partition pattern 142 may also be less than the density of the plurality of second microstructures 145 of the second partition pattern 144, so that the first partition pattern 142 is arranged corresponding to the first sub-light source array 122 on the optical path, and the second partition pattern 144 is also arranged corresponding to the second sub-light source array 124 on the optical path. Each microstructure may be understood as a micro-relief structure engraved or imprinted on the diffraction element 14 by a semiconductor. The diffraction element 14 may also be replaced by a super lens. It can be understood that the density of the plurality of first microstructures 143 may also be equal to the density of the plurality of second microstructures 145 , that is, the first partition pattern 142 may be substantially the same as the second partition pattern 144 .
[0072] The first sub-light source array 122 emits a light beam toward the corresponding first partition pattern 142 and emits a first dot matrix light 40 through diffraction modulation of the multiple first microstructures 143 of the first partition pattern 142. The second sub-light source array 124 emits a light beam toward the corresponding second partition pattern 144 and emits a second dot matrix light 50 through diffraction modulation of the multiple second microstructures 145 of the second partition pattern 144. The spot density of the first dot matrix light 40 is less than the spot density of the second dot matrix light 50. It can be understood that the first dot matrix light 40 forms a light spot array with a relatively long emission distance and relatively sparse, and the second dot matrix light 50 forms a light spot array with a relatively short emission distance and relatively dense. The light spot of the first dot matrix light 40 has stronger light energy than the light spot of the second dot matrix light 50. The first dot matrix light 40 can be emitted at a relatively longer distance, can be used to identify large objects and for synchronous positioning and real-time mapping, and has advantages in synchronous positioning and real-time mapping; because the light spot of the second dot matrix light 50 is denser than that of the first dot matrix light 40, the light spot of the second dot matrix light 50 has a higher spatial resolution than that of the first dot matrix light 40, and can be used to identify low-reflective objects and small objects, and has advantages in obstacle avoidance and identification of small objects.
[0073] The receiving component 20 is arranged at an interval from the transmitting component 10. The receiving component 20 is used to receive a first dot matrix light 40 reflected by any object in the target scene and a second dot matrix light 50 reflected by any object in the target scene, and generate first depth information according to the first dot matrix light 40 reflected by any object in the target scene, and generate second depth information according to the second dot matrix light 50 reflected by any object in the target scene. The first depth information is used to identify large objects and synchronize positioning and real-time mapping, and the second depth information is used to identify low-reflective objects, small objects and avoid obstacles.
[0074] The driving circuit 30 is electrically connected to the receiving component 20 and the light source 12, respectively. The driving circuit 30 is used to drive and light up the first sub-light source array 122 and the second sub-light source array 124 according to the instruction of the receiving component 20, so that the first sub-light source array 122 emits the first dot matrix light 40 through the first partition pattern 142, and the second sub-light source array 124 emits the second dot matrix light 50 through the second partition pattern 144, wherein the receiving component 20 and the light source 12 are started and stopped synchronously. It can be understood that in this embodiment, the depth camera 100 also includes a circuit connection substrate 70, and the transmitting component 10, the receiving component 20 and the driving circuit 30 are all arranged on the circuit connection substrate 70, and the transmitting component 10, the receiving component 20 and the driving circuit 30 can be electrically connected through the circuit connection substrate 70.
[0075] The above-mentioned depth camera 100, by setting the density of the multiple first point light sources 123 of the first sub-light source array 122 to be smaller than the density of the multiple second point light sources 125 of the second sub-light source array 124, the emitting component 10 emits the first dot matrix light 40 toward the front or the sky through the multiple first point light sources 123 of the first sub-light source array 122 and the multiple first microstructures 143 of the first partition pattern 142, and emits the second dot matrix light 50 toward the ground through the multiple second point light sources 125 of the second sub-light source array 124 and the multiple second microstructures 145 of the second partition pattern 144, the spot density of the first dot matrix light 40 is smaller than the spot density of the second dot matrix light 50, The first dot matrix light 40 forms a light spot array with a relatively long emission distance and relatively sparse, and the second dot matrix light 50 forms a light spot array with a relatively short emission distance and relatively dense. The light spot of the first dot matrix light 40 has stronger light energy than the light spot of the second dot matrix light 50. The first dot matrix light 40 can be emitted at a longer distance, and can be used to identify large objects and synchronize positioning and real-time mapping. The light spot of the second dot matrix light 50 has a higher spatial resolution than the light spot of the first dot matrix light 40, and can be used to identify low-reflective objects and small objects, which is beneficial for identifying and avoiding obstacles for low-reflective objects and small objects, so that the depth camera 100 can simultaneously achieve obstacle avoidance and synchronized positioning and real-time mapping. In addition, the receiving component 20 can clearly distinguish and differentiate the light paths according to the first dot matrix light 40 and the second dot matrix light 50 emitted by the transmitting component 10. Since the first dot matrix light 40 and the second dot matrix light 50 are both point light sources and are formed according to the first rule and the second rule, respectively, the receiving component 20 starts to calculate according to the first rule and the second rule and the first light spot received, thereby filtering out the light spots reflected back through multiple paths, which is conducive to eliminating the multipath effect, improving the measurement accuracy of the depth camera 100, and reducing the power consumption and computing power of the depth camera 100.
[0076] The first sub-light source array 122 and the second sub-light source array 124 can be configured to be driven and lit at the same time. The first sub-light source array 122 and the second sub-light source array 124 share a driving circuit 30. In this way, by limiting the first sub-light source array 122 and the second sub-light source array 124 to be driven and lit at the same time, the transmitting component 10 simultaneously emits the first dot matrix light 40 and the second dot matrix light 50, so that the depth camera 100 simultaneously outputs a frame of sparse and dense depth information, thereby achieving obstacle avoidance and synchronous positioning and real-time mapping.
[0077] In this embodiment, the receiving component 20 includes a chip 22 and a lens 24, the chip 22 and the lens 24 are arranged on the same optical axis, the chip 22 and the lens 24 can be arranged in a lens barrel 80, the chip 22 can be a TOF (time of flight) sensor, and the lens 24 can include one, two or more lenses, which can be specifically designed according to actual needs. Among them, the first dot matrix light 40 reflected by any object in the target scene and the second dot matrix light 50 reflected by any object in the target scene are both modulated by the lens 24 and incident on the chip 22, and the chip 22 generates the first depth information according to the transmission time or phase difference of the first dot matrix light 40 reflected by any object in the target scene, and generates the second depth information according to the transmission time or phase difference of the second dot matrix light 50 reflected by any object in the target scene, that is, the depth camera 100 adopts the iTOF (indirect time of flight measurement) solution or the dTOF (direct time of flight measurement) solution for depth measurement. In this way, by limiting the receiving component 20 to include the chip 22 and the lens 24, the receiving component 20 receives and generates first depth information details and second depth information details based on the first dot matrix light 40 reflected by any object in the target scene and the second dot matrix light 50 reflected by any object in the target scene.
[0078] The chip 22 is electrically connected to the driving circuit 30. In actual use, the chip 22 can send instructions to the driving circuit 30, and the driving circuit 30 simultaneously drives the first sub-light source array 122 and the second sub-light source array 124 to light up the light source 12. The frequency of the light beams emitted by the first sub-light source array 122 and the second sub-light source array 124 is configured by the chip 22, and the frequency ratio is required to meet the obstacle avoidance distance of 0.05m-1.5m and the SLAM (simultaneous positioning and real-time mapping) navigation distance of 0.1m-15m. Among them, in the iTOF scheme of measuring the distance between the depth camera 100 and any object in the target scene, the slam navigation frequency uses 37.65MHz&45.18MHz or 60.24MHz&52.71MHz to modulate the light beams emitted by the first sub-light source array 122 and the second sub-light source array 124, and the obstacle avoidance frequency uses 80.32MHz&60.24MHz to modulate the light beams emitted by the first sub-light source array 122 and the second sub-light source array 124; in the dTOF scheme of measuring the distance between the depth camera 100 and any object in the target scene, the slam navigation frequency and the obstacle avoidance frequency both use the same frequency, for example 5Mhz-7MHz.
[0079] The frequency calculation formula in the iTOF solution is: Among them, f mod The greatest common divisor frequency of the dual-frequency mixed frequency, the SLAM navigation frequency 37.65MHz & 45.18MHz f modThe maximum detection distance is 19.9m, and the obstacle avoidance frequency is 80.32MHz & 60.24MHz. mod The frequency is 20.08MHz, and the maximum detection distance is 7.47m. It is understandable that in actual use, when the detection distance of the depth camera 100 is 0m-15m, a dual-frequency frequency can be used to detect a frequency combination of 19.9m, and when the detection distance of the depth camera 100 is 0m-1.5m, a dual-frequency frequency can be used to detect a frequency combination of 7.5m.
[0080] The frequency calculation formula in the dTOF scheme is d=(c*t) / (2*f), where c is the speed of light, t is the photon flight time, and f is the frequency. When the detection distance of the depth camera 100 is 30m, a frequency of 5MHz can be used.
[0081] In this embodiment, the light intensity of the light beam emitted by the first point light source 123 of the first sub-light source array 122 is greater than the light intensity of the light beam emitted by the second point light source 125 of the second sub-light source array 124. In this way, by limiting the light intensity of the light beam emitted by the first point light source 123 of the first sub-light source array 122 to be greater than the light intensity of the light beam emitted by the second point light source 125 of the second sub-light source array 124, the first dot matrix light 40 can be projected at a longer distance relative to the second dot matrix light 50.
[0082] In this embodiment, the frequency of the light beam emitted by the first point light source 123 of the first sub-light source array 122 is greater than or equal to the frequency of the light beam emitted by the second point light source 125 of the second sub-light source array 124. In this way, by limiting the frequency of the light beam emitted by the first point light source 123 of the first sub-light source array 122 to be greater than or equal to the frequency of the light beam emitted by the second point light source 125 of the second sub-light source array 124, the first dot matrix light 40 is more conducive to synchronous positioning and real-time mapping, and the second dot matrix light 50 is more conducive to obstacle avoidance.
[0083] In this embodiment, please refer to Figure 3 The first rule is that the plurality of first point light sources 123 are arranged into a plurality of first sub-arrays 1222 (eg Figure 3 The second rule is that the plurality of second point light sources 125 are arranged into a plurality of second sub-arrays 1242 (such as Figure 3The second point light sources 125 of two adjacent layers of the second sub-arrays 1242 are arranged in a staggered manner, or the second rule is that the plurality of second point light sources 125 are randomly arranged according to the second horizontal spacing value a2 and the second vertical spacing value b2. In this way, by defining the above-mentioned first rule and second rule, the design of the first sub-light source array 122 and the second sub-light source array 124 is realized. Among them, the first horizontal spacing value a1 is greater than the second horizontal spacing value a2, and the first vertical spacing value b1 is greater than the second vertical spacing value b2.
[0084] In this embodiment, the range of the first horizontal interval value a1 is less than 1.5°, for example, 1.5°, 1.4°, 1.3°, 1.2°, 1.1°, 1.0°, 0.9°, 0.8°, 0.7°, 0.6°, 0.5°, etc., the range of the first vertical interval value b1 is less than 1.0°, for example, 0.9°, 0.8°, 0.7°, 0.6°, 0.5°, etc., the range of the second horizontal interval value a2 is less than 1.0°, for example, 0.9°, 0.8°, 0.7°, 0.6°, 0.5°, etc., the range of the second vertical interval value b2 is less than 1.0°, for example, 0.9°, 0.8°, 0.7°, 0.6°, 0.5°, etc. In this way, by limiting the range of the first horizontal interval value a1, the first vertical interval value b1, the second horizontal interval value a2 and the second vertical interval value b2, the first dot matrix light 40 is relatively sparsely designed, and the second dot matrix light 50 is relatively densely designed. The dot matrix light is conducive to distinguishing and identifying the reflected light to eliminate the multipath effect, reduce the power consumption and computing power of the depth camera 100, and at the same time make the density of the multiple first point light sources 123 of the first sub-light source array 122 less than the density of the multiple second point light sources 125 of the second sub-light source array 124. Among them, the horizontal interval value can be understood as the number of point light sources within the horizontal range of 1° field of view, and the vertical interval value can be understood as the number of point light sources within the vertical range of 1° field of view. For example, if the range of the horizontal interval value is 1°, there is one point light source in the horizontal range of 1° field of view, and for example, if the range of the vertical interval value is 0.5°, there are two point light sources in the vertical range of 1° field of view. It can be understood that the range values of the horizontal interval value and the vertical interval value are related to the horizontal resolution and the vertical resolution of the chip 22. The larger the resolution, the smaller the horizontal interval value and the vertical interval value, the relatively dense spot of the dot light, and the high spatial resolution. Conversely, the smaller the resolution, the larger the horizontal interval value and the vertical interval value, the relatively sparse spot of the dot light, and the ability to identify large objects at a farther distance.
[0085] In this embodiment, the single-point light spot divergence angle x range of the first point light source 123 and the single-point light spot divergence angle x range of the second point light source 125 are both between 0.2° and 0.4°, preferably 0.3°. In this way, by limiting the single-point light spot divergence angle x range of the first point light source 123 and the single-point light spot divergence angle x range of the second point light source 125, it is beneficial for the receiving component 20 to distinguish and identify the first dot matrix light 40 and the second dot matrix light 50. However, when the single-point light spot divergence angle x of the point light source is less than 0.2°, the chip 22 cannot effectively distinguish the reflected dot matrix light. When the single-point light spot divergence angle x of the point light source is greater than 0.4°, the second dot matrix light 50 emitted by the second sub-light source array 124 through the second partition pattern 144 cannot form a relatively dense light spot, which affects the depth camera 100's ability to resolve low-reflection objects and small objects.
[0086] In this embodiment, the range of the field of view angle formed by the first dot matrix light 40 and the second dot matrix light 50 emitted by the emitting component 10 is greater than 100° in the horizontal direction and between 5°-60° in the vertical direction. The dot matrix light emitted by the emitting component 10 is divided into the first dot matrix light 40 with relatively sparse light spots and the second dot matrix light 50 with relatively dense light spots in the vertical direction. The first dot matrix light 40 and the second dot matrix light 50 can be distributed symmetrically or asymmetrically. Exemplarily, the range of the field of view angle formed by the first dot matrix light 40 and the second dot matrix light 50 emitted by the emitting component 10 in the vertical direction is 50°, then the angular range of the field of view angle of the first dot matrix light 40 in the vertical direction can be 35°, and the angular range of the field of view angle of the second dot matrix light 50 in the vertical direction can be 15°, or the angular range of the field of view angle of the first dot matrix light 40 in the vertical direction can be 30°, and the angular range of the field of view angle of the second dot matrix light 50 in the vertical direction can be 20°.
[0087] In this embodiment, the diffraction magnification of the first microstructure 143 is smaller than the diffraction magnification of the second microstructure 145. The diffraction magnification can be understood as the first microstructure 143 and the second microstructure 145 diffracting a light spot into multiple light spots of 1*5, 2*5, 3*5, 1*3, 2*3, and 3*3. In this way, by limiting the diffraction magnification of the first microstructure 143 to be smaller than the diffraction magnification of the second microstructure 145, the spot density of the first dot matrix light 40 emitted through the first partition pattern 142 is smaller than the spot density of the second dot matrix light 50 emitted through the second partition pattern 144.
[0088] In this embodiment, the area of the first partition pattern 142 is greater than or equal to the area of the second partition pattern 144, that is, the area of the first partition pattern 142 corresponds to the required angle range of the field of view of the first dot matrix light 40 in the vertical direction, and the area of the second partition pattern 144 corresponds to the required angle range of the field of view of the second dot matrix light 50 in the vertical direction. In this way, by limiting the area of the first partition pattern 142 to be greater than the area of the second partition pattern 144, the depth camera 100 can simultaneously recognize small objects and large objects.
[0089] In this embodiment, the multiple first microstructures 143 of the first partitioning pattern 142 can also be roughly arranged according to the first rule, and the multiple second microstructures 145 of the second partitioning pattern 144 can also be roughly arranged according to the second rule, that is, the multiple first point light sources 123 of the first sub-light source array 122 are arranged in a one-to-one correspondence with the multiple first microstructures 143 of the first partitioning pattern 142, and the multiple second point light sources 125 of the second sub-light source array 124 are arranged in a one-to-one correspondence with the multiple second microstructures 145 of the second partitioning pattern 144.
[0090] It can be understood that in other embodiments, for example, see Figure 5The light source 12 may include a first sub-light source 126 and a second sub-light source 128. The first sub-light source 126 and the second sub-light source 128 are independently designed and manufactured. The first sub-light source 126 and the second sub-light source 128 are connected. It can be understood that the first sub-light source 126 and the second sub-light source 128 can be designed and manufactured on the same substrate. The first sub-light source 126 includes a first sub-light source array 122, and the second sub-light source 128 includes a second sub-light source array 124. The driving circuit 30 may include a first sub-driving circuit 32 and a second sub-driving circuit 34. The first sub-driving circuit 32 is electrically connected to the chip 22 of the receiving component 20 and the first sub-light source 126, respectively. The first sub-driving circuit 32 is used to drive and light up the first sub-light source array 122 according to the instruction of the chip 22 of the receiving component 20. The second sub-driving circuit 34 is electrically connected to the chip 22 of the receiving component 20 and the second sub-light source 128, respectively. The second sub-driving circuit 34 is used to drive and light up the second sub-light source array 124 according to the instruction of the chip 22 of the receiving component 20. In this way, by limiting the light source 12 to include the first sub-light source 126 and the second sub-light source 128, the first sub-light source 126 and the second sub-light source 128 can be designed and manufactured separately, and the first sub-light source 126 and the second sub-light source 128 can also be designed and manufactured on a common substrate, which is beneficial to simplifying the manufacturing process of the light source 12 and facilitating mass production; by limiting the driving circuit 30 to include the first sub-driving circuit 32 and the second sub-driving circuit 34, the first sub-driving circuit 32 and the second sub-driving circuit 34 can respectively drive the first sub-light source 126 and the second sub-light source 128 to light up, simplifying the algorithm for driving the first sub-light source 126 and the second sub-light source 128 to light up, which is beneficial to improving the detection distance capability of the depth camera 100, reducing hardware costs, and reducing size.
[0091] It can be understood that in other embodiments, the first sub-light source array 122 and the second sub-light source array 124 can be arranged in the same array, that is, the multiple first point light sources 123 and the multiple second point light sources 125 on the light source 12 are evenly arranged, and the power of each first point light source 123 and each second point light source 125 is the same, wherein the multiple first point light sources 123 in the first sub-light source array 122 can be selectively turned on so that the density of the multiple first point light sources 123 emitting light beams in the first sub-light source array 122 is less than the density of the multiple second point light sources 125 emitting light beams in the second sub-light source array 124. In this way, by limiting the light source 12 to evenly set the multiple first point light sources 123 and the multiple second point light sources 125, the manufacturing process of the light source 12 is simplified and mass production is facilitated. Among them, the redundant first point light sources 123 in the first sub-light source array 122 can also be used as spare point light sources.
[0092] In this embodiment, the chip 22 of the receiving component 20 has multiple pixel sub-areas, and the chip 22 of the receiving component 20 is configured to obtain multiple pixel amplitude highest coordinate points corresponding to the first dot matrix light 40 and the second dot matrix light 50 according to the first rule, the second rule and the computer program algorithm, and output the corresponding multiple pixel amplitude highest coordinate points as multiple effective pixel points, and generate first depth information and second depth information according to the multiple effective pixel points. The computer program algorithm is a preset program algorithm, that is, the chip 22 finds multiple valid pixel points corresponding to the first dot matrix light 40 and the second dot matrix light 50 reflected by any object in the target scene on multiple pixel sub-areas of the chip 22 through the first horizontal interval value a1 and the first vertical interval value b1 in the first rule, and the second horizontal interval value a2 and the second vertical interval value b2 in the second rule, each pixel sub-area includes multiple pixel points formed by the reflected first dot matrix light 40 and the second dot matrix light 50, that is, according to each horizontal interval value and each vertical interval value, find multiple pixel points corresponding to the dot matrix light in each pixel sub-area, calculate the coordinate point with the highest pixel amplitude value among the multiple pixel points in each pixel sub-area, output the corresponding multiple pixel amplitude highest coordinate points in the multiple pixel sub-areas as multiple valid pixel points, and generate corresponding depth information according to the multiple valid pixel points. In this way, by limiting the way in which the chip 22 generates the first depth information details and the second depth information details, it is beneficial to reduce the output bandwidth of the chip 22 and the back-end computing power burden, reduce the power consumption and computing power of the depth camera 100, and improve the frame rate.
[0093] See also Figure 6 The embodiment of the present application further provides a terminal 1. The terminal 1 includes a body 200 and a depth camera 100 as described in the above embodiment, and the depth camera 100 is arranged on the body 200. Among them, the terminal 1 can be a sweeping robot, a service robot, a lawn mowing robot, etc., and the body 200 can be a robot body.
[0094] In the above-mentioned terminal 1, the depth camera 100 sets the density of the multiple first point light sources 123 of the first sub-light source array 122 to be smaller than the density of the multiple second point light sources 125 of the second sub-light source array 124. The emitting component 10 emits the first dot matrix light 40 toward the front or the sky through the multiple first point light sources 123 of the first sub-light source array 122 and the multiple first microstructures 143 of the first partition pattern 142, and emits the second dot matrix light 50 toward the ground through the multiple second point light sources 125 of the second sub-light source array 124 and the multiple second microstructures 145 of the second partition pattern 144. The spot density of the first dot matrix light 40 is smaller than the spot density of the second dot matrix light 50. The first dot matrix light 40 forms a relatively sparse light spot array with a relatively long emission distance, and the second dot matrix light 50 forms a relatively dense light spot array with a relatively short emission distance. The light spot of the first dot matrix light 40 has stronger light energy than the light spot of the second dot matrix light 50. The first dot matrix light 40 can be emitted at a longer distance, and can be used to identify large objects and synchronize positioning and real-time mapping. The light spot of the second dot matrix light 50 has a higher spatial resolution than the light spot of the first dot matrix light 40, and can be used to identify low-reflective objects and small objects, which is conducive to identifying and avoiding obstacles for low-reflective objects and small objects, so that the depth camera 100 can simultaneously achieve obstacle avoidance and synchronize positioning and real-time mapping. In addition, the receiving component 20 can clearly distinguish and distinguish the light path according to the first dot matrix light 40 and the second dot matrix light 50 emitted by the transmitting component 10, which is conducive to eliminating the multipath effect, improving the measurement accuracy of the depth camera 100 of the terminal 1, and reducing the power consumption and computing power of the depth camera 100 of the terminal 1.
[0095] It is obvious to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application.
[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.
Claims
1. A depth camera, characterized in that: include: An emitting component, used for emitting a first dot matrix light and a second dot matrix light toward a target scene, wherein the first dot matrix light and the second dot matrix light are configured such that an emission distance of the first dot matrix light is greater than an emission distance of the second dot matrix light, the emitting component comprises a light source and a diffraction element, the light source comprises a first sub-light source array and a second sub-light source array, the first sub-light source array comprises a plurality of first point light sources arranged according to a first rule, the second sub-light source array comprises a plurality of second point light sources arranged according to a second rule, the density of the plurality of first point light sources of the first sub-light source array is less than the density of the plurality of second point light sources of the second sub-light source array, and the diffraction element comprises a first partition pattern a first sub-light source array and a second sub-light source array, the first sub-light source array and the second sub-light source array are arranged correspondingly, the first sub-light source array includes a plurality of first microstructures, the second sub-light source array includes a plurality of second microstructures, the first sub-light source array emits a light beam toward the first sub-light source array and emits the first dot matrix light through the plurality of first microstructures of the first sub-light source array, the second sub-light source array emits a light beam toward the second sub-light source array and emits the second dot matrix light through the plurality of second microstructures of the second sub-light source array, and the spot density of the first dot matrix light is less than the spot density of the second dot matrix light; a receiving component, configured to receive the first dot matrix light reflected back by a target scene and the second dot matrix light reflected back by the target scene, and to generate first depth information according to the first dot matrix light reflected back by the target scene, and to generate second depth information according to the second dot matrix light reflected back by the target scene; The driving circuit is electrically connected to the receiving component and the light source respectively, and is used to drive and light up the first sub-light source array and the second sub-light source array according to the instruction of the receiving component.
2. The depth camera according to claim 1, wherein: The first rule is that multiple first point light sources are arranged into several layers of first sub-arrays according to a first horizontal spacing value, and several layers of the first sub-arrays are arranged according to a first vertical spacing value, and the first point light sources of two adjacent layers of the first sub-arrays are staggered; the second rule is that multiple second point light sources are arranged into several layers of second sub-arrays according to a second horizontal spacing value, and several layers of the second sub-arrays are arranged according to a second vertical spacing value, and the second point light sources of two adjacent layers of the second sub-arrays are staggered, or, the second rule is that multiple second point light sources are randomly arranged according to the second horizontal spacing value and the second vertical spacing value, wherein the horizontal spacing value is the number of point light sources within a horizontal range of 1° field of view, and the vertical spacing value is the number of point light sources within a vertical range of 1° field of view.
3. The depth camera according to claim 2, wherein: The range of the first horizontal interval value is less than 1.5°, the range of the first vertical interval value is less than 1.0°, the range of the second horizontal interval value is less than 1.0°, and the range of the second vertical interval value is less than 1.0°.
4. The depth camera according to claim 3, wherein: The single-point light spot divergence angle range of the first point light source and the single-point light spot divergence angle range of the second point light source are both between 0.2° and 0.4°.
5. The depth camera according to claim 1, wherein: The light source includes a first sub-light source and a second sub-light source, the first sub-light source and the second sub-light source are connected, the first sub-light source includes the first sub-light source array, and the second sub-light source includes the second sub-light source array; the driving circuit includes a first sub-driving circuit and a second sub-driving circuit, the first sub-driving circuit is electrically connected to the receiving component and the first sub-light source respectively, the first sub-driving circuit is used to drive and light up the first sub-light source array according to the instruction of the receiving component, the second sub-driving circuit is electrically connected to the receiving component and the second sub-light source respectively, and the second sub-driving circuit is used to drive and light up the second sub-light source array according to the instruction of the receiving component; And / or, the first sub-light source array and the second sub-light source array are configured to be driven and lit at the same time.
6. The depth camera according to claim 1, wherein: A density of the plurality of first microstructures in the first division pattern is smaller than a density of the plurality of second microstructures in the second division pattern.
7. The depth camera according to claim 1, wherein: The diffraction magnification of the first microstructure is smaller than the diffraction magnification of the second microstructure.
8. The depth camera according to claim 1, wherein: The light intensity of the light beam emitted by the first point light source is greater than the light intensity of the light beam emitted by the second point light source, and / or the frequency of the light beam emitted by the first point light source is greater than or equal to the frequency of the light beam emitted by the second point light source.
9. The depth camera according to claim 1, wherein: The receiving component is configured to obtain multiple pixel amplitude highest coordinate points corresponding to the first dot matrix light and the second dot matrix light according to the first rule, the second rule and the computer program algorithm, and output the corresponding multiple pixel amplitude highest coordinate points as multiple valid pixel points respectively, and generate the first depth information and the second depth information based on the multiple valid pixel points.
10. A terminal, characterized in that: The device comprises a body and a depth camera as claimed in any one of claims 1 to 9, wherein the depth camera is arranged on the body.
Citation Information
Cited By
Depth camera and mobile robot
CN119716900A