Binocular camera and robot

Abstract

The present application discloses a binocular camera and a robot, which relate to the field of machine vision technology. The binocular camera comprises a substrate, and a fixed frame arranged on the substrate, on which dot matrix projection modules are arranged at intervals, and the two dot matrix projection modules have an output light path at a preset angle, and the two dot matrix projection modules are at an equal vertical distance from the substrate; the binocular camera also comprises a first light receiving module and a second light receiving module arranged on the substrate, and the first light receiving module and the second light receiving module are used to respectively collect light reflection information of the two dot matrix projection modules. It can improve the field of view angle, thereby improving the spatial three-dimensional reconstruction capability.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a binocular camera and a robot. Background Art

[0002] Binocular stereo vision is an important form of machine vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the object under test from different positions. It calculates the position deviation between corresponding points in the images to obtain the three-dimensional geometric information of the object.

[0003] In the prior art, binocular stereo vision mostly uses active binocular structured light solutions to reconstruct the three-dimensional space. In the existing active binocular structured light cameras, the viewing angle is limited by the field of view of the dot matrix projector, resulting in a small viewing angle. The small viewing angle will inevitably lead to a small operating space, and it is impossible to detect large-scale three-dimensional obstacles, which affects the robot's obstacle avoidance, simultaneous localization and mapping (SLAM) or navigation functions. Summary of the invention

[0004] The purpose of this application is to provide a binocular camera and a robot that can improve the field of view and thus improve the three-dimensional reconstruction capability of space.

[0005] The embodiment of the present application is implemented as follows:

[0006] According to one aspect of an embodiment of the present application, a binocular camera is provided, comprising a substrate and a fixing frame arranged on the substrate, wherein dot matrix projection modules are arranged on the fixing frame at intervals, and the output light paths of the two dot matrix projection modules are at a preset angle, and the vertical distances between the two dot matrix projection modules and the substrate are equal; the binocular camera also comprises a first light receiving module and a second light receiving module arranged on the substrate, and the first light receiving module and the second light receiving module are used to respectively collect light reflection information of the two dot matrix projection modules.

[0007] Optionally, the fixing frame is in the form of an isosceles triangle structure, and the two dot matrix projection modules are respectively located on two opposite sides of the isosceles triangle.

[0008] Optionally, the binocular camera also includes a closed shell and a transparent cover plate arranged on one side of the shell, the substrate, the fixing frame, the dot matrix projection module, the first light receiving module and the second light receiving module are all located in the closed shell, and the substrate is arranged parallel to the transparent cover plate.

[0009] Optionally, the distance between the two dot matrix projection modules is Wherein, d is the distance between the two dot matrix projection modules, 2β is the field of view of the dot matrix projection module, 2θ is the field of view of the binocular camera, and h is the minimum application distance expected by the binocular camera.

[0010] Optionally, the dot matrix projection module includes at least one dot matrix projector; when the number of the dot matrix projectors is greater than or equal to two, the dot matrix projectors of each of the dot matrix projection modules are on the same straight line, and the two straight lines are parallel to each other.

[0011] Optionally, the dot matrix projector includes a light source, and a collimating lens and a diffraction optical element located on the outgoing light path of the light source.

[0012] Optionally, the fixing frame includes positioning seats that are arranged at intervals, and the two dot matrix projection modules are respectively located on positioning surfaces of the positioning seats, and the two positioning surfaces respectively coincide with two sides of an isosceles triangle.

[0013] Optionally, the first light receiving module and the second light receiving module are respectively located on two opposite sides of the two dot matrix projection modules.

[0014] Optionally, the fixing frame is made of heat conductive material.

[0015] Another aspect of an embodiment of the present application provides a robot comprising a binocular camera as described in any one of the above.

[0016] The beneficial effects of the embodiments of the present application include:

[0017] The binocular camera provided in the embodiment of the present application provides stable support for the dot matrix projection module, the first light receiving module and the second light receiving module through a substrate and a fixing frame arranged on the substrate, so as to ensure the stability of the relative positions between the dot matrix projection module, the first light receiving module and the second light receiving module. By setting the outgoing light paths of the two dot matrix projection modules arranged on the fixing frame at a preset angle, it is beneficial to improve the field of view of the binocular camera compared with the use of a single projector. When the field of view of the binocular camera is increased, it is beneficial for the first light receiving module and the second light receiving module to receive a larger range of speckle pattern information, expand the depth reconstruction range of the binocular camera, and thus improve the spatial three-dimensional reconstruction capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 One of the structural schematic diagrams of the binocular camera provided in the embodiment of the present application;

[0020] Figure 2 The second structural diagram of the binocular camera provided in the embodiment of the present application;

[0021] Figure 3 A diagram showing the positional relationship between the dot matrix projection module and the transparent cover provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of the connection between the fixing frame and the dot matrix projector provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of the connection between the positioning seat and the dot matrix projection module provided in an embodiment of the present application.

[0024] Icon: 100 - binocular camera; 120 - fixing bracket; 122 - positioning seat; 1222 - positioning surface; 130 - dot matrix projection module; 132 - dot matrix projector; 140 - first light receiving module; 150 - second light receiving module; 160 - closed shell; 170 - transparent cover. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0027] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In addition, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0028] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0029] With the improvement of people's living standards, indoor robots based on intelligent navigation solutions have gradually entered people's lives, and the 3D perception system is its most core part to realize functions such as SLAM and obstacle avoidance. At present, binocular stereo vision mostly uses active binocular structured light solutions to reconstruct the three-dimensional space. However, in actual use, its effect is unsatisfactory. The main problem is that the obstacle avoidance ability is poor. Its main sensors are mostly located on the top of the robot, with a small viewing angle and a small visual range, which inevitably leads to a small operating space and cannot realize the detection of large-scale three-dimensional obstacles.

[0030] The depth field angle of active binocular structured light depends on the field angle of the camera. Limited by the technology of micro-nano optics, under the premise of ensuring optical performance, the existing field angle is up to about 60×80, which can only be used in some specific scenarios such as access control and door locks. In the application scenario of robot intelligent navigation, the required depth reconstruction field angle can reach 120×80. The existing cameras obviously cannot achieve such a large field angle, which limits the three-dimensional reconstruction capability in practical applications. Based on this, the embodiment of the present application specifically proposes the following scheme to improve the field angle, thereby improving the spatial three-dimensional reconstruction capability.

[0031] Please refer to Figure 1 The present embodiment provides a binocular camera 100, including a substrate, and a fixing frame 120 arranged on the substrate, on which dot matrix projection modules 130 are arranged at intervals, the exit light paths of the two dot matrix projection modules 130 are at a preset angle, and the vertical distances between the two dot matrix projection modules 130 and the substrate are equal; the binocular camera 100 also includes a first light receiving module 140 and a second light receiving module 150 arranged on the substrate, and the first light receiving module 140 and the second light receiving module 150 are used to respectively collect light reflection information of the two dot matrix projection modules 130.

[0032] Specifically, the parameters of the two dot matrix projection modules 130 spaced apart on the fixing frame 120 are the same, and the vertical distances between the two dot matrix projection modules 130 and the substrate are equal. In this way, the two dot matrix projection modules 130 are located at the same installation height. When in use, the speckle patterns projected by the dot matrix projection modules 130 are equal in size at the same distance, which is conducive to ensuring the consistency of the patterns projected by the two dot matrix projection modules 130, so as to reduce the difficulty of calculation. In practical applications, dot matrix projection modules 130 with different parameter information can also be set as needed to meet diverse needs.

[0033] The binocular camera 100 of the present application is implemented based on the optical triangulation measurement principle of active binocular structured light three-dimensional vision. When in use, the two dot matrix projection modules 130 project a certain pattern of structured light onto the surface of the object to form a three-dimensional image of a light strip modulated by the surface shape of the object being measured on the surface of the object. The three-dimensional image is collected by the first light receiving module 140 and the second light receiving module 150 to obtain a two-dimensional distorted image of the light strip. The degree of distortion of the light strip depends on the relative position between the dot matrix projection module 130 and the first light receiving module 140 and the second light receiving module 150, respectively, and the surface profile (height) of the object. When the relative position between the dot matrix projection module 130, the first light receiving module 140 and the second light receiving module 150 is constant, the three-dimensional profile of the surface of the object can be reproduced by the distorted two-dimensional light strip image coordinates to achieve the purpose of spatial three-dimensional reconstruction.

[0034] It should be noted that when using active binocular structured light for spatial three-dimensional reconstruction, the depth information cannot be calculated in the area where the speckle pattern is not projected, due to the projection range of the binocular camera 100 speckle pattern, that is, the field of view. In the embodiment of the present application, by making the outgoing light paths of the two dot matrix projection modules 130 present a preset angle, the field of view of the binocular camera 100 can be improved, and then the light reflection information (image information) collected by the first light receiving module 140 and the second light receiving module 150 is used to calculate the parallax offset of the same-name point, and finally perform depth calculation and depth compensation to generate high-resolution and high-precision image depth information. Among them, the first light receiving module 140 and the second light receiving module 150 can use a receiving camera, and its photosensitive chip is a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) to collect the speckle pattern of the space to be measured.

[0035] The binocular camera 100 provided in the embodiment of the present application provides stable support for the dot matrix projection module 130, the first light receiving module 140 and the second light receiving module 150 through the substrate and the fixing frame 120 arranged on the substrate, so as to ensure the stability of the relative positions between the dot matrix projection module 130, the first light receiving module 140 and the second light receiving module 150. By setting the outgoing light paths of the two dot matrix projection modules 130 arranged on the fixing frame 120 at a preset angle, it is beneficial to improve the field of view of the binocular camera 100 compared with the use of a single projector. When the field of view of the binocular camera 100 is increased, it is beneficial for the first light receiving module 140 and the second light receiving module 150 to receive a larger range of speckle pattern information, expand the depth reconstruction range of the binocular camera 100, and thus improve the spatial three-dimensional reconstruction capability.

[0036] like Figure 1 As shown, the fixing frame 120 is an isosceles triangle structure, and the two dot matrix projection modules 130 are respectively located on two opposite sides of the isosceles triangle.

[0037] Specifically, the fixing frame 120 is an isosceles triangle structure, that is, the fixing frame 120 adopts an isosceles triangle bracket, so that the structure of the fixing frame 120 is more stable and reliable. In addition, by locating the two dot matrix projection modules 130 on the two opposite waists of the isosceles triangle, the dot matrix projection module 130 and the fixing frame 120 can be stably connected. When installing and fixing, the outgoing light path of the dot matrix projection module 130 can be perpendicular to the waist of the isosceles triangle. In this way, the base angle of the isosceles triangle determines the size of the preset angle of the outgoing light paths of the two dot matrix projection modules 130. When assembling different models of binocular cameras 100, the required outgoing light path angle can be adjusted by replacing different fixing frames 120, which is conducive to simplifying the assembly structure, reducing the difficulty of operation, and improving assembly efficiency.

[0038] like Figure 2 As shown, the binocular camera 100 also includes a closed shell 160 and a transparent cover 170 arranged on one side of the shell. The substrate, the fixing frame 120, the dot matrix projection module 130, the first light receiving module 140 and the second light receiving module 150 are all located in the closed shell 160, and the substrate and the transparent cover 170 are arranged in parallel.

[0039] Specifically, the above-mentioned form is advantageous for protecting components such as the dot matrix projection module 130, the first light receiving module 140, and the second light receiving module 150 through the transparent cover 170 and the closed shell 160, thereby ensuring the stability of the binocular camera 100 during use, such as sealing to prevent the entry of dust or water vapor, thereby avoiding interference from the external environment. It should be noted that the embodiment of the present application does not impose any specific restrictions on the setting form of the closed shell 160. For example, the closed shell 160 can be cylindrical, truncated cone, or other shapes, as long as it can ensure that the field of view of the binocular camera 100 is not affected and the line of sight of the first light receiving module 140 and the second light receiving module 150 is not blocked.

[0040] like Figure 2 and Figure 3 As shown, the distance between the two dot matrix projection modules 130 is Wherein, d is the distance between the two dot matrix projection modules 130 , 2β is the field of view of the dot matrix projection module 130 , 2θ is the field of view of the binocular camera 100 , and h is the minimum application distance expected by the binocular camera 100 .

[0041] Specifically, Figure 3 yes Figure 2 The simplified geometric model of the dot matrix projection module 130 and the fixing frame 120 is shown in FIG. 1 . Assume that the two dot matrix projection modules 130 are Figure 3 Point B and point E in the diagram. The plane where the straight line GN is located is the plane where the transparent cover 170 is located, and the fixing frame 120 is an isosceles triangle of △JCI. Among them, ∠GBF and ∠NEF are the field of view angles of the two dot matrix projectors 132, respectively, set to 2β, straight line BP and straight line EM are the angle bisectors of ∠GBF and ∠NEF, respectively. When the output light path of the dot matrix projection module 130 is perpendicular to the waist of the isosceles triangle, straight line BP and straight line EM are perpendicular to CJ and IJ. Extend straight line GB and straight line NE, and compare them to point A. From the geometric relationship, it can be seen that the intersection points F, J, and A are on the same straight line. ∠GAN is the field of view angle required for the final splicing, set to 2θ. The edge rays BF and EF of the two dot matrix projectors 132 will intersect at point F, which means that the minimum application distance of the product is FJ, set to h, otherwise there will be an area without scattered spots (such as the area surrounded by FBJE as shown in the figure), which makes it impossible to perform depth reconstruction.

[0042] Under the premise of knowing the field of view 2β of a single dot matrix projector 132, the final splicing field of view 2θ and the minimum application distance h, the isosceles triangle angle ɑ( Figure 3 ∠JCI) in and the distance BE between the two dot matrix projection modules 130.

[0043] From the geometric relationship of triangles, we can know that:

[0044] ∠PBJ=∠PBF+∠FBJ (1)

[0045] ∠BJF=∠BDJ+∠DBJ (2)

[0046] ∠ABE+∠EBJ+∠PBJ+∠GBP=180 (3)

[0047] ∠JCI=∠JBE (4)

[0048] ∠FBJ+∠BJF+∠BFJ=180 (5)

[0049] Substituting ∠PBJ=90 into the above formula, we can get:

[0050] ∠BFJ=2β-θ, ∠FBJ=90-β, ∠JCI=α=θ-β

[0051] In ΔBFJ, according to the triangle sine theorem:

[0052]

[0053] From formula (6), we can know that:

[0054]

[0055] In ΔBDJ, there is the identity:

[0056]

[0057] From formula (7) and formula (8), it can be seen that the distance d between the two dot matrix projection modules 130 is:

[0058]

[0059] It can be seen from the above formula that as long as the field of view 2β of a single dot matrix projection module 130, the desired spliced ​​field of view 2θ and the desired minimum application distance h are determined, the relative distance between the isosceles triangle fixing frame 120 and the left and right dot matrix projection modules 130 can be designed according to the above formula. Among them, the desired minimum application distance h can be determined according to the installation application environment of the product.

[0060] like Figure 1 and Figure 4 As shown, the dot matrix projection module 130 includes at least one dot matrix projector 132 ; when the number of the dot matrix projectors 132 is greater than or equal to two, the dot matrix projectors 132 of each dot matrix projection module 130 are on the same straight line, and the two straight lines are parallel to each other.

[0061] Specifically, each dot matrix projection module 130 may include only one dot matrix projector 132, or each dot matrix projection module 130 may have two or more dot matrix projectors 132 according to the complexity of the object, which is conducive to improving the density of the speckle pattern per unit area, thereby improving the three-dimensional reconstruction capability. It can be understood that the area projected by the dot matrix projector 132 is generally rectangular. When the dot matrix projectors 132 of each dot matrix projection module 130 are on the same straight line and the two straight lines are parallel to each other, the projection area between the two dot matrix projection modules 130 can be better distributed to improve the utilization rate of the projection beam and avoid the lack of beam projection area at the light outlet of the binocular camera 100.

[0062] In an optional embodiment of the present application, the dot matrix projector 132 includes a light source, and a collimating lens and a diffraction optical element located on the outgoing light path of the light source.

[0063] Specifically, the light source can be any one of a light emitting diode (LED), a semiconductor laser (LD), and a vertical cavity surface emitting laser (VCSEL). The light beam emitted by the light source is collimated by a collimating lens to make the light beam emitted in parallel, and then undergoes the shaping diffraction effect of a diffractive optical element to form a specific speckle pattern.

[0064] like Figure 5 As shown, the fixing frame 120 includes positioning seats 122 arranged at intervals, and the two dot matrix projection modules 130 are respectively located on the positioning surfaces 1222 of the positioning seats 122, and the two positioning surfaces 1222 respectively coincide with the two waists of the isosceles triangle.

[0065] Specifically, the above-mentioned form can be considered as cutting off the unnecessary positions of the isosceles triangle, and only retaining the two corners of the triangle to reduce costs. It is understandable that the positioning seat 122 can also be in the form of a right-angled trapezoid, so as to increase the height of the setting position of the dot matrix projection module 130 according to actual needs.

[0066] like Figure 1 As shown, the first light receiving module 140 and the second light receiving module 150 are respectively located on opposite sides of the two dot matrix projection modules 130. In this way, the first light receiving module 140 and the second light receiving module 150 collect information respectively, and measure the depth information by integrating information of different dimensions.

[0067] Optionally, the fixing frame 120 is made of a heat-conducting material. For example, the fixing frame 120 can be made of copper or aluminum, or can be made of heat-conducting silicon or ceramic, etc., and can be flexibly set according to the actual use environment.

[0068] The present application also discloses a robot, including the binocular camera 100 in the aforementioned embodiment. The robot includes the same structure and beneficial effects as the binocular camera 100 in the aforementioned embodiment. The structure and beneficial effects of the binocular camera 100 have been described in detail in the aforementioned embodiment, and will not be repeated here.

[0069] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A binocular camera, characterized in that: It comprises a substrate, and a fixing frame arranged on the substrate, wherein dot matrix projection modules are arranged at intervals on the fixing frame, the emission light paths of the two dot matrix projection modules are at a preset angle, and the vertical distances between the two dot matrix projection modules and the substrate are equal; the binocular camera further comprises a first light receiving module and a second light receiving module arranged on the substrate, the first light receiving module and the second light receiving module are used to respectively collect light reflection information of the two dot matrix projection modules, the first light receiving module and the second light receiving module are respectively located on opposite sides of the two dot matrix projection modules, and the fixing frame is made of heat conductive material; The fixing frame is in an isosceles triangle structure, and the two dot matrix projection modules are respectively located on two opposite sides of the isosceles triangle; The distance between the two dot matrix projection modules is d= , where d is the distance between the two dot matrix projection modules, 2β is the field of view of the dot matrix projection module, 2θ is the field of view of the binocular camera, and h is the minimum application distance expected by the binocular camera.

2. The binocular camera according to claim 1, characterized in that: The binocular camera also includes a closed shell and a transparent cover plate arranged on one side of the shell. The substrate, the fixing frame, the dot matrix projection module, the first light receiving module and the second light receiving module are all located in the closed shell, and the substrate is arranged parallel to the transparent cover plate.

3. The binocular camera according to any one of claims 1 to 2, characterized in that: The dot matrix projection module includes at least one dot matrix projector; when the number of the dot matrix projectors is greater than or equal to two, the dot matrix projectors of each of the dot matrix projection modules are on the same straight line, and the two straight lines are parallel to each other.

4. The binocular camera according to claim 3, characterized in that: The dot matrix projector includes a light source, and a collimating lens and a diffraction optical element located on the outgoing light path of the light source.

5. The binocular camera according to claim 1, characterized in that: The fixing frame comprises positioning seats arranged at intervals, the two dot matrix projection modules are respectively located on the positioning surfaces of the positioning seats, and the two positioning surfaces respectively coincide with the two waists of an isosceles triangle.

6. A robot, characterized in that: Comprising the binocular camera described in any one of claims 1-5.

Images