A Relative Imaging Measurement Method of a Navigation Camera with Dual-System Visual Fusion

Abstract

The present invention relates to a relative imaging measurement method for a navigation camera with dual-system visual fusion, which is used to obtain relative navigation information when a lunar rover autonomously travels on the lunar surface. This method utilizes the principle of passive imaging of visible light binocular stereo vision to obtain three-dimensional point cloud information and gray-scale information of the lunar surface; it also utilizes the principle of TOF active light ranging to obtain three-dimensional point cloud information of the lunar surface; by fusing the TOF active light imaging system and the binocular visible light passive imaging system, the lunar surface depth is restored in areas where the lunar surface texture is missing or repeated. The present invention can integrate the advantages of the active light and passive light imaging measurement systems to ensure that the lunar rover obtains stable and reliable lunar surface depth and gray-scale information.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving of lunar rovers, and particularly relates to a relative imaging measurement method for a navigation camera with dual-system visual fusion. Background Art

[0002] When the lunar rover drives autonomously on the lunar surface, it is necessary to obtain the surrounding environment of the lunar surface, including grayscale information or depth distance information. Common imaging measurement means include binocular stereo imaging or lidar. Among them, binocular stereo imaging is passive light imaging, and the stereo recovery and lunar surface imaging effect are greatly affected by light. The imaging effect of targets in the shadow area is poor and there is a phenomenon of depth recovery failure. At the same time, due to the lack of texture and the characteristic of texture repetition on the lunar surface itself, the accuracy of depth data recovery using binocular grayscale images is poor and it is easy to fail; Lidar, as an active light measurement system, is not affected by light and can stably and reliably obtain three-dimensional information of the lunar surface. However, due to the disadvantages of large volume and high power consumption of traditional lidar and the inability to obtain grayscale information of the lunar surface, its use on the lunar surface is relatively limited. Summary of the Invention

[0003] The purpose of the present invention is to propose a relative imaging measurement method for a navigation camera with dual-system visual fusion, which fuses the active light and passive light measurement systems to obtain the grayscale and three-dimensional information of the lunar surface to support the autonomous driving of the lunar rover. The present invention also provides a navigation camera that can use the above method, and a lunar rover configured with the above navigation camera.

[0004] In order to achieve the above purpose, a technical solution of the present invention is to provide a relative imaging measurement method for a navigation camera with dual-system visual fusion, which is used to obtain relative navigation information when the lunar rover drives autonomously on the lunar surface, and is characterized in that

[0005] For the data obtained by the heads of two visible light cameras, using the passive imaging principle of visible light binocular stereo vision, three-dimensional point cloud information and grayscale information of the lunar surface are obtained;

[0006] For the data obtained by the head of a TOF camera, using the TOF active light ranging principle, three-dimensional point cloud information of the lunar surface is obtained;

[0007] Fuse the TOF active light imaging system and the binocular visible light passive imaging system, fuse the two sets of three-dimensional point cloud information of the lunar surface, and perform lunar surface depth recovery on the lunar surface texture missing or texture repeating areas.

[0008] Optionally, project the three-dimensional coordinates of the target point obtained by the head of the TOF camera onto the binocular visible light image planes of the heads of the two visible light cameras at the same time, and then select an ROI area near the projection point to extract and match local feature points to obtain the three-dimensional information of the target point.

[0009] Optionally, two visible light camera heads are respectively arranged on both sides of the TOF camera head through a horizontal mounting bracket.

[0010] Another technical solution of the present invention is to provide a navigation camera, which includes two visible light camera heads, a TOF camera head, and an algorithm function module; the algorithm function module further includes:

[0011] A binocular stereo vision depth recovery module, which uses the principle of passive imaging of visible light binocular stereo vision to process the data obtained by the two visible light camera heads to obtain lunar surface three-dimensional point cloud information and grayscale information;

[0012] A TOF camera depth recovery module, which uses the principle of TOF active light ranging to process the data obtained by the TOF camera head to obtain lunar surface three-dimensional point cloud information;

[0013] A dual-mode fusion depth recovery module, which fuses the TOF active light imaging mode and the binocular visible light passive imaging mode, fuses the two sets of lunar surface three-dimensional point cloud information, and performs lunar surface depth recovery on the areas with missing or repeated lunar surface textures.

[0014] Optionally, the dual-mode fusion depth recovery module projects the three-dimensional coordinates of the target point obtained by the TOF camera head onto the binocular visible light image planes of the two visible light camera heads at the same time, and then selects an ROI area near the projection point to extract and match local feature points to obtain the three-dimensional information of the target point.

[0015] Optionally, the navigation camera further includes a mounting bracket; the two visible light camera heads are respectively arranged on both sides of the TOF camera head through a horizontal mounting bracket.

[0016] Optionally, the navigation camera further includes a power supply and information processing box, which is internally provided with a power module, a core processor, and corresponding peripheral circuits; the algorithm function module processes data through the core processor.

[0017] Another technical solution of the present invention is to provide a lunar rover, which is provided with any one of the above navigation cameras; when the lunar rover autonomously travels on the lunar surface, relative navigation information is obtained through the navigation camera.

[0018] The method of the present invention combines the advantages of the TOF active light imaging and binocular passive light imaging dual modes, ensuring that the lunar rover can obtain stable and reliable lunar surface depth and grayscale information for use in lunar surface autonomous driving.

[0019] The advantages and technical effects of the present invention are as follows:

[0020] The working distance is not less than 50m; it is less affected by light; it can obtain lunar surface gray-scale images, restore the lunar surface three-dimensional point cloud data within this distance in the field of view, fuse the TOF active light module and the binocular visible light module, and restore the three-dimensional point cloud data of the missing or repeated texture areas on the lunar surface. Brief Description of the Drawings

[0021] Figure 1 It is a schematic diagram of the navigation camera;

[0022] Figure 2 It is a schematic diagram of binocular camera depth restoration;

[0023] Figure 3 It is a schematic diagram of TOF camera depth restoration,

[0024] Figure 4 It is a schematic diagram of the dual-system fusion. Detailed Implementation Manner

[0025] When the lunar rover is driving autonomously on the lunar surface, after obtaining the gray-scale and depth information of the lunar surface by using the navigation camera, the navigation information is analyzed, which is the measurement means required for the lunar rover to obtain navigation information during autonomous driving on the lunar surface.

[0026] The present invention provides a relative imaging measurement method for a navigation camera with dual-system fusion, which fuses the TOF (Time of Flight) active light and binocular passive light imaging mechanisms and is used to obtain relative navigation information when the lunar rover is driving autonomously on the lunar surface. The present invention also provides a navigation camera that can implement the above method, as well as a lunar rover equipped with this navigation camera.

[0027] In the method of this embodiment, the visible light binocular stereo vision passive imaging principle is used to obtain the lunar surface three-dimensional point cloud information and gray-scale information; and the TOF active light ranging principle is used to obtain the lunar surface three-dimensional point cloud information; through the active and passive fusion system of binocular visible light and TOF cameras, the lunar surface depth restoration is performed on the missing or repeated texture areas of the lunar surface. It mainly completes functions such as lunar surface environment perception, three-dimensional reconstruction of the lunar surface, and scene monitoring within a range of not less than 50m.

[0028] As Figure 1 shown, in the navigation camera using the above method, the hardware part includes a TOF camera module, a binocular visible light module, a mounting bracket 4, and a power supply and information processing box 5.

[0029] The binocular visible light module includes two visible light camera heads 2; the TOF camera module includes one TOF camera head 3. These two visible light camera heads 2 are arranged on both sides of the TOF camera head 3 through a horizontal mounting bracket 4. Exemplarily, the visible light camera head 2 mainly includes parts such as an optical lens, a light shield, and electronics; the TOF camera head 3 mainly includes parts such as an optical lens, a light shield, a laser, and electronics; these parts can be configured accordingly according to the technical knowledge in the art and will not be listed one by one. The mark 1 schematically represents the lunar rover 1.

[0030] The power supply and information processing box 5 is provided with a power supply and information processing module, which includes a power module, a core processor, and corresponding peripheral circuits. The algorithm function modules in the core processor include a binocular stereo vision depth recovery module, a TOF camera depth recovery module, and a dual-system fusion depth recovery module.

[0031] Using the binocular vision system composed of two visible light camera heads (corresponding to the left and right cameras), the three-dimensional data of the scene can be recovered by using feature point extraction and matching technology, and its working principle is as Figure 2 shown. The binocular stereo vision depth recovery module, according to the disparity x l -x r of the target in the left and right cameras, the binocular baseline length b, and the camera focal length information f, can obtain the three-dimensional coordinates (x, y, z) of the target point; among them, (x l , y l ) and (x r , y r ) are the positions of the target point in the image planes of the left and right cameras respectively.

[0032]

[0033]

[0034]

[0035] Since the accuracy of the binocular vision system is affected by the surface texture and the imaging is very dependent on the texture of the object surface, the binocular is more suitable for scenes with rich textures, or an active imaging method - using structured light projection with patterns and solving the problem of corresponding point matching through the deformed patterns reflected back. Therefore, for the scene with relatively less rich lunar surface texture, there are disadvantages such as matching failure and long algorithm time consumption in the feature point extraction and matching stage of the left and right cameras using the scene graph. Therefore, there are certain limitations in only using binocular cameras to recover the three-dimensional point cloud of the scene. Especially when the scene distance is far and the texture information is lost, the binocular vision system will have the situation of algorithm failure.

[0036] To this end, the present invention utilizes the principle of TOF active optical ranging to obtain lunar surface three-dimensional point cloud information, and through the fusion of active and passive systems, the lunar surface depth is restored in areas with missing or repeated textures on the lunar surface.

[0037] The TOF camera head uses a TOF detector to measure the distance to an object by controlling the exposure and sampling circuits. When it works, it acquires image data at phases of 90°, 180°, 270°, and 360° respectively; as Figure 3 shown, the TOF camera depth recovery module calculates the phase difference between the transmitted light and the received light based on these four frames of image data.

[0038] The light intensities of the four frames of images DCS0, DCS1, DCS2, and DCS3 are:

[0039] DCS0:

[0040] DCS1:

[0041] DCS2:

[0042] DCS3:

[0043] where A is the amplitude of the received signal energy;

[0044] The calculation formula of the azimuth function atan2 is as shown in Equation (1), and the output angle range is (-π, π).

[0045]

[0046] From this, the time of light transmission can be calculated according to formula (2), and then the distance can be calculated according to formula (3).

[0047]

[0048]

[0049] where f LED is the frequency of the LED, c is the speed of light, DCS0 to DCS3 are detector data, t OFFSET and D OFFSET are the offsets of time and distance respectively.

[0050] The TOF camera head can provide the three-dimensional coordinate information of the scene target points. The dual-system fusion depth recovery module further fuses the data of the TOF camera head and the binocular visible light module: The specific principle is as Figure 4As shown, the three-dimensional coordinates of the target points obtained by the TOF camera head 3 are simultaneously projected onto the binocular visible light image plane 21, and then an ROI region 6 is selected near the projection point 7 to perform local feature point extraction and matching to obtain the three-dimensional information of this point.

[0051] Although the content of the present invention has been introduced in detail through the above preferred embodiments, it should be recognized that the above description should not be regarded as a limitation of the present invention. After those skilled in the art have read the above content, various modifications and alternatives to the present invention will be obvious. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims

1. A relative imaging measurement method for a navigation camera with dual - system visual fusion, which is used to obtain relative navigation information when a lunar rover autonomously travels on the lunar surface. Characterized in that, For the data obtained by the heads of two visible - light cameras, using the principle of passive imaging of visible - light binocular stereo vision, three - dimensional point cloud information and gray - scale information of the lunar surface are obtained. For the data obtained by the head of a TOF camera, using the principle of TOF active light ranging, three - dimensional point cloud information of the lunar surface is obtained. Fusing the TOF active - light imaging system and the binocular visible - light passive imaging system, fusing the two sets of three - dimensional point cloud information of the lunar surface, and performing lunar surface depth recovery for areas with missing or repeated lunar surface textures.

2. The relative imaging measurement method for a navigation camera with dual - system visual fusion according to claim 1, Characterized in that, The three - dimensional coordinates of the target point obtained by the TOF camera head are simultaneously projected onto the binocular visible - light image planes of the heads of the two visible - light cameras, and then an ROI area is selected near the projection points for local feature point extraction and matching to obtain the three - dimensional information of the target point.

3. The relative imaging measurement method for a navigation camera with dual - system visual fusion according to claim 1 or 2, Characterized in that, The heads of the two visible - light cameras are respectively arranged on both sides of the TOF camera head through a horizontal mounting bracket.

4. A navigation camera, Characterized in that, It includes two visible - light camera heads, a TOF camera head, and an algorithm function module; the algorithm function module further includes: A binocular stereo vision depth recovery module, which uses the principle of passive imaging of visible - light binocular stereo vision to process the data obtained by the heads of the two visible - light cameras to obtain three - dimensional point cloud information and gray - scale information of the lunar surface. A TOF camera depth recovery module, which uses the principle of TOF active light ranging to process the data obtained by the TOF camera head to obtain three - dimensional point cloud information of the lunar surface. A dual - system fusion depth recovery module, which fuses the TOF active - light imaging system and the binocular visible - light passive imaging system, fuses the two sets of three - dimensional point cloud information of the lunar surface, and performs lunar surface depth recovery for areas with missing or repeated lunar surface textures.

5. The navigation camera according to claim 4, Characterized in that, The dual - system fusion depth recovery module projects the three - dimensional coordinates of the target point obtained by the TOF camera head onto the binocular visible - light image planes of the heads of the two visible - light cameras simultaneously, and then selects an ROI area near the projection points for local feature point extraction and matching to obtain the three - dimensional information of the target point.

6. The navigation camera according to claim 4, Characterized in that, It further includes a mounting bracket; the heads of the two visible - light cameras are respectively arranged on both sides of the TOF camera head through a horizontal mounting bracket.

7. The navigation camera according to claim 4 or 5, Characterized in that, It further includes a power supply and information processing box, which is internally provided with a power module, a core processor, and corresponding peripheral circuits; the algorithm function module processes data through the core processor.

8. A lunar rover, Characterized in that, It is provided with the navigation camera according to any one of claims 4 to 7; when the lunar rover autonomously travels on the lunar surface, relative navigation information is obtained through the navigation camera.

Images