Single-sensor dual-channel stereoscopic vision module and system

Abstract

The invention discloses a single-sensor dual-channel stereoscopic vision module and system, and belongs to the technical field of optical imaging, the module comprises a first reflection element group, a first optical filter, a second reflection element group, a second optical filter and a light combination prism; the first incident light passes through the first branch light path, and specifically passes through the first reflection element group and the first light filter to be transmitted to the light combination prism; the second incident light passes through a second branch light path, and specifically passes through a second reflection element group and a second light filter to be transmitted to the light combination prism; the light combination prism combines the first incident light and the second incident light which are filtered and have different wavelengths into a beam of dual-channel imaging light and projects the dual-channel imaging light to a subsequent imaging device; the incident points of the two incident lights are different, and the projection centers of the two branch light paths are different and have the same optical path. The device is simple in structure, simple in treatment process, good in adaptability and low in manufacturing cost.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of optical imaging, and particularly relates to a single-sensor dual-channel stereo vision module and system. BACKGROUND

[0002] The core principle of a binocular stereo vision system (BSVS) is to simulate human eyes through two cameras with spatially separated positions, and to reconstruct three-dimensional information of an object by calculating the disparity between images. Ranging is an important application of the binocular stereo vision system. In the ranging process of the traditional binocular stereo vision system, two images with disparity are first collected by using a binocular camera, then binocular camera calibration is performed to determine the internal parameters (such as distortion coefficients of the two cameras) and external parameters (such as relative translation matrix and rotation matrix of the two cameras) of the system, the collected two images are corrected and feature matched based on the calibrated parameters, a disparity map is constructed taking one of the images as a base map, a depth map of the shooting scene is generated based on the disparity map, and the depth information of the shooting scene is calculated based on the depth value.

[0003] However, this technology has the following disadvantages: (1) In general binocular stereo vision modeling tasks, external parameter calibration of the binocular system needs to be realized through a calibration board and calibration software. (2) There is a slight time delay in image acquisition and return of the left and right image sensors, which will cause the target object in the left and right images to have an incorrect corresponding relationship in high-sophisticated and high-dynamic application scenarios, and then cause the accuracy of point cloud generation and ranging to decrease or fail. (3) The pictures returned by the left and right image sensors need to be processed synchronously, which puts high requirements on the processing performance of the processing system. (4) The system is composed of two image sensors, so the back-end processing system needs to have at least two high-speed transmission interfaces for receiving image data returned by the image sensors. (5) The consistency of the two cameras is required to be very high. The two modules need to be as consistent as possible, and the optical axes need to point to the same direction. The traditional binocular vision system is prone to cause position and angle changes between the two lenses of the binocular camera during transportation and long-term use, which reduces the consistency and causes the depth detection accuracy to decrease. This puts high requirements on the manufacturing process and product control consistency during mass production of the product. The formed deviation needs to be compensated by an algorithm. The use of software capability to correct the deviation at the hardware level puts requirements on the algorithm and computing power.

[0004] Therefore, the traditional binocular vision system has the disadvantages of bulky system structure, complex processing process, weak adaptability, and high manufacturing cost. SUMMARY

[0005] The single-sensor dual-channel stereo vision module and system provided by the embodiments of the present application can solve the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a single-sensor dual-channel stereo vision module, comprising: a first reflective element group, a first filter, a second reflective element group, a second filter, and a light-combining prism; The first reflective element group and the first filter are sequentially disposed on one side of the light combining prism along the first direction, and the second reflective element group and the second filter are sequentially disposed on the other side of the light combining prism in the opposite direction. The first reflective element group is used to propagate the first incident light onto the first filter, and the first filter is used to filter the first incident light to obtain filtered first incident light; the second reflective element group is used to propagate the second incident light onto the second filter, and the second filter is used to filter the second incident light to obtain filtered second incident light; the beam combining prism is used to combine the filtered first incident light and the filtered second incident light into a single dual-channel imaging beam. The first incident light and the second incident light have different incident points. The optical path lengths of the first incident light from its incident point to the first branch of the beam combining prism and the second incident light from its incident point to the second branch of the beam combining prism are equal. The first projection center and the second projection center are not at the same location, and the distance between the beam combining prism and the first projection center is... The distance between the light-combining prism and the second projection center They are equal; the first projection center is the projection center of the first branch optical path, the second projection center is the projection center of the second branch optical path, and the wavelengths of the filtered first incident light and the filtered second incident light are different.

[0007] Secondly, embodiments of the present invention provide a single-sensor dual-channel stereo vision system, comprising: As described in the first aspect, the single-sensor dual-channel stereo vision module is used to propagate first incident light and second incident light with different incident points through first branch optical paths and second branch optical paths with different projection centers and equal optical paths to an internal beam combining prism; and before inputting the first incident light and second incident light into the beam combining prism, the first incident light and second incident light are filtered according to different filtering wavelengths; the beam combining prism combines the filtered first incident light and filtered second incident light with different wavelengths into a single beam of dual-channel imaging light. An imaging lens assembly, wherein the imaging lens assembly is used to propagate the dual-channel imaging light onto the imaging device; An imaging device, wherein the imaging device is used to obtain a binocular stereo vision image based on the dual-channel imaging light; The processor is used for splitting the binocular stereovision image according to the wavelengths of the filtered first incident light and the filtered second incident light to obtain a first image and a second image; matching the first image and the second image, and constructing a disparity map according to the first image and the second image based on a matching result; and generating a depth map based on the disparity map to determine distance information of a shooting scene according to a depth value of each pixel point on the depth map.

[0008] The beneficial effects of the embodiment of the present application compared with the prior art are as follows: the present application filters two incident lights with different incident points and at least partially overlapping shooting scenes at different wavelengths before the two incident lights reach the light combiner prism, the light combiner prism combines the two lights with different wavelengths into a double-channel imaging light, and the double-channel imaging light is transmitted to the same imaging device; since the incident points and the projection centers of the two light paths are different, the imaging results of different wavelengths based on the double-channel imaging light when the imaging device is imaging have a certain parallax, and the binocular vision image can be obtained; since the optical paths of the two light paths are the same, the two light beams can reach the light combiner prism at the same time, so that the two light beams can be imaged at the same time by one imaging device, the number of imaging devices in the system is reduced, the external parameter calibration process required when two imaging devices are used is avoided, and the consistency of the two imaging devices in the system and the high processing performance of the system are not required; thus, the system structure and the processing process can be simplified, the system adaptability can be improved, and the manufacturing cost of the system can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 It is a schematic diagram of the principle of binocular vision ranging; Figure 2 It is a structural schematic diagram of a single-sensor double-channel stereovision module provided by the present application; Figure 3 It is an imaging principle schematic diagram of a CMOS imaging device provided by the embodiment of the present application; Figure 4 It is a wavelength schematic diagram of different color lights provided by the embodiment of the present application; Figure 5a It is a schematic diagram of a first image provided by the embodiment of the present application; Figure 5b It is a schematic diagram of a second image provided by the embodiment of the present application; Figure 5c It is a schematic diagram of a binocular stereovision image provided by the embodiment of the present application; Figure 6 It is a specific structural diagram of a single-sensor double-channel stereovision module provided by the embodiment of the present application; Figure 7A size schematic diagram of a single-sensor double-channel stereo vision module provided for an embodiment of the present application is shown in FIG. 1. Figure 8 A structure schematic diagram of a single-sensor double-channel stereo vision system provided for an embodiment of the present application is shown in FIG. 2. Figure 9a A schematic diagram of a first image provided for an embodiment of the present application is shown in FIG. 3. Figure 9b A schematic diagram of a second image provided for an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0010] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0011] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or "comprising", when used in the present specification and in the accompanying claims, means the inclusion of the indicated feature, element, step, operation, integer, and / or group of features, elements, steps, operations, integers, and / or groups thereof, but does not exclude the presence or addition of one or more other features, elements, steps, operations, integers, and / or groups thereof.

[0012] It is also to be understood that the terminology "and / or" when used in the present specification and in the accompanying claims, means that the associated listed items are both individually present and / or any combination of one or more of the associated listed items can be present.

[0013] As used in the present specification and in the accompanying claims, the term "if' can be interpreted as meaning "when" or "upon" or "in response to a determination" or "in response to a detection" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted as meaning "upon a determination" or "in response to a determination" or "upon a detection [of the described condition or event]" or "in response to a detection [of the described condition or event]" depending on the context.

[0014] In addition, in the description of the present specification and in the accompanying claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0015] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.

[0016] The principle of binocular vision ranging is explained below with reference to the diagram: Figure 1 The diagram shown illustrates the principle of binocular vision ranging.

[0017] The basic principle of binocular vision relies on the difference in perspective between two cameras (i.e., parallax). By comparing two images taken from different angles, a binocular vision system can calculate the parallax value of each point in the image, thereby estimating the actual distance of the object from the camera.

[0018] As an example, see Figure 1 In a traditional binocular vision system, the projection centers of the two cameras are respectively and The line connecting these two projection centers forms the baseline, and the length of the baseline is... Any point in the captured three-dimensional space. The imaging point of the left camera is The imaging point of the right camera is According to the principle of rectilinear propagation of light, a point in three-dimensional space... It is the intersection of the lines connecting the projection centers of the two cameras to the image points. (Line segment) and These are the distances from the imaging points of the left and right cameras to the left imaging plane, respectively. When a binocular vision system images... Parallax of a point equal .

[0019] For example, parallax is the difference in position of an object in two images taken from different viewpoints. The magnitude of parallax is inversely proportional to the distance of the object from the camera. The greater the parallax, the closer the object is; the smaller the parallax, the farther away the object is.

[0020] In one example, a traditional binocular vision system can determine the position and focal length of two cameras within the system. The vertical distance between an object and a baseline is calculated using the principle of triangulation. .

[0021] For example, the depth of the object (i.e. the vertical distance between the object and the baseline ) can satisfy the following formula: , Specifically, the conventional binocular vision system is to acquire two images with parallax of the same scene at different angles of view by two cameras with different positions, then to determine the internal parameters (such as distortion coefficients of the two cameras, etc.) and external parameters (such as the relative translation matrix and rotation matrix of the two cameras) of the system through binocular camera calibration, to correct and match the features of the two acquired images based on the calibrated parameters, to construct a disparity map based on one of the images as the base map, to generate a depth map of the photographed scene based on the disparity map, and to calculate the depth information of the photographed scene based on the depth value.

[0022] Since this method needs the assistance of a calibration board and calibration software when calibrating the external parameters, and needs two imaging devices and image processing devices, the processing performance of the system and the consistency of the two cameras are required to be high, which leads to the disadvantages of the traditional binocular vision system, such as bulky system structure, complex processing process, weak adaptability, and high manufacturing cost.

[0023] Therefore, the present application provides a single-sensor dual-channel stereo vision module, which makes two incident lights with different incident points pass through two optical paths with different projection centers but the same optical path to reach the imaging device. Since the projection centers and incident points of the two optical paths are different, the imaging device has a certain parallax when imaging the two light beams, and the binocular vision image can be obtained. Since the optical paths of the two optical paths are equal, the two incident light beams can reach the imaging device at the same time, so that the number of imaging devices in the system can be reduced, the external parameter calibration process required when using two imaging devices can be avoided, and the consistency of the two imaging devices in the system and the high processing performance of the system are not required. Therefore, the system structure and processing process can be simplified, the system adaptability can be improved, and the manufacturing cost of the system can be reduced.

[0024] The present application will be further described in detail below in conjunction with specific embodiments, but the embodiments of the present application are not limited thereto.

[0025] Embodiment 1 Figure 2 The structure schematic diagram of the single-sensor dual-channel stereo vision module provided by the present application is shown. As an example but not limitation, the module can include a first reflection element group 210, a first filter 220, a second reflection element group 230, a second filter 240, and a light combination prism 250.

[0026] In one possible implementation, referring to Figure 2 The first reflective element group 210 and the first filter 220 can be sequentially arranged on one side of the light combining prism 250 along a first direction (indicated by the first arrow 201 in Figure 2 The second reflective element group 230 and the second filter 240 can be sequentially arranged on the other side of the light combining prism 250 against the first direction.

[0027] Specifically, the first incident light and the second incident light, which have different incident points but at least partially overlapping scenes, are incident from the first reflective element group 210 and the second reflective element group 230 respectively. The first incident light passes through the first reflective element group 210 and the first filter 220 along a first branch light path to the light combining prism 250, and the second incident light passes through the second reflective element group 230 and the second filter 240 along a second branch light path to the light combining prism. During the propagation, the first reflective element group 210 adjusts the direction of the first incident light to one input direction (for example, the first direction) of the light combining prism 250, and the first filter 220 performs filtering processing on the first incident light; similarly, the second reflective element group 230 adjusts the direction of the second incident light to another input direction (for example, the second direction indicated by the second arrow 202 in Figure 2 The second filter 240 performs filtering processing on the second incident light. Finally, the light combining prism 250 combines the filtered first incident light and the filtered second incident light with different wavelengths into a double-channel imaging light.

[0028] For example, the optical path of the first branch light path and the second branch light path is equal, so that the first incident light and the second incident light can reach the imaging device at the same time.

[0029] For example, the first projection center and the second projection center of the imaging device are not at the same position, and the distance between the light combining prism 250 and the first projection center and the second projection center is equal.

[0030] Specifically, the first projection center can be the projection center of the first branch light path, and the second projection center can be the projection center of the second branch light path.

[0031] In one example, referring to Figure 3 Since the surface structure of a general Complementary Metal Oxide Semiconductor (CMOS) imaging device is a Bayer matrix structure, that is, a plurality of light sensing devices are used to receive red, green and blue spectrum light respectively, and the light received by the group of light sensing devices is combined to obtain a pixel point with RGB color characteristics; therefore, referring to Figure 4The wavelength of the filtered first incident light can be the wavelength 435nm of blue light, and the wavelength of the filtered second incident light can be the wavelength 500nm-800nm of green light to red light.

[0032] For example, the wavelength-different first incident light and the wavelength-different second incident light are projected to the light combiner 250 through the first branch light path and the second branch light path respectively, and the wavelength-different two light beams reaching the light combiner 250 at the same time are combined into a double-channel imaging light by the light combiner 250, and the double-channel imaging light is transmitted to the same imaging device, and the imaging device can image according to the double-channel imaging light to obtain binocular stereoscopic vision images. In subsequent distance measurement, the binocular stereoscopic vision images can be split to realize double-channel imaging, and then combined into three-channel restored color. In this way, left and right perspective pictures (see Figure 5a 、 Figure 5b ) and overlapping field of view pictures (see Figure 5c ) without delay can be obtained by one-time shooting.

[0033] Since only one CMOS imaging device is needed to obtain the double-perspective images based on the module, only the camera intrinsic parameters (i.e., the distortion coefficient of imaging) need to be concerned in the subsequent distance measurement process, and the relative parameters (relative translation relationship and relative rotation relationship) of the two CMOS imaging devices do not need to be concerned, so that the system complexity can be reduced.

[0034] The present application is characterized in that two incident lights with different incident points but at least partially overlapping scenes are projected to a light combiner through two light paths with different projection centers but the same optical path, and the two incident lights are filtered with different wavelengths before reaching the light combiner, and the light combiner combines the two light beams with different wavelengths into a double-channel imaging light transmitted to the same imaging device. Since the two light paths are different in distance and projection center, the imaging results of different wavelengths based on the double-channel imaging light when the imaging device images are different, and binocular vision images can be obtained. Since the optical paths of the two light paths are the same, the two light beams can reach the light combiner at the same time, so that the two light beams can be imaged by one imaging device, the number of imaging devices in the system is reduced, the external parameter calibration process required when using two imaging devices is avoided, and the consistency of the two imaging devices in the system and the high processing performance of the system are not required. Therefore, the system structure and processing process can be simplified, the system adaptability can be improved, and the manufacturing cost of the system can be reduced.

[0035] Embodiment 2 Based on the embodiment 1, Figure 6 The specific structure diagram of the single-sensor double-channel stereovision module provided by the embodiment of the present application is shown.

[0036] In a possible implementation manner, see Figure 2The first direction can be perpendicular to the second direction, and the exit directions of the first branch optical path and the second branch optical path can be the first direction and the second direction, respectively. The incident direction of the two incident beams can also be the second direction.

[0037] In one example, the first reflective element group 210 may include a path along the first branch optical path (see...). Figure 6 The reflective elements F1, F2, F3, F4, and F5 are sequentially arranged along the optical path indicated by the solid black arrow. These reflective elements are used to deflect the first beam at different angles. ,angle ,angle ,angle ,angle Output later.

[0038] For example, if the exit direction of the first branch light path is to be perpendicular to the incident direction of the first incident light, then the angle... ,angle ,angle ,angle ,angle The sum must satisfy , It is a non-negative integer.

[0039] Optionally, angle ,angle ,angle All can be equal to ,angle ,angle All can be equal to At this point, the first projection center is located at the optical center of the reflecting element F1 (see...). Figure 6 (point FOL in the middle).

[0040] For example, see Figure 6 The reflecting element F1 deflects the first incident light clockwise by 90° (i.e., It can be at a 45° angle to the first direction (i.e., The angle; the reflecting element F2 deflects the first beam clockwise by 90° (i.e., The reflector F3 deflects the first beam counterclockwise by 90°, and can itself form a 135° angle with the first direction (i.e., ...). The reflective element F4 deflects the first beam counterclockwise by 90° and can itself form a 45° angle with the first direction; the reflective element F5 deflects the first beam clockwise by 90° and can itself form a 45° angle with the first direction.

[0041] In one example, the second set of reflecting elements 230 can include reflecting element F6, reflecting element F7, reflecting element F8, and reflecting element F9 arranged in sequence along the second branch light path (see the light path indicated by the dotted arrow in FIG. 6B). Figure 6 These reflecting elements are used to deflect the second light beam by angle , angle , angle , angle , angle , angle , angle , angle , angle , angle

[0042] , angle , angle , angle , angle , angle , angle , angle , angle , angle , angle , angle , angle

[0043] , angle , angle , angle , angle , angle , angle , angle Figure 6 , angle , angle

[0044] For example, referring to FIG. 6B, reflecting element F6 deflects the second light beam counterclockwise by 90°, and can form an angle of 135° with the first direction; reflecting element F7 deflects the second light beam counterclockwise by 90°, and can form an angle of 45° with the first direction; reflecting element F8 deflects the second light beam clockwise by 90°, and can form an angle of 45° with the first direction; and reflecting element F9 deflects the second light beam clockwise by 90°, and can form an angle of 135° with the first direction. Figure 6 In one possible implementation, referring to FIG. 6B,

[0045] Figure 7 Figure 7For ease of explanation, the filter is omitted. For ease of manufacturing, the optical centers of reflective elements F1, F2, F5, F6, and F7 can be located on the same horizontal line parallel to the first direction, while the optical centers of reflective elements F4, F3, F8, and F9 can be located on another horizontal line parallel to the first direction.

[0046] Specifically, the distance between the beam combining prism 250 and the reflecting element F9 can be adjusted. The distance between reflective element F7 and reflective element F8 The distance between reflective element F4 and reflective element F5 The distance between reflective element F3 and reflective element F2 All equal to constants .

[0047] In one example, see similarly. Figure 7 For ease of manufacturing, the spacing between reflective element F8 and reflective element F9 can also be adjusted. The distance between the beam combining prism 250 and the reflective element F5 The distance between reflective element F4 and reflective element F3 All equal to constants .

[0048] For example, in Figure 7 In the structure shown, the first projection center of the single-sensor dual-channel stereo vision module is at point FOL, the second projection center is at point FOR, and the baseline is the line connecting the two projection centers with a length of [missing information]. If the baseline is 200mm, the constant is... ,constant They are equal to 10mm and 15mm respectively; then under the condition: (Equal optical path length) (Assuming the distance from the combining prism 250 to the projection center is equal) the spacing between adjacent reflective elements can be calculated: , , , , , , , , .

[0049] The application is characterized in that two incident lights with different incident points but at least partially overlapped scenes are respectively transmitted through two light paths with different projection centers but the same optical path to a light combination prism, and the two incident lights are respectively filtered at different wavelengths before reaching the light combination prism; the light combination prism combines the two lights with different wavelengths into a double-channel imaging light and transmits the double-channel imaging light to the same imaging device; since the incident points and the projection centers of the two light paths are different, the imaging results of the double-channel imaging light based on the double-channel imaging light are different in wavelength, and binocular vision images can be obtained; since the optical paths of the two light paths are the same, the two light beams can reach the light combination prism at the same time, so that the two light beams can be imaged by one imaging device, the number of imaging devices in the system is reduced, the external parameter calibration process required when two imaging devices are used is avoided, and the consistency of the two imaging devices in the system and the high processing performance of the system are not required; thus, the system structure and the processing process can be simplified, the system adaptability can be improved, and the manufacturing cost of the system can be reduced.

[0050] Embodiment 3 Figure 8 A structure schematic diagram of a single-sensor double-channel stereo vision system provided by an embodiment of the application is shown. As an example but not limitation, the system can include a single-sensor double-channel stereo vision module 810, an imaging lens group 820, an imaging device 830, and a processor 840.

[0051] In some embodiments, the single-sensor double-channel stereo vision module 810 can be the single-sensor double-channel stereo vision module provided by the above-mentioned embodiment 1 / embodiment 2, which is used to transmit a first incident light and a second incident light with different incident points to an internal light combination prism through branch light paths with different projection centers and the same optical path; and the first incident light and the second incident light are filtered according to different filter wavelengths before being input to the light combination prism; the internal light combination prism combines the filtered first incident light and the filtered second incident light with different wavelengths into a double-channel imaging light; the imaging lens group 820 can transmit the double-channel imaging light to the imaging device 830, and the imaging device 830 can image based on the double-channel imaging light to obtain binocular stereo vision images. The processor 840 can split the binocular stereo vision images according to the wavelengths of the filtered first incident light and the second incident light to obtain a first image (see the imaging diagram of three left-channel first incident lights in FIG. 11) and a second image (see the imaging diagram of three second incident lights in FIG. 11), then match the two images, construct a disparity map based on the matched images, generate a depth map according to the disparity map, and determine the distance information of the shooting scene according to the depth value of each pixel point on the depth map. Figure 9a Figure 9b

[0052] ​​In one example, the processor 840 can find the corresponding point of each pixel in the first image in the second image based on a stereo matching algorithm, to obtain the corresponding relationship of the pixel points; and based on the corresponding relationship, obtain the disparity between the corresponding pixel points (equal to the difference between the column coordinates of the two pixel points).

[0053] In an example, the corresponding pixel points in the first image and the second image have the same shooting target.

[0054] It should be understood that not all pixel points in the first image necessarily have a corresponding relationship.

[0055] In an example, the commonly used stereo matching algorithm that can be implemented on the Open Source Computer Vision Library (OpenCV) can be basically divided into two categories: local algorithm and non-local (i.e., global) algorithm. The local algorithm has a small amount of calculation, but the matching quality is relatively low; the global algorithm omits the cost aggregation and adopts the method of optimizing the energy function, and has a higher matching quality, but also has a larger amount of calculation.

[0056] Optionally, the processor 840 can use the Semi-Global Block Matching (SGBM) algorithm in the local algorithm to perform matching. The cost part of this algorithm uses the BT (Birchfield-Tomasi) cost with certain pixel sampling invariance, and covers the cost aggregation method, and has a better processing effect.

[0057] In one example, the depth value of each target in the shooting scene (equal to the distance from the position of the target to the system baseline) satisfies the following formula: , wherein, , are the column coordinates of the pixel points of the target in the second image and the first image, respectively.

[0058] The application filters two incident lights with different incident points but at least partially overlapping scenes and different wavelengths respectively before the two incident lights reach a light combiner through two light paths with different projection centers but the same optical path, and the light combiner combines the two lights with different wavelengths into a double-channel imaging light and transmits the double-channel imaging light to the same imaging device; since the incident points and the projection centers of the two light paths are different, the imaging results of different wavelengths based on the double-channel imaging light when the imaging device images are different, and binocular vision images can be obtained; since the optical paths of the two light paths are the same, the two light beams can reach the light combiner at the same time, so that the two light beams can be imaged at the same time through one imaging device, the number of imaging devices in the system is reduced, the external parameter calibration process required when two imaging devices are used is avoided, and the consistency of the two imaging devices in the system and the high processing performance of the system are not required; thus, the system structure and the processing process can be simplified, the system adaptability can be improved, and the manufacturing cost of the system can be reduced.

[0059] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

Claims

1. A single-sensor dual-channel stereo vision module, characterized by, The first reflection element group, the first filter, the second reflection element group, the second filter and the light combination prism are sequentially arranged on one side of the light combination prism along a first direction, and the second reflection element group and the second filter are sequentially arranged on the other side of the light combination prism against the first direction. The first reflection element group is configured to propagate first incident light to the first filter, and the first filter is configured to filter the first incident light to obtain filtered first incident light. The second reflection element group is configured to propagate second incident light to the second filter, and the second filter is configured to filter the second incident light to obtain filtered second incident light. The first incident light and the second incident light have different incident points, the optical path of the first incident light from the incident point to the first branch light path of the light combiner is equal to the optical path of the second incident light from the incident point to the second branch light path of the light combiner, the first projection center and the second projection center are not in the same position, and the distance between the light combiner and the first projection center is equal to the distance between the light combiner and the second projection center . The first projection center is the projection center of the first branch light path, the second projection center is the projection center of the second branch light path, and the filtered first incident light and the filtered second incident light have different wavelengths.

2. The module of claim 1, wherein, The light combination prism is configured to combine the filtered first incident light and the filtered second incident light into a double-channel imaging light.

3. The module of claim 2, wherein, The first branch light path and the second branch light path are arranged along the first direction and a second direction, respectively. The reflection elements F1, F2, F3, F4, F5 are respectively used to deflect the first incident light by an angle , , , , , , , , , , , is a non-negative integer.

4. The module of claim 3, wherein, the angle , the angle , the angle is equal to , the angle , the angle is equal to .

5. The module of claim 4, wherein, The first direction is perpendicular to the second direction. The reflection element F6, the reflection element F7, the reflection element F8, and the reflection element F9 are respectively used to deflect the second incident light by an angle , an angle , an angle , an angle , an angle , an angle , an angle , an angle satisfies , is a non-negative integer.

6. The module of claim 5, wherein, the angle , the angle is equal to , the angle , the angle is equal to .

7. The module of claim 6, wherein, The distance between the reflecting element F9 and the reflecting element F8 The distance between the light combining prism and the reflecting element F5 The distance between the reflecting element F4 and the reflecting element F3 are equal to a constant The distance between the light combining prism and the reflecting element F9 The distance between the reflecting element F7 and the reflecting element F8 The distance between the reflecting element F4 and the reflecting element F5 The distance between the reflecting element F3 and the reflecting element F2 are equal to another constant .

8. The module of claim 7, wherein, The distance between the first projection center and the second projection center is 200 mm, the constant , the constant is equal to 10 mm, 15 mm, respectively.

9. The module of claim 1, wherein, The first reflection element group includes reflection elements F1, F2, F3, F4 and F5 arranged along the first branch light path.

10. A single-sensor dual-channel stereo vision system, characterized by, The second reflection element group includes reflection elements F6, F7, F8 and F9 arranged along the second branch light path. The wavelength of the filtered first incident light is 435 nm, and the wavelength of the filtered second incident light is 500 nm-800 nm. The single-sensor double-channel stereo vision module according to any one of claims 1-9 is configured to propagate first incident light and second incident light with different incident points to an internal light combination prism through first branch light paths and second branch light paths with different projection centers and equal optical paths, respectively. The light combination prism combines the filtered first incident light and the filtered second incident light with different wavelengths into a double-channel imaging light. An imaging lens group is configured to propagate the double-channel imaging light to an imaging device. The imaging device is configured to image based on the double-channel imaging light to obtain a binocular stereo vision image. The processor is configured to split the binocular stereovision image according to wavelengths of the filtered first incident light and the filtered second incident light to obtain a first image and a second image, match the first image and the second image, and construct a disparity map according to the first image and the second image based on a matching result, and generate a depth map based on the disparity map to determine distance information of a shooting scene according to a depth value of each pixel point on the depth map.

Images