A multi-mode depth camera and mobile platform

Abstract

A multi-mode depth camera is characterized by comprising: a first projector for projecting pulsed laser signals; a second projector for projecting multiple discrete collimated beams; a receiver for receiving the pulsed laser signals and the reflected signals of the multiple discrete collimated beams; and a processor for obtaining first Time-of-Flight (TOF) depth data based on the time difference of the pulsed laser signals, obtaining second TOF depth data based on the phase difference of the multiple discrete collimated beams, or obtaining structured light depth data based on parallax, and controlling the switching between first mode, second mode, and third mode of the first projector, the second projector, and the receiver. This invention enables the depth camera to automatically select different modes according to different distances, improving the measurement range and accuracy.

Description

Technical Field

[0001] This invention relates to the field of depth measurement technology, and more specifically, to a multi-mode depth camera and mobile platform. Background Technology

[0002] Depth measurement technology can obtain depth data of a target object, thereby generating 3D images with depth information, such as RGBD. Compared to 2D images, 3D images contain more data, enabling better target identification. Due to technological advancements, 3D images are already being used in fields such as distance detection, facial recognition, 3D modeling, and VR. Depth measurement technology primarily utilizes Time-of-Flight (TOF) and structured light technologies, making it adaptable to various application scenarios.

[0003] TOF stands for Time-of-Flight. TOF technology is a precise ranging technique that measures the round-trip time of a light pulse between a transmitting / receiving device and a target object. It is further divided into direct ranging (d-TOF) and indirect ranging (i-TOF). Indirect ranging (i-TOF) measures the phase delay of the reflected light signal relative to the emitted light signal, and then calculates the time of flight from this phase delay. Based on the modulation and demodulation method, it can be divided into continuous wave (CW) modulation and demodulation methods and pulse modulation (PM) modulation and demodulation methods. TOF ranging technology does not require complex image processing calculations and can maintain high accuracy over long distances.

[0004] Structured light technology involves emitting a structured light beam towards a spatial object, then capturing the structured light pattern formed by the modulated and reflected beam. Finally, triangulation is used to calculate the object's depth. Commonly used structured light patterns include irregular spot patterns, stripe patterns, and phase-shifting patterns. Structured light technology offers very high accuracy at close range and performs well in low-light environments, but it is susceptible to interference in strong light. In contrast, Time-of-Flight (TOF) technology exhibits better anti-interference performance in strong light environments than structured light technology. Summary of the Invention

[0005] To address this, the present invention employs a first projector and a second projector to emit pulsed laser signals and multiple discrete collimated beams, respectively, and switches between a first mode, a second mode, and a third mode. This allows the depth camera to automatically select different modes based on different distances, thereby improving the measurement range and accuracy.

[0006] In a first aspect, the present invention provides a multi-mode depth camera, characterized in that it comprises:

[0007] The first projector is used to project pulsed laser signals;

[0008] The second projector is used to project multiple discrete collimated beams;

[0009] A receiver for receiving the pulsed laser signal and the reflected signals of the multiple discrete collimated beams;

[0010] The processor is configured to obtain first TOF depth data based on the time difference of the pulsed laser signal, obtain second TOF depth data based on the phase difference of the multiple discrete collimated beams or obtain structured light depth data based on parallax, and control the switching between first mode, second mode and third mode of the first projector, the second projector and the receiver.

[0011] In the first mode, the first projector projects a pulsed laser signal, the receiver receives the pulsed laser signal, and the processor obtains the first TOF depth data through the time difference;

[0012] In the second mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the second TOF depth data;

[0013] In the third mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the structured light depth data; when the processor obtains the structured light depth data, it also simultaneously obtains the second TOF depth data and uses it for matching region localization of the structured light depth data.

[0014] Optionally, the multi-mode depth camera is characterized in that, when switching between the first mode, the second mode and the third mode, the nearest distance value is used.

[0015] Optionally, the multi-mode depth camera is characterized in that it determines whether a mode switch is needed based on the first TOF depth data in the first mode, the second TOF depth data in the second mode, and the structured light depth data in the third mode.

[0016] Optionally, the multi-mode depth camera is characterized in that the signal from one of the first projectors or the second projector is received by at least two of the receivers.

[0017] Optionally, the multi-mode depth camera is characterized in that one of the receivers receives signals from at least two of the first projectors or the second projector.

[0018] Optionally, the multi-mode depth camera is characterized in that, in the third mode, when the processor obtains the structured light depth data, it includes:

[0019] Step S1: Project a laser signal onto the target object and receive the reflected signal from the target object; wherein the reflected signal includes a first signal and a second signal, the first signal is used to acquire the second TOF depth data, the second signal is used to acquire the structured light depth data, and the second TOF depth data and the structured light depth data are in one-to-one correspondence;

[0020] Step S2: Determine the value of the second TOF depth data one by one according to the position on the depth map;

[0021] Step S3: If the value of the second TOF depth data is in the first range, then the value of the TOF depth data is used as the final depth value of the location;

[0022] Step S4: If the value of the second TOF depth data is in the second range, then calculate the structured light depth data at the location and use it as the final depth value at the location;

[0023] Step S5: Generate a depth fusion depth map based on the final depth value.

[0024] Optionally, the multi-mode depth camera is characterized in that, in step S4, calculating the structured light depth data at the location and using it as the final depth value at the location includes:

[0025] Step S41: Obtain the second TOF depth data at the location as the initial depth value of the structured light at the location, and then calculate the disparity corresponding to the initial depth value of the structured light based on the known parameters;

[0026] Step S42: Set a window for matching near the parallax; the window is determined based on the second TOF depth data;

[0027] Step S43: Calculate the optimal matching score position within the window as the final matching result.

[0028] Optionally, in the multi-mode depth camera, the window is centered at the location and extended to obtain a boundary by a second threshold; wherein the second threshold is inversely proportional to the accuracy of the second TOF depth data.

[0029] Secondly, the present invention provides a mobile platform, characterized in that it includes:

[0030] ontology; and

[0031] A multi-mode depth camera as described in any of the preceding claims.

[0032] Optionally, the mobile platform is characterized in that the body is a vehicle body, a drone fuselage, a robot body, or a ship body.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] This invention selects different modes based on different distances, combining structured light technology with TOF technology to improve the effective measurement range and achieve high measurement accuracy over a large range.

[0035] This invention utilizes TOF depth data for processing, which means that only a portion of the structured light depth data needs to be calculated, thereby saving the computing power required for structured light image calculation, reducing resource consumption, and at the same time, obtaining accurate depth data relatively quickly.

[0036] This invention utilizes the advantages of TOF depth data acquisition—fast speed and no need for complex calculations—to reduce the amount of structured light depth data processing by processing TOF depth data, thereby saving time, increasing image output speed, and reducing response time.

[0037] This invention utilizes the strong anti-interference ability of Time-of-Flight (TOF) against strong light to limit the matching range of structured light data, thereby improving the matching effect of structured light data and enhancing the accuracy of structured light data under strong light. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0039] Figure 1 This is a schematic diagram of the structure of a multi-mode depth camera according to an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram illustrating time consumption in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of a multi-mode depth camera arrangement in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of another multi-mode depth camera arrangement in an embodiment of the present invention;

[0043] Figure 5 This is a flowchart illustrating the steps of a processor processing structured light depth data in an embodiment of the present invention.

[0044] Figure 6 This is a flowchart illustrating the steps for calculating the final depth value in an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of a drone structure according to an embodiment of the present invention;

[0046] Figure 8 This is a schematic diagram of a robot structure according to an embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of the structure of an intelligent vehicle according to an embodiment of the present invention. Detailed Implementation

[0048] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0049] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0051] The present invention provides a multi-mode depth camera, which aims to solve the problems existing in the prior art.

[0052] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0053] The multi-mode depth camera provided in this embodiment of the invention emits pulsed laser signals and multiple discrete collimated beams respectively by setting a first projector and a second projector, and switches between a first mode, a second mode and a third mode. This allows the depth camera to automatically select different modes according to different distances, thereby improving the measurement range and accuracy, and also improving the anti-interference of structured light depth data against strong light.

[0054] Figure 1 This is a schematic diagram of the structure of a multi-mode depth camera according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a multi-mode depth camera comprising:

[0055] The first projector 100 is used to project pulsed laser signals;

[0056] The second projector 200 is used to project multiple discrete collimated beams;

[0057] Receiver 300 is used to receive the pulsed laser signal and the reflected signal of the multiple discrete collimated beams;

[0058] The processor 400 is configured to obtain first TOF depth data based on the time difference of the pulsed laser signal, obtain second TOF depth data based on the phase difference of the multiple discrete collimated beams or obtain structured light depth data based on parallax, and control the switching between first mode, second mode and third mode of the first projector, the second projector and the receiver.

[0059] In the first mode, the first projector projects a pulsed laser signal, the receiver receives the pulsed laser signal, and the processor obtains the first TOF depth data through the time difference;

[0060] In the second mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the second TOF depth data;

[0061] In the third mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the structured light depth data; when the processor obtains the structured light depth data, it also simultaneously obtains the second TOF depth data and uses it for matching region localization of the structured light depth data.

[0062] Specifically, receiver 300 and first projector 100 constitute a d-TOF system for measuring distant targets. Receiver 300 and second projector 200 constitute an i-TOF system and a structured light system for measuring nearby targets. The depth camera in this embodiment switches between a first mode and a second mode, or between a first mode and a third mode. When switching between the first mode, the second mode, and the third mode, the closest distance value is used.

[0063] In the first mode, the receiver 300 and the first projector 100 constitute a d-TOF system for measuring targets at a relatively long distance. If the first TOF depth data is less than a first threshold but greater than a second threshold, the system switches to the second mode. If the first TOF depth data is less than the second threshold, the system switches to the third mode.

[0064] In the second mode, the receiver 300 and the second projector 200 constitute an i-TOF system for measuring targets at medium distances. Second TOF depth data is obtained based on the phase difference between the signal received by the receiver 300 and the signal projected by the second projector 200. If the second TOF depth data is greater than a first threshold, the system switches to the first mode. If the second TOF depth data is less than the second threshold, the system switches to the third mode.

[0065] In the third mode, the receiver 300 and the second projector 200 constitute both an i-TOF system and a structured light system for measuring nearby targets. The second TOF depth data is used to limit the processing of the structured light depth data, making the matching range smaller, thereby reducing the computational load and accelerating the matching speed. Simultaneously, leveraging the strong anti-interference characteristic of TOF data against strong light, the anti-interference capability of the structured light data is enhanced. If the structured light depth data is smaller than the second threshold and less than the first threshold, the system switches to the second mode; if the structured light depth data is larger than the first threshold, the system switches to the first mode.

[0066] Figure 2 This is a schematic diagram illustrating time consumption in an embodiment of the present invention. Figure 2As shown, both TOF and structured light acquire signals during time period t1. During time period t2, TOF completes the acquisition and judgment of depth data. Simultaneously, structured light completes the calculation of structured light depth data during time period t3. Due to the complexity of structured light depth data calculation, which consumes significant time and resources, t3 is much longer than t2. However, using the scheme of this embodiment, the time required for structured light calculation is represented by time period t4. It is clear from the figure that t4 is less than t3, meaning that the time to obtain the depth fusion depth map in this embodiment is much shorter than in existing technologies. It should be noted that the time reduction in this embodiment is not fixed but varies with the target object. Generally, the more data within the TOF measurement range, the more time this embodiment saves.

[0067] Figure 3 This is a schematic diagram of a multi-mode depth camera arrangement according to an embodiment of the present invention. A first projector 100, a second projector 200, and a receiver 300 are arranged on the same plane and sequentially. The light projected by the first projector 100 and the second projector 200 is received by two adjacent receivers 300, resulting in measurement values ​​within different ranges. A single receiver 300 simultaneously receives light projected by two adjacent first projectors 100 and two adjacent second projectors 200. A processor 400 controls the actions of the first projector 100, the second projector 200, and the receiver 300, ensuring that at any given time, the same receiver 300 receives only light projected from the same projector, i.e., light projected by one first projector 100 or one second projector 200. The specific arrangement positions of the first projector 100, the second projector 200, and the receiver 300 can be set according to their respective emission and reception angles.

[0068] Figure 4 This is a schematic diagram of another multi-mode depth camera arrangement in an embodiment of the present invention. A first projector 100, a second projector 200, and a receiver 300 are arranged on different planes. Multiple first projectors 100 are arranged on one plane. Multiple second projectors 200 are arranged on one plane. Multiple receivers 300 are arranged on one plane. Each receiver 300 has two first projectors 100 and two second projectors 200 arranged around it, and can receive corresponding signals. The signal from the same first projector 100 can be received by two adjacent receivers 300. The signal from the same second projector 200 can be received by two adjacent receivers 300. The processor 400 controls the actions of the first projectors 100, the second projectors 200, and the receivers 300, so that at any given time, the receiver 300 only receives the projected light from the same type of projector, i.e., the reflected signal from the first projector 100 or the reflected signal from the second projector 200.

[0069] Figure 5This is a flowchart illustrating the steps of a processor processing structured light depth data in an embodiment of the present invention. Figure 5 As shown in the figure, a method for processing structured light depth data by a processor in this embodiment of the invention includes the following steps.

[0070] Step S1: Project a laser signal onto the target object and receive the reflected signal from the target object.

[0071] In this step, the reflected signal includes a first signal and a second signal. The first signal is used to acquire Time-of-Flight (TOF) depth data, and the second signal is used to acquire structured light depth data. The TOF depth data and the structured light depth data are in one-to-one correspondence. The first signal and the second signal are reflection signals of the target object acquired simultaneously, or reflection signals from simultaneously emitted laser signals. The TOF depth data and the structured light depth data are in one-to-one correspondence on the depth map, meaning that each position on the depth map has corresponding TOF depth data and structured light depth data.

[0072] In this embodiment, the TOF can be either i-TOF or d-TOF. Since the measurement distance of d-TOF is much greater than that of i-TOF, the structured light beams that can be matched with i-TOF and d-TOF also differ to some extent. The effective measurement distances of TOF and structured light can partially overlap to adapt to scenarios requiring high precision. For example, the TOF measurement distance is (1.5m, 5m), and the structured light measurement distance is (1m, 2m). Alternatively, the effective measurement distances of TOF and structured light can be completely separate to adapt to scenarios requiring a larger measurement range. For example, the TOF measurement distance is (3m, 10m), and the structured light measurement distance is (1m, 2m).

[0073] Step S2: Determine the value of the TOF depth data one by one according to the position on the depth map.

[0074] In this step, after obtaining the signal in step S1, the TOF depth data value can be quickly obtained. The TOF depth data is calculated using a linear formula, which is simple and fast. A pre-set first threshold is used as the standard for selecting either TOF depth data or structured light depth data for the current position. The value of the first threshold varies depending on the combination of different TOF and structured light technologies. For example, the first threshold value differs when the effective measurement distances of TOF and structured light can partially overlap, versus when there is no overlap. Even for the same combination of TOF and structured light technologies, the selection of the first threshold can differ. For example, when the effective measurement distances of TOF and structured light can partially overlap, the first threshold can be arbitrarily selected within the overlapping area. However, it should be noted that in this case, different values ​​of the first threshold will lead to different results.

[0075] Step S3: If the value of the TOF depth data is in the first range, then the value of the TOF depth data is used as the final depth value of the location.

[0076] In this step, the first range is at least partially the effective measurement range of the TOF depth data.

[0077] In some embodiments, the first range is the effective measurement range of the TOF depth data. The first threshold is the minimum value of the effective measurement range of the TOF depth data. In this case, all locations with relatively reliable TOF depth data are used as the final depth value. This embodiment ensures that all TOF depth data is confirmed as the final depth value, minimizing the amount of structured light depth data processing and achieving the fastest processing speed.

[0078] In some embodiments, the first range is a portion of the effective measurement range of the TOF depth data. The first threshold is greater than the minimum value of the effective measurement range of the TOF depth data. Preferably, the first threshold is the depth value with the smallest difference between the TOF depth data and the structured light depth data. Since there is usually a certain difference between the TOF depth data and the structured light depth data, and this difference varies at different locations, this embodiment can obtain optimal coherence after the fusion of the TOF depth data and the structured light depth data. Compared to the prior art, which uses the value of the TOF depth data to correct the value of the structured light depth data, or vice versa, this embodiment is more efficient, achieving both good fusion effect and very high efficiency.

[0079] In some embodiments, the first range is larger than the effective measurement range of the TOF depth data. The first threshold is smaller than the minimum value of the effective measurement range of the TOF depth data. This embodiment is applicable to situations where the effective measurement distances of TOF and structured light do not overlap. The first threshold is for discontinuous regions in the effective measurement distances of TOF and structured light. The data in the discontinuous regions does not need to be precise and can tolerate larger errors, and the selection of the first threshold can also be based on specific purposes. For example, when a faster response speed is required, the first threshold should be as small as possible; when higher accuracy is required, the first threshold should be selected at the point where the sum of the errors of the TOF depth data and the structured light depth data is minimized.

[0080] Step S4: If the value of the TOF depth data is in the second range, then calculate the structured light depth data of the location and use it as the final depth value of the location.

[0081] In this step, the sum of the second range and the first range equals the total effective range of the TOF depth data and the structured light depth data. The total effective range refers to the range between the minimum and maximum values ​​of the four endpoints of the effective ranges of the TOF depth data and the structured light depth data. For example, if the TOF measurement distance is (3m, 10m) and the structured light measurement distance is (1m, 2m), then the total effective range is (1m, 10m). In this case, if the first range is [2.5m, 10m), then the second range is (1m, 2.5m).

[0082] Step S5: Generate a depth fusion depth map based on the final depth value.

[0083] In this step, all locations are traversed to obtain the final depth value of the entire image. The final depth value is one of TOF depth data or structured light depth data.

[0084] Figure 6 This is a flowchart illustrating one step in calculating the final depth value according to an embodiment of the present invention. Figure 6 As shown, a method for calculating the final depth value in an embodiment of the present invention includes the following steps.

[0085] Step S41: Obtain the TOF depth data at the location as the initial depth value of the structured light at the location, and then calculate the disparity corresponding to the initial depth value of the structured light based on the known parameters.

[0086] In this step, since there is only one depth value at the same location, and TOF depth data is obtained very quickly, it can be used for the calculation of structured light depth data. Known parameters are those related to the data calculation. For example, when implementing this invention using a depth camera, the known parameters refer to the camera's intrinsic and extrinsic parameters. Using these known parameters, the disparity at the current location can be calculated, that is, the disparity corresponding to the initial depth value of the structured light.

[0087] Step S42: Set a window near the parallax to match.

[0088] In this step, the window expands to obtain a boundary centered on the parallax and limited by a second threshold; wherein, the second threshold is inversely proportional to the accuracy of the TOF depth data. In the prior art, a fixed-size window is usually used for matching. Matching needs to be calculated at each depth within the measurement range, and then the depth with the best matching value is calculated from all depth ranges as the depth value at that location. This method requires traversing all depths within the measurement range. For example, when the effective measurement range is (1m, 10m), all depths within that range need to be matched. This embodiment, however, uses the depth value obtained from TOF as the initial depth value to narrow the range to be matched. A second threshold is set as the depth range to which the initial depth value needs to be expanded outwards. For example, if the initial depth value is 5m and the second threshold is 0.2m, then the depth value corresponding to the window range is [4.8m, 5.2m], which significantly reduces the amount of data and computation during matching compared to (1m, 10m). The accuracy of the TOF depth data varies at different locations, so different second thresholds are used. When the accuracy of the TOF depth data is the same at different locations, the second threshold is also the same. The higher the accuracy of the TOF depth data, the smaller the second threshold. The window is the area to be searched on the structured light image, and its corresponding depth value is narrowed by the TOF depth data, which greatly reduces the process of structured light calculation.

[0089] Step S43: Calculate the optimal matching score position within the window as the final matching result.

[0090] In this step, all depth values ​​within the window are calculated and matched, and the position with the highest matching score is taken as the final matching result, thus obtaining the final depth value.

[0091] This embodiment uses TOF depth data to constrain structured light depth data, which greatly reduces the range of structured light depth data during matching, thereby significantly reducing the data matching time of structured light and greatly improving the efficiency of structured light depth data calculation, enabling rapid acquisition of structured light depth data.

[0092] Figure 7This is a schematic diagram of a drone structure according to an embodiment of the present invention. The main body is the drone fuselage. There are multiple first projectors 100, second projectors 200, and receivers 300, for example, four in each case. These four first projectors 100, second projectors 200, and receivers 300 are evenly mounted on the side of the drone fuselage. In some embodiments, a first projector 100, a second projector 200, and a receiver 300 are also mounted on the bottom of the drone for measuring the distance to the ground. When the drone is equipped with a gimbal and a conventional camera, the multi-mode depth camera on the bottom can be integrated with the conventional camera and can provide depth information to the conventional camera. Alternatively, the first projectors 100, second projectors 200, and receivers 300 on the bottom can be staggered with the conventional camera. Multiple first projectors 100, second projectors 200, and receivers 300 can better realize the drone's acceleration, deceleration, stopping, obstacle avoidance, object tracking, and positioning functions.

[0093] Figure 8 This is a schematic diagram of a robot structure according to an embodiment of the present invention. The main body is the robot body. The number of first projectors 100, second projectors 200, and receivers 300 can all be multiple, for example, four each. The four first projectors 100, second projectors 200, and receivers 300 are evenly mounted on the side of the robot body. The robot body can move with multiple first projectors 100, second projectors 200, and receivers 300 to acquire depth images from multiple different orientations, thereby identifying target objects, determining changes in the distance between the target object and the moving platform, and thus better controlling the movement of the robot body to achieve various functions.

[0094] Figure 9This is a schematic diagram of the structure of an intelligent vehicle according to an embodiment of the present invention. The main body is the vehicle body. There are multiple first projectors 100, second projectors 200, and receivers 300, for example, four in each case. All four first projectors 100, second projectors 200, and receivers 300 are mounted on the sides of the vehicle body. For example, one first projector 100, second projector 200, and receiver 300 is mounted at the front of the vehicle, one at the rear, one on the left side, and one on the right side. The vehicle itself can move multiple first projectors 100, second projectors 200, and receivers 300 along the road to construct a 360-degree panoramic depth image along the travel route, serving as a reference map; or it can acquire depth images from different directions to identify target objects and determine changes in the distance between the target object and the moving platform, thereby better controlling the vehicle's movement and achieving information acquisition during unmanned driving. For example, when the vehicle is traveling on the road, if it detects that the distance between the target object and the vehicle is decreasing and the target object is a pothole in the road, the vehicle decelerates with a first acceleration; if it detects that the distance between the target object and the vehicle is decreasing and the target object is a person, the vehicle decelerates with a second acceleration. The absolute value of the first acceleration is less than the absolute value of the second acceleration. Performing different operations based on different target objects when the distance decreases makes the vehicle more intelligent.

[0095] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0096] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A multi-mode depth camera, characterized in that, include: The first projector is used to project pulsed laser signals; The second projector is used to project multiple discrete collimated beams; A receiver is used to receive the pulsed laser signal and the reflected signals of the multiple discrete collimated beams; The processor is configured to obtain first TOF depth data based on the time difference of the pulsed laser signal, obtain second TOF depth data based on the phase difference of the multiple discrete collimated beams or obtain structured light depth data based on parallax, and control the first projector, the second projector and the receiver to switch between a first mode, a second mode and a third mode. In the first mode, the first projector projects a pulsed laser signal, the receiver receives the pulsed laser signal, and the processor obtains the first TOF depth data through the time difference; In the second mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the second TOF depth data; In the third mode, the first projector and the second projector project alternately, and the processor alternately obtains the first TOF depth data and the structured light depth data; When the processor obtains the structured light depth data, it also simultaneously obtains the second TOF depth data and uses it for locating the matching region of the structured light depth data.

2. A multi-mode depth camera according to claim 1, characterized in that, When switching between the first mode, the second mode, and the third mode, the nearest distance value shall be used.

3. A multi-mode depth camera according to claim 1, characterized in that, Based on the first TOF depth data in the first mode, the second TOF depth data in the second mode, and the structured light depth data in the third mode, it is determined whether a mode switch is needed.

4. A multi-mode depth camera according to claim 1, characterized in that, The signal from one of the first projectors or the second projector is received by at least two of the receivers.

5. A multi-mode depth camera according to claim 1, characterized in that, One of the receivers receives signals from at least two of the first projectors or the second projector.

6. A multi-mode depth camera according to claim 1, characterized in that, In the third mode, when the processor obtains the structured light depth data, it includes: Step S1: Project a laser signal onto the target object and receive the reflected signal from the target object; wherein the reflected signal includes a first signal and a second signal, the first signal is used to acquire the second TOF depth data, the second signal is used to acquire the structured light depth data, and the second TOF depth data and the structured light depth data are in one-to-one correspondence; Step S2: Determine the value of the second TOF depth data one by one according to the position on the depth map; Step S3: If the value of the second TOF depth data is in the first range, then the value of the TOF depth data is used as the final depth value of the location; Step S4: If the value of the second TOF depth data is in the second range, then calculate the structured light depth data at the location and use it as the final depth value at the location; Step S5: Generate a depth fusion depth map based on the final depth value.

7. A multi-mode depth camera according to claim 6, characterized in that, In step S4, calculating the structured light depth data at the location and using it as the final depth value at the location includes: Step S41: Obtain the second TOF depth data at the location as the initial depth value of the structured light at the location, and then calculate the disparity corresponding to the initial depth value of the structured light based on the known parameters; Step S42: Set a window for matching near the parallax; the window is determined based on the second TOF depth data; Step S43: Calculate the optimal matching score position within the window as the final matching result.

8. A multi-mode depth camera according to claim 7, characterized in that, The window is centered at the location and its boundaries are expanded to a limit of a second threshold; wherein the second threshold is inversely proportional to the accuracy of the second TOF depth data.

9. A mobile platform, characterized in that, include: ontology; and A multi-mode depth camera according to any one of claims 1-8.

10. A mobile platform according to claim 9, characterized in that, The body can be a vehicle body, a drone fuselage, a robot body, or a ship body.

Images