Method, device and equipment for generating point cloud data

Abstract

The invention discloses a point cloud data generation method, device and equipment, and the method comprises the steps: transmitting electromagnetic waves to carry out multi-round scanning on a target object, and receiving an echo signal reflected by the target object in each round of scanning; the scanning track of the multi-round scanning comprises a plurality of sub-tracks, one sub-track corresponds to one round of scanning, and the plurality of sub-tracks comprise at least one pair of sub-tracks; the first pair of sub-tracks in the at least one pair of sub-tracks comprises the ith sub-track and the jth sub-track, the ith sub-track corresponds to the ith round of scanning, the jth sub-track corresponds to the jth round of scanning, the electromagnetic wave emission frequency of the ith round of scanning is different from the electromagnetic wave emission frequency of the jth round of scanning, and the first pair of sub-tracks is any one pair of sub-tracks in the at least one pair of sub-tracks; and for the first pair of sub-tracks, generating point cloud data based on the echo signal on the i-th sub-track and the echo signal on the j-th sub-track, wherein the point cloud data is used for indicating the distance between the target object and the point cloud data.

Description

Technical Field

[0001] This application relates to the field of radar sensors, and more particularly to a method, apparatus, and device for generating point cloud data. Background Technology

[0002] Multi-sensor fusion integrates information obtained from different sensors, avoiding the limitations and uncertainties of individual sensor perception, forming a more comprehensive perception and recognition of the environment or target, improving the system's external perception capability, and is widely used in fields such as fault detection, remote sensing, human health monitoring, robotic systems, human-computer interaction, target recognition and tracking, simultaneous localization and mapping, and advanced driver assistance systems.

[0003] Sensors are crucial for a system to perceive the external world. The main sensor combinations include radar-camera, camera-LiDAR, and radar-camera-LiDAR. Cameras accurately capture contour, texture, and color distribution information, aiding in the classification and identification of different targets. Millimeter-wave radar and LiDAR can obtain three-dimensional data of targets, which can reconstruct the real spatial environment to the greatest extent possible.

[0004] Radar-visual fusion refers to the fusion of image and point cloud data to reconstruct a 3D spatial environment. The main drawback of existing radar-visual fusion methods is that while camera-output images have high resolution, radar / LiDAR-output point clouds are sparse, resulting in a significant resolution gap between the point cloud and the image. Therefore, the point cloud density needs further improvement. Summary of the Invention

[0005] This application provides a method, apparatus, and device for generating point cloud data, which can be used to solve the problem of fuzzy distance, thereby generating high-density point clouds.

[0006] In a first aspect, this application provides a detection device, which includes: (1) an electromagnetic beam scanning module for emitting electromagnetic waves to perform multiple scans on a target object; the scanning trajectory of the multiple scans includes multiple sub-trajectories, one sub-trajectory corresponds to one scan, and the multiple sub-trajectories include at least one pair of sub-trajectories; the first pair of sub-trajectories in the at least one pair of sub-trajectories includes an i-th sub-trajectory and a j-th sub-trajectory, the i-th sub-trajectory corresponds to the i-th scan, the j-th sub-trajectory corresponds to the j-th scan, the electromagnetic wave emission frequency of the i-th scan is different from the electromagnetic wave emission frequency of the j-th scan, where i and j are positive integers, and the first pair of sub-trajectories is any pair of sub-trajectories in the at least one pair of sub-trajectories; (2) an echo receiving module for receiving the echo signal reflected by the target object in each scan; (3) a point cloud generation module for generating point cloud data based on the echo signal received on the i-th sub-trajectory and the echo signal received on the j-th sub-trajectory, the point cloud data being used to indicate the depth distance between the device and the target object.

[0007] By employing the above design, multiple scans are performed using different transmission frequencies, and the depth distance between the detection device and the target object is determined based on two sub-trajectories that use different transmission frequencies, rather than calculating the depth distance through the flight time of the echo signal. This can overcome the limitations of fuzzy distance and generate high-density point clouds.

[0008] In one possible design, the device further includes an image generation module; the image generation module is used to generate a target image based on the echo signal received by the echo receiving module, the target image including the target object.

[0009] Through the above design, the detection device outputs both point cloud data and images. Compared with sensor combinations such as cameras and radar, the detection device can realize a true radar-vision integrated machine.

[0010] In one possible design, the scanning direction of the i-th scan is a first direction, and the scanning direction of the j-th scan is a second direction, wherein the first direction is the same as the second direction, or the first direction is opposite to the second direction.

[0011] The above design provides multiple scanning methods.

[0012] In one possible design, the first pair of sub-trajectories is any two adjacent sub-trajectories among the plurality of sub-trajectories.

[0013] With the above design, since the object parts scanned by the two adjacent sub-trajectories are closer, the accuracy of the point cloud data generated based on the two adjacent sub-trajectories is higher.

[0014] In one possible design, the first echo signal and the second echo signal included in the first pair of echo signals are two echo signals that are adjacent in position in the i-th sub-track and the j-th sub-track.

[0015] With the above design, since the object parts illuminated by two adjacent echo signals are closer, the accuracy of the point cloud data generated based on two adjacent echo signals is higher.

[0016] Secondly, this application provides a point cloud data generation method, the method comprising: acquiring echo information, the echo information including echo signals reflected by the target object during a multi-round scanning process using electromagnetic waves; the scanning trajectory of the multi-round scanning includes multiple sub-trajectories, one sub-trajectory corresponding to one round of scanning, the multiple sub-trajectories including at least one pair of sub-trajectories; the first pair of sub-trajectories in the at least one pair of sub-trajectories includes an i-th sub-trajectory and a j-th sub-trajectory, the i-th sub-trajectory corresponding to the i-th round of scanning, the j-th sub-trajectory corresponding to the j-th round of scanning, the electromagnetic wave emission frequency of the i-th round of scanning being different from the electromagnetic wave emission frequency of the j-th round of scanning, wherein i and j are positive integers, and the first pair of sub-trajectories being any pair of sub-trajectories in the at least one pair of sub-trajectories; generating point cloud data based on the echo signals on the i-th sub-trajectory and the echo signals on the j-th sub-trajectory, the point cloud data being used to indicate the distance between the device and the target object.

[0017] In one possible design, the method further includes: generating a target image based on the received echo signal, the target image including the target object.

[0018] In one possible design, the scanning direction of the i-th scan is a first direction, and the scanning direction of the j-th scan is a second direction, wherein the first direction is the same as the second direction, or the first direction is opposite to the second direction.

[0019] In one possible design, the first pair of sub-trajectories is any two adjacent sub-trajectories among the plurality of sub-trajectories.

[0020] In one possible design, the echo signal received on the i-th sub-track includes a plurality of first echo signals, and the echo signal received on the j-th sub-track includes a plurality of second echo signals; the plurality of first echo signals and the plurality of second echo signals form a plurality of pairs of echo signals, each pair of echo signals including a first echo signal and a second echo signal;

[0021] Point cloud data for generating a sub-trajectory based on the echo signals received on the i-th sub-trajectory and the echo signals received on the j-th sub-trajectory includes:

[0022] For the first pair of echo signals among multiple pairs of echo signals, the following steps are performed: Determine the unambiguous distance based on the time information of the first echo signal and the time information of the second echo signal; the first pair of echo signals is any pair among the multiple pairs of echo signals; the time information of the first echo signal indicates the time difference between the transmission time of the first electromagnetic wave and the reception time of the first echo signal, wherein the first electromagnetic wave is an electromagnetic wave transmitted before the first echo signal is received; the time information of the second echo signal indicates the time difference between the transmission time of the second electromagnetic wave and the reception time of the second echo signal, wherein the second electromagnetic wave is an electromagnetic wave transmitted before the second echo signal is received.

[0023] The point cloud data includes multiple unambiguous distances determined based on the multiple pairs of echo signals.

[0024] In one possible design, the first echo signal and the second echo signal included in the first pair of echo signals are two echo signals that are adjacent in position in the i-th sub-track and the j-th sub-track.

[0025] Thirdly, this application provides a vehicle including the detection device of the first aspect of this application and any possible design of the first aspect.

[0026] Fourthly, this application provides a camera, including the detection device of the first aspect of this application and any possible design of the first aspect.

[0027] Fifthly, this application provides a terminal device, including the detection device of the first aspect of this application and any possible design of the first aspect. The terminal device includes robots, drones, wearable smart devices, vehicle-mounted terminal devices, etc.

[0028] In a sixth aspect, this application provides an electronic device including at least one processor, at least one memory, a display screen, and a transceiver; wherein the one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the technical solutions of the second aspect of this application and any possible design of the second aspect.

[0029] The seventh aspect provides another electronic device, the electronic device comprising: a module / unit for performing the methods of the second aspect or any possible design of the second aspect; these modules / units may be implemented in hardware or by hardware executing corresponding software.

[0030] The eighth aspect provides a chip coupled to a memory in an electronic device for calling a computer program stored in the memory and executing the technical solutions of the second aspect of this application and any possible design of the second aspect.

[0031] A ninth aspect provides a computer-readable storage medium comprising a computer program that, when executed on an electronic device, causes the electronic device to perform the technical solutions described in the second aspect above and any possible design of the second aspect.

[0032] The tenth aspect provides a computer program comprising instructions that, when executed on a computer, cause the computer to perform the technical solutions described in the second aspect above and any possible design of the second aspect.

[0033] For the beneficial effects described in the second to tenth aspects above, please refer to the description of the beneficial effects in the first aspect above, which will not be repeated here. Attached Figure Description

[0034] Figure 1 This is one of the existing technical solutions;

[0035] Figure 2 This is a second existing technical solution;

[0036] Figure 3 This is a third existing technical solution;

[0037] Figure 4 This is a schematic diagram of a distance detection scenario;

[0038] Figure 5 A schematic diagram illustrating the fuzzy distance problem;

[0039] Figure 6 This is a schematic diagram of the structure of a detection device 10 provided in an embodiment of this application;

[0040] Figure 7 A schematic diagram of a scanning direction provided in an embodiment of this application;

[0041] Figure 8 A schematic diagram of a single-transmitter, single-receiver structure of a detection device 10 provided in an embodiment of this application;

[0042] Figure 9 A flowchart illustrating a point cloud data generation method provided in an embodiment of this application;

[0043] Figure 10A This is a schematic diagram of a scanning method provided in an embodiment of this application;

[0044] Figure 10BA schematic diagram illustrating different pulse transmission frequencies provided in an embodiment of this application;

[0045] Figure 10C A schematic diagram of a scanning path provided for an embodiment of this application;

[0046] Figure 11A This is a schematic diagram of an echo signal numbering method provided in an embodiment of this application;

[0047] Figure 11B A schematic diagram illustrating the composition of an echo signal pair provided in an embodiment of this application;

[0048] Figure 11C A schematic diagram illustrating the determination of three-dimensional coordinates of a point cloud, provided as an embodiment of this application;

[0049] Figure 12A This is a schematic diagram illustrating another scanning method provided in an embodiment of this application;

[0050] Figure 12B This is a schematic diagram of another scanning path provided in an embodiment of this application;

[0051] Figure 12C This is a schematic diagram of an echo signal numbering method provided in an embodiment of this application;

[0052] Figure 12D A schematic diagram illustrating another configuration of echo signal pairs provided in an embodiment of this application;

[0053] Figure 12E A schematic diagram illustrating another method for determining the three-dimensional coordinates of a point cloud, as provided in an embodiment of this application;

[0054] Figure 13A This is a schematic diagram illustrating yet another scanning method provided in an embodiment of this application;

[0055] Figure 13B This is another schematic diagram of a scanning path provided in the embodiments of this application;

[0056] Figure 14A A schematic diagram of a sub-trajectory shape provided in an embodiment of this application;

[0057] Figure 14B This is a schematic diagram of another sub-trajectory shape provided in an embodiment of this application;

[0058] Figure 15 This is a schematic diagram of an image generation process provided in an embodiment of this application;

[0059] Figure 16A This is a schematic diagram of echo signal mapping provided in an embodiment of this application;

[0060] Figure 16BThis is a schematic diagram of an image generation process provided in an embodiment of this application;

[0061] Figure 17 This is a schematic diagram of the structure of an electromagnetic wave transmitting module 11 provided in an embodiment of this application;

[0062] Figure 18 This is a schematic diagram of the structure of a beam scanning module 12 provided in an embodiment of this application;

[0063] Figure 19 This is a schematic diagram of another beam scanning module 12 provided in an embodiment of this application;

[0064] Figure 20 This is a schematic diagram of the structure of an echo receiving module 13 provided in an embodiment of this application;

[0065] Figure 21 This is a schematic diagram of an echo signal processing flow provided in an embodiment of this application;

[0066] Figure 22 This is a schematic diagram of another detection device provided in an embodiment of this application. Detailed Implementation

[0067] First, some terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0068] 1. Radar / Radio Detection and Ranging (RADAR) is used to directionally transmit electromagnetic energy into space. By analyzing the electromagnetic waves reflected by objects in space, the direction, height, speed, and relative distance of those objects can be calculated, and the shape of the objects can also be detected.

[0069] 2. LiDAR (Light Laser Detection and Ranging) is short for laser detection and ranging system. Compared with traditional sensors such as cameras and ultrasonic sensors, LiDAR has the advantages of high measurement accuracy, fast response speed, and strong anti-interference ability. LiDAR has been widely used in the fields of intelligent driving and autonomous driving.

[0070] When a lidar is in operation, it emits electromagnetic waves toward a target object (which can be a person, object, or animal, and can be referred to as the target). After receiving the echo signal reflected back from the target object, the lidar processes the electromagnetic waves and the echo signal to obtain a point cloud.

[0071] 3. Point clouds are sampling points obtained after a detection device probes an object. Each point cloud corresponds to a set of three-dimensional coordinates (x, y, z). z is the depth distance between the detection device and the object.

[0072] The following introduces several existing technical solutions for improving point cloud density.

[0073] Option 1: One of the options for receiving and sending more data;

[0074] See Figure 1 As shown, in Figure 1 In this array, ALx is an onboard array that emits light, while RLx is a photodetector used to receive the echoes of electromagnetic waves reflected from the surface of an object. ALx can emit multiple electromagnetic beams simultaneously, thereby increasing point cloud density.

[0075] Option Two: The second option is to receive and send more.

[0076] See Figure 2 As shown, multiple transceiver modules are used to increase point cloud density. Scheme 2 is similar to Scheme 1, both increasing the number of transceiver modules to improve point cloud density. The difference is that in Scheme 2, the multiple transceiver modules are responsible for different view fields, while in Scheme 1, the multiple transceiver modules are in the same view field.

[0077] The disadvantages of Scheme 1 and Scheme 2 include: multiple transmitters and receivers require a large number of light sources and detectors or multiple optical paths, resulting in a large number of components, which will increase material and assembly costs, leading to higher costs, and the products are larger in size and have limited application scenarios.

[0078] Option 3: Multiple collections via face-to-face delivery;

[0079] Another approach to improving point cloud density using lidar is flash lidar, such as... Figure 3 As shown, a surface light source illuminates the target, and a surface array detector measures the distance. The density of the point cloud in a flash lidar depends on the density of the surface array detector; increasing the number of pixels in the surface array detector can improve the point cloud density. However, this approach is limited by the size constraints of the surface array detector elements, and the current point cloud density remains relatively low.

[0080] Option 4: Increase the electromagnetic wave transmission frequency.

[0081] like Figure 4 As shown, the point cloud density can be increased by increasing the laser's emission frequency, that is, the number of laser pulses M emitted per unit time. One pulse signal corresponds to the emission of one electromagnetic wave. It can be understood that for a single-receiver, single-transmitter module, the more points emitted per unit time, the greater the point cloud density.

[0082] However, when using Scheme 4 to improve point cloud density, a distance ambiguity problem arises, leading to a shortened radar measurement range. For example... Figure 5As shown, for pulse radar, to avoid aliasing, the time interval between the echo signal and the transmitted pulse is shorter than the pulse transmission time interval. When the repetition frequency of the transmitted pulse increases, the pulse period, that is, the pulse transmission time interval, shortens. At this time, when the distance to the target (such as a person, vehicle, or object) is greater than the maximum distance corresponding to the pulse repetition period, the target echo does not fall within the current period, i.e., aliasing occurs. The distance measured at this time is not the true distance, and is called fuzzy distance.

[0083] As those skilled in the art will know, the modulation frequency of electromagnetic waves is related to point cloud density and detection distance. A higher modulation frequency results in higher point cloud density, but a shorter maximum detection distance to the surrounding environment. This is because, to prevent aliasing, the next electromagnetic wave should only be emitted after receiving the echo reflected from the electromagnetic wave after it has traveled a certain detection distance. Understandably, the greater the detection distance, the longer the flight time of the electromagnetic wave / echo. Therefore, the modulation frequency of the electromagnetic wave must usually be selected based on the actual environment and cannot be arbitrarily changed, resulting in the point cloud density being limited by the ambiguity distance and unable to be further improved.

[0084] The technical solution provided in this application will be introduced next.

[0085] This application provides a method for generating high-density point cloud data. This method can overcome the limitations of fuzzy distance and achieve high-density point cloud data output on the hardware basis of a single transmitter and single receiver module. Optionally, the device can also output images. Compared with the combination of hardware devices that output images from a camera and point cloud data from sensors such as radar, this application can realize a true integrated radar and vision sensor.

[0086] The types of detection devices may include, but are not limited to: radar sensors, laser sensors, cameras, etc.

[0087] In one application scenario, the detection device is used in vehicles, such as in-vehicle navigation systems and autonomous driving systems. It can be used to generate images of the vehicle while measuring the distance between the vehicle and surrounding objects, thus helping the vehicle avoid obstacles.

[0088] In one application scenario, the detection device is used in a smart terminal, which includes, but is not limited to, mobile phones, wearable devices (such as watches, bracelets, etc.), augmented reality (AR) devices, virtual reality (VR) devices, etc.

[0089] In one application scenario, the detection device is used in monitoring equipment, such as security monitoring and electronic traffic police. The generated point cloud data and images can be used to assist traffic police in determining responsibility.

[0090] In one application scenario, this detection device is used in robots, such as medical robotic arms, handling robots, production robots, and entertainment robots, to detect the distance between the robot and surrounding objects, assisting the robot in avoiding obstacles and positioning. It can also be applied in machine vision scenarios for robots to identify materials.

[0091] In one application scenario, the detection device is used in aerospace equipment, such as airplanes and drones, and can be specifically used in aerospace navigation systems.

[0092] Figure 6 This is a schematic diagram of a detection device provided in an embodiment of this application. Figure 6 As shown, the detection device 10 includes an electromagnetic wave transmitting module 11, a beam scanning module 12, an echo receiving module 13, an image generating module 15, a point cloud generating module 14, and a clock control module 16.

[0093] 1) Electromagnetic wave transmitting module 11, used to transmit electromagnetic waves. The electromagnetic waves may include, but are not limited to, laser sources, radio frequency sources, or terahertz sources. In this application, module 11 has at least two different transmission frequencies. Different transmission frequencies can be used to transmit electromagnetic waves when performing different rounds of scanning. In one example, electromagnetic wave transmitting module 11 receives the scanning trajectory signal sent by beam scanning module 12 and switches the transmission frequency according to the scanning trajectory signal.

[0094] 2) Beam scanning module 12 is used to deflect electromagnetic waves to scan and irradiate an object. The scanning method includes multiple rounds of scanning. In each round of scanning, module 11 emits electromagnetic waves at least two different transmission frequencies. For example, in the first round of scanning, the transmission frequency is the first transmission frequency. In the second round of scanning, the transmission frequency is the second transmission frequency. In the third round of scanning, the transmission frequency is the first transmission frequency. In the second round of scanning, the transmission frequency is the second transmission frequency. This cycle continues until the set number of scanning rounds is reached.

[0095] In one example, multiple scans are performed row by row. Module 12 can deflect the electromagnetic beam incident from module 11 via a scanning mirror, thereby changing the exit direction of the electromagnetic beam. In hardware, the scanning mirror is driven to rotate horizontally and vertically. Horizontal rotation deflects the emitted electromagnetic wave to scan the target horizontally. Vertical rotation changes the path of the incident electromagnetic wave to different rows for scanning. If the vertical direction remains stationary, module 12 will repeatedly scan the same row back and forth.

[0096] Based on this, module 12 can also be used to output scan trajectory signals and angle signals. The scan trajectory signal indicates different scan cycles. The angle signal may include the horizontal rotation angle (denoted as angle signal x) and the vertical rotation angle (denoted as angle signal y).

[0097] 3) Echo receiving module 13, used to receive the echo signal reflected back by the electromagnetic waves scanned to the object. Furthermore, module 13 uses the point cloud data generation method provided in this application embodiment to process the received echo signal and generate depth and distance data for multiple point clouds.

[0098] From a hardware perspective, in one example, see [link to hardware documentation]. Figure 7 As shown, the detection device 10 is a single-transmitter, single-receiver detector, meaning module 11 includes a single transmitter and module 13 includes a single receiver. In another example, the detection device 10 is a multiple-transmitter, multiple-receiver detector, meaning module 11 includes multiple transmitters and module 13 includes multiple receivers. The detection device 10 proposed in this application can achieve high-density point cloud output based on single-transmitter, single-receiver operation. In another example, see... Figure 8 As shown, the detection device 10 is a multiple-transmitter, multiple-receiver detector, such as... Figure 8 As shown, the detection device 10 includes multiple transmitters and multiple receivers. It can be understood that N transmitters and N receivers can increase the point cloud density by up to N times compared to single transmitter and single receiver. Therefore, the following description uses single transmitter and single receiver as an example.

[0099] 4) Point cloud generation module 14, used to generate point cloud data based on angle signals and depth distance generated by echo receiving module 13. Optionally, it can also combine scanning trajectory signals to generate point cloud data.

[0100] 5) Image generation module 15, used to generate an image based on the echo signal, the image containing the target object.

[0101] The clock control module 16 is used to output clock signals to each module for control, synchronization, triggering, etc.

[0102] The structure of a detection device 10 has been described above. It should be noted that... Figure 6 As merely one example, the detection device 10 designed in this application may include relative Figure 6 It may have more or fewer components. For example, in another implementation, the detection device 10 does not include module 14, but only includes modules 11-13, 15 and 16.

[0103] The following is based on Figure 6 Taking the detection device 10 shown as an example, this application provides a method for generating point cloud data.

[0104] Figure 9This is a flowchart illustrating a method for generating point cloud data provided in an embodiment of this application. The method may include the following steps:

[0105] Step 901: Emits electromagnetic waves to scan the target object multiple times and receives the echo signal reflected by the target object in each scan.

[0106] In this embodiment, the scanning method may include multiple rounds of scanning, during which at least two transmission frequencies are used. Each round of scanning corresponds to a sub-trajectory, and the scanning path of multiple rounds of scanning includes the sub-trajectory corresponding to each round of scanning.

[0107] See Figure 10A As shown, with Figure 10A The scanning parameters shown are used for scanning. Taking one line scan as an example, combined with... Figure 10B The scanning process includes the following steps: During the first line scan, the electromagnetic wave emitting module 11 emits electromagnetic waves at a transmission frequency f1. After completing the first line scan, it quickly resets (i.e., the angle signal x is reset) and moves to the next line (e.g., the angle signal y+1) to perform the second line scan. During the second line scan, the electromagnetic wave emitting module 11 emits electromagnetic waves at a transmission frequency f2. After completing the second line scan, it quickly resets and moves to the next line to perform the third line scan. This process continues in this manner.

[0108] Each scan line corresponds to a sub-trajectory, such as Figure 10C As shown, the scanning trajectory in the first row is sub-trajectory 1, the scanning trajectory in the second row is sub-trajectory 2, the scanning trajectory in the third row is sub-trajectory 3, and so on. Taking a laser pulse as an example, Figure 10C Each black dot represents a laser pulse. It can be seen that the emission interval between laser pulses varies with different emission frequencies. The higher the emission frequency, the shorter the emission interval; conversely, the lower the emission frequency, the longer the emission interval.

[0109] Step 902: For each pair of sub-trajectories in the multiple sub-trajectories, generate the depth distance of multiple sampling points corresponding to each pair of echo signals included in each pair of sub-trajectories.

[0110] Among them, a pair of sub-trajectories includes two sub-trajectories, denoted as the i-th sub-trajectory and the j-th sub-trajectory respectively. The i-th sub-trajectory corresponds to the i-th round of scanning, and the j-th sub-trajectory corresponds to the j-th round of scanning. The electromagnetic wave emission frequency of the i-th round of scanning is different from that of the j-th round of scanning, where i and j take different positive integers.

[0111] In one example, the two sub-trajectories in a pair of sub-trajectories are adjacent. Figure 10CTaking the multiple sub-trajectories shown as an example, the multiple pairs of sub-trajectories are as follows: the first pair: (sub-trajectory 1, sub-trajectory 2), the second pair: (sub-trajectory 2, sub-trajectory 3), the third pair: (sub-trajectory 3, sub-trajectory 4), the fourth pair: (sub-trajectory 4, sub-trajectory 5), and so on.

[0112] A pair of sub-trajectories includes multiple pairs of echo signals, and each pair of echo signals includes two adjacent echo signals in the two sub-trajectories. The position of the echo signal can be: the position after aligning the row and column of the laser pulse signal (electromagnetic wave) preceding the echo signal. Adjacent position can refer to two echo signals in the same column in two adjacent rows after reordering, that is, two echo signals with the same column number.

[0113] For example, see Figure 11A As shown in (a), each black dot represents a laser pulse signal, and the white circle represents the received echo signal. The echo signals are aligned with the previous adjacent laser pulse signal. It is worth noting that if aliasing occurs, such as... Figure 11A If two echo signals are received at a certain transmission interval in neutron trajectory 3 (a), then the two echo signals are arranged in the order of reception time. See also... Figure 11A As shown in (b), alternatively, one echo signal can be retained while another is discarded. The rearranged echo signals are then numbered by row and column according to the scanning order. For example, multiple echo signals in sub-track 1 can be sorted and numbered as d. 11 d 12 , ...,d 1m The multiple echo signals in sub-trajectory 2 are sorted and numbered as d. 21 d 22 , ...,d 2m Among them, d 11 This represents the first echo signal in the first row, i.e., the first echo signal on sub-track 1. d 12 This represents the second echo signal in the first row, d. 1m This represents the m-th echo signal in the first row. d 21 This indicates the first echo signal in the second row, i.e., the first echo signal on sub-track 2. d 22 d represents the second echo signal in the second row, and d2m represents the m-th echo signal in the second row. That is, d nm In this context, n represents the row number and m represents the column number.

[0114] It should be noted that, Figure 11A The echo signal sorting method shown is only one example. This application can also sort the echo signals according to the receiving time order, without any specific limitation.

[0115] based on Figure 11AThe sorting method shown in (b) includes multiple pairs of echo signals in the first pair of sub-trajectories, see [reference]. Figure 11B The first pair of echo signals (d) 11 d 21 ), the second pair of echo signals (d 12 d 22 ), the third pair of echo signals (d 13 d 23 ), and so on.

[0116] In step 902, each pair of echo signals is processed to obtain the depth distance corresponding to each pair of echo signals.

[0117] For example, see Figure 11B As shown, based on (d 11 d 21 Generate depth distance D 11 Based on (d) 12 d 22 Generate depth distance D 12 Based on (d) 13 d 23 Generate depth distance D 13 And so on. Here, depth distance represents the distance from the detection device 10 to the target. The depth distance of different point clouds represents the distance from the detection device 10 to different sampling points on the target.

[0118] Taking a pair of echo signals (including the first echo signal and the second echo signal) as an example, the processing method for the pair of echo signals may include: determining the unambiguous distance based on the time information of the first echo signal and the time information of the second echo signal.

[0119] The time information is used to indicate the flight time of the echo signal. For example, the time information of the first echo signal indicates the flight time Δt of the first echo signal, where Δt = the time difference between the reception time of the first echo signal and the transmission time of the first pulse signal (see [reference]). Figure 11A As shown in (a), the first pulse signal is the laser pulse signal adjacent to the first echo signal. Similarly, the time information of the second echo signal indicates the flight time of the second echo signal.

[0120] The following describes the process of determining depth and distance based on the time information of the echo signal:

[0121] First, based on Figure 5 This application introduces the concept for solving the fuzzy distance problem. For example... Figure 5 As shown, for the blurred distance caused by aliasing, the true distance (i.e., the non-blurred distance, denoted as D) is D = n*d per +fuzzy distance; where d perRepresents the distance corresponding to the pulse emission time interval, and the calculation method is d per Represents the distance corresponding to the pulse emission time interval, where T per Represents the pulse interval corresponding to the pulse emission frequency f per ; c represents the speed of light.

[0122] It is set that in this embodiment, f1 > f2, and it satisfies where f r Is the greatest common divisor of f1 and f2, and both N and a are positive integers. For example, in one example, let a be equal to 1. For instance, when f1 = 10 MHz, f2 = 9.9 MHz, f r = 10 5 Hz, a = 1.

[0123] Based on d 11 , d 21 Generates D based on the flight time of the echo signal represented 11 , combined with Figure 10B Understand, the calculation process is as follows:

[0124]

[0125] ΔT 11 = Δt 11 + n1 * T1 = Δt 21 + n2 * T2 Formula (2)

[0126] where, Δt 11 Represents the time difference of the echo signal identified by d 11 ; Δt 21 Represents the time difference of the echo signal identified by d 21 ; 0 ≤ Δt 11 < T1, 0 ≤ Δt 21 < T2; n1, n2 are integers greater than or equal to 0.

[0127] Since It can be obtained that (N + 1) * T1 = N * T2. Therefore, Formula (2) can be transformed as follows:

[0128]

[0129] In one example, when Δt 11 > Δt 21 Based on Formula (3), it can be obtained that n1 > n2. When n1 and n2 are both integers, it can be obtained that n1 = n2 + 1. Substituting this into Formula (2) gives:

[0130]

[0131] Substituting formula (4) into formulas (2) and (1), we can obtain D. 11 .

[0132] In another example, when Δt 11 ≥Δt 21 When n1 = n2, we can obtain n1 = n2. Substituting this into formula (2), we get:

[0133]

[0134] Substituting formula (5) into formulas (2) and (1), we can obtain D. 11 .

[0135] Similarly, D can be calculated. 12 D 13 D 1m .

[0136] It is worth noting that, in this embodiment, the relationships and values ​​of the above formulas and calculation parameters can be configured as preset values ​​according to the above derivation results. Alternatively, the above formulas and calculation parameters can be obtained by the program executing the derivation process, and this embodiment does not limit this.

[0137] Similarly, the second pair of sub-trajectories includes multiple pairs of echo information, including: (d 21 d 31 ), (d 22 d 32 ), (d 23 d 33 ), ..., (d 2m d 3m The third pair of sub-trajectories includes multiple pairs of echo information, including: (d) 31 d 41 ), (d 32 d 42 ), (d 33 d 43 ), ..., (d 3m d 4m And so on. Based on the above method, processing each pair of echo signals yields D. 21 D 22 D 23 D 2m And so on.

[0138] Step 903: Based on the angle signal and the depth distance determined in step 902, generate point cloud data of the target object.

[0139] As mentioned above, a point cloud includes the three-dimensional coordinates (x, y, z) of the sampling points. In this embodiment of the application, the x coordinate can be the angle signal x, the y coordinate can be the angle signal y, and the z coordinate is the depth distance D.

[0140] Point cloud generation module 14 receives the angle signal generated by beam scanning module 12. (See also...) Figure 11C As shown, the angle signal includes angle signal x and angle signal y. It should be noted that... Figure 11C All signals shown are time-synchronized. The angle signals x and y corresponding to each echo signal are determined based on the reception time of each echo signal (or according to the transmission time of the pulse signal corresponding to each echo signal), and based on the depth distance D of the echo signal. nm Generate point clouds. Here, the echo signal refers to the echo signal on one of the sub-trajectories in each pair of sub-trajectories.

[0141] like Figure 11C As shown, (x 11 ,y 11 D 11 ), (x 12 ,y 12 D 12 ),…(x 1m, y 1m, D 1m (x) constitutes the point cloud in the first row. 21, y 21 D 21 ),(x 22 ,y 22 D 22 ),…(x 2m, y 2m, D 2m This forms the second row of point clouds, and so on.

[0142] In the above text Figures 11A to 11C The examples shown below have the same scanning direction. Examples with different scanning directions will be introduced below.

[0143] by Figure 12A Taking the scanning parameters shown as an example, the scanning process includes: During the first line scan, the transmission frequency f1 is used, and the scanning direction is forward. After completing the first line scan, a recovery motion is performed, reversing the horizontal direction, and the angle signal y+1, i.e., moving to the next line for the second line scan. During the second line scan, the transmission frequency f2 is used, and the scanning direction is reversed. After completing the second line scan, the third line scan is performed. And so on. Correspondingly, the scanning path is as follows: Figure 12B As shown, the scanning path is serpentine. Based on the sorting and numbering method described above, the received echo signals are numbered, see [link to documentation]. Figure 12C As shown.

[0144] exist Figure 12C In this context, multiple pairs of sub-trajectories may include: (sub-trajectories 1, sub-trajectories 2), (sub-trajectories 2, sub-trajectories 3), (sub-trajectories 3, sub-trajectories 4), and so on. Taking a pair of sub-trajectories as an example, the multiple pairs of echo signals in this sub-trajectories pair may include: (d 11 d 2m ), (d 12 d 2(m-1) ), (d 1m d 21 And so on. See also Figure 12D As shown, based on (d 11 d 2m Generate depth distance D 11 Based on (d) 12 d 2(m - 1) Generate depth distance D 12 Based on (d) 13 d 2(m-2) Generate depth distance D 13 And so on. See also... Figure 12E As shown, based on the angle signal and depth distance D nm Generate a point cloud. The specific methods and procedures are described above and will not be repeated here.

[0145] It should be noted that:

[0146] (1) The above example illustrates that the sub-trajectories do not overlap. In the embodiments of this application, multiple sub-trajectories may overlap or not overlap. For example, in one example, the sub-trajectories partially overlap. For example, combined with Figure 13A The scanning parameters shown illustrate a scanning process that may include: First, a first scan along the first row using transmission frequency f1. After completing the first scan, a rapid reset (i.e., angle signal x is reset), leaving angle signal y unchanged, followed by a second scan along the first row using transmission frequency f2. After completing the second scan, a rapid reset, angle signal y+1 (i.e., moving to the next row), and a third scan along the second row using transmission frequency f1. After completing the third scan, a rapid reset, leaving angle signal y unchanged, and a fourth scan along the second row using transmission frequency f2. This process continues in this manner. Figure 13B The sub-trajectory 1 corresponding to the first round of scanning overlaps with the sub-trajectory 2 corresponding to the second round of scanning. The sub-trajectory 3 corresponding to the third round of scanning overlaps with the sub-trajectory 4 corresponding to the fourth round of scanning. That is, the sub-trajectories partially overlap.

[0147] In another example, the sub-trajectories all overlap. For example, in the above... Figure 13A In the example, when multiple scans only include two scans of the same row, all sub-trajectories overlap, for example, all sub-trajectories only include Figure 13B The sub-trajectories 1 and 2, or only sub-trajectories 3 and 4, etc.

[0148] (2) The examples above illustrate sub-trajectories with straight line segments as examples, meaning that the angle signal y remains constant during the scanning process of each row. Besides straight line segments, embodiments of this application also support the formation of sub-trajectories with different shapes. For example, Figure 14A Another scanning method is illustrated, where each sub-trajectory is wavy. For example, Figure 14B Another scanning method is illustrated, where each sub-trajectory is linear. This application does not limit the scope of this method.

[0149] (3) The scanning pattern in the above example is only one example. This application also supports other scanning methods, such as using the same transmission frequency for multiple consecutive rows. Specifically, for example, the first, second, and third rows are scanned using transmission frequency f1, the fourth row is scanned using transmission frequency f2, the fifth and sixth rows are scanned using transmission frequency f1, and the seventh and eighth rows are scanned using transmission frequency f2. In short, at least two different transmission frequencies can be used in multiple rounds of scanning.

[0150] (4) In the above example, the number of output points in each round of scanning is the same. This application embodiment also supports multiple rounds of scanning with different output point numbers, including: some rounds have different output point numbers, for example, the number of output points in the first row of scanning is different from the number of output points in the second round of scanning; the number of output points in the first, third, and fifth rows (n+1 rows) is the same; the number of output points in the second, fourth, and sixth rows (2n rows) is the same, where n is an integer. Alternatively, all rounds have different output point numbers, that is, the number of output points in each round is different. When the number of output points is different, only paired echo signals can be processed.

[0151] (5) The above description uses two transmission frequencies for multi-round scanning as an example. In the embodiments of this application, multi-round scanning may also use more than two transmission frequencies. For example, the first row of scanning uses transmission frequency f1, the second row of scanning uses transmission frequency f2, the third row of scanning uses transmission frequency f3, the fourth row of scanning uses transmission frequency f1, the fifth row of scanning uses transmission frequency f2, the sixth row of scanning uses transmission frequency f3, and so on.

[0152] The various scanning methods described above can be combined arbitrarily. For example, multiple rounds of scanning can be performed using two or more scanning frequencies, the sub-trajectory can be any shape such as a straight line segment, a wavy line, or a broken line, the number of output points in each row of scanning can be the same or different, and the electromagnetic wave emission frequencies of adjacent rows can be the same or different, etc., in any combination of these methods. This application does not limit this.

[0153] (6) The above method of dividing sub-trajectory pairs by their adjacent positions is one example. This application does not limit this approach. For instance, in another example, the two sub-trajectories in a pair are not adjacent. For example, Figure 11A In (a), the first and second row sub-trajectories form a pair, and the third and fourth row sub-trajectories form a pair. For example, Figure 11A In (a), the first and fourth row sub-trajectories form a pair, and the second and fifth row sub-trajectories form a pair. For example, Figure 13B In this embodiment, sub-trajectory 1 and sub-trajectory 2 are a pair, sub-trajectory 3 and sub-trajectory 4 are a pair, and so on. Alternatively, sub-trajectory 1 and sub-trajectory 3 are a pair, sub-trajectory 2 and sub-trajectory 4 are a pair, and so on. The method of dividing sub-trajectory pairs in this application embodiment is not limited. Regardless of the method of dividing sub-trajectory pairs, the processing method of the echo signal pairs included in each pair of sub-trajectory pairs in this application embodiment is the same, and will not be repeated here.

[0154] By employing the above design, multiple scans are performed using different transmission frequencies, and the depth distance between the detection device and the target object is determined based on the echo signals of two sub-trajectories scanned using different transmission frequencies, rather than calculating the depth distance through the flight time of the echo signals. This approach can overcome the limitations of fuzzy distance and generate high-density point clouds.

[0155] Step 904: Generate a target image based on the echo signal. The target image includes the target object.

[0156] In one implementation: a target image is generated based on the grayscale signal of the echo signal, the target image being a grayscale map used to indicate the brightness of an object. For example, the image generation module 15 may rearrange the received echo signal into a two-dimensional image frame based on a pulse time series, and then generate the target image based on the depth signal of the echo signal corresponding to each pixel in the two-dimensional image.

[0157] See Figure 15 As shown, the method for generating the target image may include the following steps:

[0158] Step 1501: Map the echo signals received by module 12 one by one based on one or more parameters such as pulse time series, angle signal, and scanning trajectory signal.

[0159] like Figure 16A As shown in (a), each black dot represents a emitted laser pulse signal, and the white circle represents a received echo signal. The echo signals are aligned with the preceding adjacent pulse signal. It is worth noting that due to ambiguity issues, there may be many aliased echo signals; for example, if two or more echo signals are received within a single transmission interval. If aliasing occurs, one echo signal can be discarded, such as retaining the echo signal closest to the pulse signal for alignment. (Figure 1) This indicates that there is no echo signal at this location.

[0160] The row and column of each echo signal can be determined based on the angle signal or the scan trajectory signal. Figure 16A (b) exemplarily illustrates the rearranged echo signals. The rearranged echo signals are numbered according to row and column numbers, generating an index value for each echo signal. This index value is used to indicate the position of the echo signal in the two-dimensional image, for example, d. 11 This represents the first echo signal in the first row, i.e., the first echo signal on sub-track 1. d 12 This represents the second echo signal in the first row, d. 1m This represents the m-th echo signal in the first row. d 21 This indicates the first echo signal in the second row, i.e., the first echo signal on sub-track 2. d 22 d represents the second echo signal in the second row, and d2m represents the m-th echo signal in the second row. Where d nm In this context, n represents the row number and m represents the column number.

[0161] Step 1502: Grid the spatial location indicated by the scanning parameters and integerize the coordinate values ​​to generate an image array.

[0162] For example, the number of scan rows is determined based on the angle signal y to determine the number of pixel rows in a 2D image, and the number of pixel columns in a 2D image is determined based on a pulse time series. The image array includes the integer coordinates of each pixel, as shown in [see...]. Figure 16B As shown in (a), the image array includes the integer coordinates of each pixel. It should be understood that... Figure 16B The integer coordinate values ​​shown in (a) are only one example.

[0163] Step 1503: Assign the corresponding echo signal to all pixels (or coordinate points) based on the index value of the echo signal.

[0164] For example, see Figure 16B As shown in (b), the echo signal is assigned to the corresponding pixel position according to the index value. For example, suppose the index value of the echo signal indicates the position in the first row and first column (e.g., d). 11 If the indicated position is in the first row and second column (e.g., d), then the echo signal is assigned to the pixel corresponding to the coordinate (0, 0). 12 If the echo signal is (0,1), then the echo signal is assigned to the pixel corresponding to coordinate (0,1). And so on.

[0165] Step 1504: Generate the target image based on the grayscale signal of the echo signal corresponding to each pixel.

[0166] It should be noted that, Figure 16BThe method of compiling the coordinate values ​​of each pixel shown in (a) is only an example and is not limited to any specific method. In addition, the mapping method between echo signals and pulse signals when generating the image and the sorting method of echo signals based on pulse signals when calculating the depth distance D can be independent of each other, can use different methods, or can use the same method, and is not limited to any specific method.

[0167] In another implementation, replacing the grayscale signal of the echo signal corresponding to each pixel with the depth signal of the echo signal in the above implementation can generate another target image.

[0168] Through the above design, the detection device 10 outputs both point cloud data and images. Compared with the combination of sensors such as cameras and radar, the detection device can realize a true radar-vision integrated machine.

[0169] The following describes a hardware control method for some modules in the detection device 10, in conjunction with the above-described method embodiments.

[0170] Electromagnetic wave transmitting module 11: Used to receive scanning trajectory signals (such as row numbers) and set the electromagnetic wave transmission frequency (or laser pulse repetition frequency) according to the scanning trajectory signals. See also Figure 17 As shown, in one example, the electromagnetic wave transmitting module 11 includes a laser driver, a laser diode, and an optical path. The laser driver receives... Figure 9 In this method embodiment, the scanning trajectory signal, i.e., the line signal, outputs an electrical signal of a certain frequency to control the laser diode to emit light. The line signal is used to trigger a change in the frequency of the electrical signal driving the laser, thereby changing the laser pulse repetition frequency. The laser diode emits laser light, which is then input into the beam scanning module 12 after passing through the optical path. The optical path is used to collimate the laser light emitted by the laser diode; in a specific embodiment, the optical path can be a collimating lens or a collimating lens group.

[0171] Beam scanning module 12: Used to receive electromagnetic beams, scan electromagnetic beams, and output scan trajectory information and angle signals. Optionally, it can also output pulse time series.

[0172] The beam scanning module 12 can use a MEMS scanning mirror, a voice coil scanning mirror, an optical phased array (OPA), or other methods to deflect the laser beam to scan the target object from both forward and reverse directions. For example, see [link to MEMS scanning mirror documentation]. Figure 18 As shown, in this embodiment, the beam scanning module 12 is composed of a 2D MEMS scanning mirror and its driving circuit. Alternatively, it can be composed of a galvanometer-type 2D scanning mirror and its drive, a 2D voice coil-type scanning mirror and its drive, or a 2D optical phased array and its drive. The 2D aspect includes angle signals x and y, meaning the drive can control the rotation of the MEMS scanning mirror in the x-direction and the y-direction. See also... Figure 19 As shown, the scanning beam can also be re-emitted after passing through a suitable shaping optical path. Figure 18 and Figure 19 The example shown is a single-transmitter, single-receiver module. When the detection device 10 includes a multi-transmitter, multi-receiver module, it can also scan multiple beams simultaneously.

[0173] Echo receiver module 13: Used to receive echo signals, see [link / reference] Figure 20 As shown in (a), the echo signal entering the echo receiving module 13 can be focused onto the photodetector by a lens. Alternatively, see Figure 20 As shown in (b), the echo signal can directly illuminate the photodetector. The photodetector is matched with the light source, and the photodetector includes, but is not limited to, photoelectric detectors (PDs), avalanche photodiodes (APDs), single-photon avalanche diodes (SPADs), photomultiplier tubes (PMTs), and multi-pixel photon counters (MPPCs). Optionally, a filter of the corresponding wavelength band can be added in front of the photodetector. Specific details are not limited.

[0174] The echo receiving module 13 is also used to output depth and distance information. See [link / reference] Figure 21 As shown, in this embodiment, the echo signal can be a laser beam. The laser beam is received by the detector after passing through a shaping optical path. In a specific implementation, the shaping optical path is a focusing lens, and the detector can be an APD and its circuit, or a SiPM and its circuit, or a PD and its circuit, or a PMT and its circuit. The time-to-digital conversion unit is composed of a time-to-digital converter (TDC). In a specific implementation, the electrical pulse signal from the electromagnetic wave transmitting module and the electrical pulse signal generated by the detector unit are input into the ADC circuit. The ADC circuit is used to extract the reflectivity intensity carried by the echo signal to obtain the grayscale signal. The other input is into the TDC, which outputs a time vector, which is further processed according to... Figure 9 The method shown in step 902 of the method embodiment generates depth distance (D) based on the time vector of adjacent line echo signals. nm The output is sent to the point cloud generation module.

[0175] This application also provides a detection device 2400. Figure 22 This is a schematic diagram of a detection device 2400 provided in an embodiment of this application. The detection device 2400 can be applied to vehicles, cameras, or various terminal devices mentioned above. (See also...) Figure 22 As shown, the detection device 2400 may include a transceiver 2401 and a chip 2402.

[0176] Functionally, in one alternative design, the transceiver 2401 is used to perform... Figure 6 The device 10 shown illustrates the functions of modules 11 to 13, and chip 2402 executes the functions of modules 14 to 15. In one hardware design, transceiver 2401 may include modules 11 to 13. Chip 2402 may be, but is not limited to, any one or more of the following processors: central processing unit (CPU), data processing unit (DPU), system-on-chip (SoC), field-programmable gate array (FPGA), graphics processing unit (GPU), application-specific integrated circuit (ASIC).

[0177] Based on the above embodiments, this application also provides a computer program that, when run on a computer, causes the computer to perform... Figure 9 The illustrated embodiment provides a point cloud data generation method.

[0178] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform... Figure 9 The illustrated embodiment provides a point cloud data generation method. The storage medium can be any available medium accessible to a computer. For example, but not limited to, a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible to a computer.

[0179] Based on the above embodiments, this application also provides a chip, which is used to read a computer program stored in a memory and implement... Figure 9 The illustrated embodiment provides a point cloud data generation method.

[0180] Based on the above embodiments, this application provides a chip system including a processor for supporting computer devices to implement... Figure 9The illustrated embodiment provides a point cloud data generation method. In one possible design, the chip system further includes a memory for storing necessary programs and data for the computer device. The chip system may consist of chips or may include chips and other discrete components.

[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0184] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0185] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating point cloud data, characterized in that, Applied to a detection device, the method includes: Electromagnetic waves are emitted to perform multiple scans on a target object, and the echo signals reflected by the electromagnetic waves from each scan to the target object are received. The scanning trajectory of the multiple scans includes multiple sub-trajectories, with one sub-trajectory corresponding to one scan. The multiple sub-trajectories include at least one pair of sub-trajectories. The first pair of sub-trajectories in the at least one pair of sub-trajectories includes an i-th sub-trajectory and a j-th sub-trajectory. The i-th sub-trajectory corresponds to the i-th scan, and the j-th sub-trajectory corresponds to the j-th scan. The electromagnetic wave emission frequency of the i-th scan is different from that of the j-th scan, where i and j are positive integers. The first pair of sub-trajectories is any pair of sub-trajectories in the at least one pair of sub-trajectories. Point cloud data is generated based on the echo signals on the i-th sub-track and the j-th sub-track, and the point cloud data is used to indicate the distance between the device and the target object.

2. The method as described in claim 1, characterized in that, The method further includes: A target image is generated based on the received echo signal, the target image including the target object.

3. The method as described in claim 1 or 2, characterized in that, The scanning direction of the i-th round of scanning is a first direction, and the scanning direction of the j-th round of scanning is a second direction. The first direction is the same as the second direction, or the first direction is opposite to the second direction.

4. The method according to any one of claims 1-3, characterized in that, The first pair of sub-trajectories is any two adjacent sub-trajectories among the plurality of sub-trajectories.

5. The method according to any one of claims 1-4, characterized in that, The echo signal received on the i-th sub-track includes a plurality of first echo signals, and the echo signal received on the j-th sub-track includes a plurality of second echo signals; the plurality of first echo signals and the plurality of second echo signals form a plurality of pairs of echo signals, each pair of echo signals including a first echo signal and a second echo signal; Point cloud data for a sub-trajectory is generated based on the echo signals received on the i-th sub-trajectory and the echo signals received on the j-th sub-trajectory, including: For the first pair of echo signals among multiple pairs of echo signals, the following steps are performed: Determine the unambiguous distance based on the time information of the first echo signal and the time information of the second echo signal; the first pair of echo signals is any pair among the multiple pairs of echo signals; the time information of the first echo signal indicates the time difference between the transmission time of the first electromagnetic wave and the reception time of the first echo signal, wherein the first electromagnetic wave is an electromagnetic wave transmitted before the first echo signal is received; the time information of the second echo signal indicates the time difference between the transmission time of the second electromagnetic wave and the reception time of the second echo signal, wherein the second electromagnetic wave is an electromagnetic wave transmitted before the second echo signal is received. The point cloud data includes multiple unambiguous distances determined based on the multiple pairs of echo signals.

6. The method as described in claim 5, characterized in that, The first echo signal and the second echo signal included in the first pair of echo signals are two echo signals that are adjacent in position in the i-th sub-track and the j-th sub-track.

7. A detection device, characterized in that, The device includes: The electromagnetic wave transmitting module is used to transmit electromagnetic waves according to the set transmission frequency corresponding to the scanning round; A beam scanning module is used to deflect electromagnetic waves to perform multiple scans on a target object. The scanning trajectory of the multiple scans includes multiple sub-trajectories, with one sub-trajectory corresponding to one scan round. The multiple sub-trajectories include at least one pair of sub-trajectories. The first pair of sub-trajectories in the at least one pair of sub-trajectories includes an i-th sub-trajectory and a j-th sub-trajectory. The i-th sub-trajectory corresponds to the i-th scan round, and the j-th sub-trajectory corresponds to the j-th scan round. The electromagnetic wave emission frequency of the i-th scan round is different from that of the j-th scan round, where i and j are positive integers. The first pair of sub-trajectories is any pair of sub-trajectories in the at least one pair of sub-trajectories. The echo receiving module is used to receive the echo signal reflected by the target object in each round of scanning, and generate multiple point cloud corresponding depth distances based on the echo signals on the two sub-trajectories included in each pair of sub-trajectories. The point cloud generation module is used to generate point cloud data based on the depth distances corresponding to multiple point clouds.

8. The apparatus as claimed in claim 7, characterized in that, The device also includes an image generation module; The image generation module is used to generate a target image based on the echo signal, and the target image includes the target object.

9. The apparatus as claimed in claim 7 or 8, characterized in that, The scanning direction of the i-th round of scanning is a first direction, and the scanning direction of the j-th round of scanning is a second direction. The first direction is the same as the second direction, or the first direction is opposite to the second direction.

10. The apparatus according to any one of claims 7-9, characterized in that, The first pair of sub-trajectories is any two adjacent sub-trajectories among the plurality of sub-trajectories.

11. The apparatus according to any one of claims 7-10, characterized in that, The first echo signal and the second echo signal included in the first pair of echo signals are two echo signals that are adjacent in position in the i-th sub-track and the j-th sub-track.

12. A vehicle, characterized in that, Includes a detection device for performing the method as described in any one of claims 1-6.

13. A camera, characterized in that, Includes a detection device for performing the method as described in any one of claims 1-6.

14. A terminal device, characterized in that, Includes a detection device for performing the method as described in any one of claims 1-6.

Images