Point cloud data generation method, apparatus, and device

Abstract

A point cloud data generation method, an apparatus, and a device. The method comprises: emitting electromagnetic waves to perform multiple rounds of scanning on a target object, and receiving echo signals reflected by the target object in each round of scanning, a scanning trajectory of the multiple rounds of scanning comprising a plurality of sub-trajectories, one sub-trajectory corresponding to one round of scanning, the plurality of sub-trajectories including at least one pair of sub-trajectories, a first pair of sub-trajectories among the at least one pair of sub-trajectories comprising an i-th sub-trajectory and a j-th sub-trajectory, the i-th sub-trajectory corresponding to an i-th round of scanning, the j-th sub-trajectory corresponding to a j-th round of scanning, the electromagnetic wave emission frequency of the i-th round of scanning being different from that of the j-th round of scanning, and the first pair of sub-trajectories being any pair of sub-trajectories among the at least one pair of sub-trajectories; and for the first pair of sub-trajectories, generating point cloud data on the basis of the echo signals on the i-th sub-trajectory and the echo signals on the j-th sub-trajectory, the point cloud data being used for indicating a distance to the target object.

Description

A point cloud data generation method, device and equipment

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese Patent Application No. 202411137224.7, filed on August 16, 2024, and entitled "A Point Cloud Data Generation Method, Device and Equipment", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the field of radar sensors, in particular to a point cloud data generation method, device and equipment. BACKGROUND

[0004] Multi-sensor fusion comprehensively utilizes information obtained by different sensors, which can avoid the perception limitations and uncertainties of a single sensor, form a more comprehensive perception and recognition of the environment or target, improve the external perception ability of the system, and is widely used in fault detection, remote sensing, human health monitoring, robot systems, human-computer interaction, target recognition and tracking, simultaneous localization and mapping, advanced driving assistance systems and other fields.

[0005] Sensors are the key to the system's perception of the external world, and the main sensor combinations include radar-camera, camera-lidar and radar-camera-lidar. Cameras can accurately capture contour, texture and color distribution information, which helps to classify and identify different targets. Millimeter wave radar and lidar can obtain three-dimensional data of the target, and three-dimensional data can maximize the recovery of the real spatial environment.

[0006] Radar-visual fusion refers to the fusion of images and point cloud data to construct a restored three-dimensional spatial environment. The main drawback of existing radar-visual fusion is that the resolution of the camera output image is high, while the point cloud output by the radar / lidar is sparse, and the resolution gap between the point cloud and the image is relatively large. The point cloud density needs to be further improved. SUMMARY

[0007] The present application provides a point cloud data generation method, device and equipment, which can be used to solve the problem of fuzzy distance, thereby generating high-density point cloud.

[0008] In a first aspect, the present application provides a detection device, which comprises: (1) an electromagnetic beam scanning module, configured to emit electromagnetic waves to perform multiple rounds of scanning on a target object; the scanning track of the multiple rounds of scanning comprises multiple sub-tracks, one sub-track corresponds to one round of scanning, and the multiple sub-tracks comprise at least one pair of sub-tracks; the first pair of sub-tracks in the at least one pair of sub-tracks comprises an i-th sub-track and a j-th sub-track, the i-th sub-track corresponds to an i-th round of scanning, the j-th sub-track corresponds to a j-th round of scanning, the electromagnetic wave emission frequency of the i-th round of scanning is different from the electromagnetic wave emission frequency of the j-th round of scanning, where i and j are positive integers, and the first pair of sub-tracks is any pair of sub-tracks in the at least one pair of sub-tracks; (2) an echo receiving module, configured to receive echo signals reflected by the target object in each round of scanning; and (3) a point cloud generating module, configured to generate point cloud data based on the echo signals received on the i-th sub-track and the echo signals received on the j-th sub-track, the point cloud data being used to indicate the depth distance between the device and the target object.

[0009] Through the above design, multiple rounds of scanning are performed by using different emission frequencies, and the depth distance between the detection device and the target object is determined based on two sub-tracks scanned by using different emission frequencies, instead of calculating the depth distance by the time of flight of the echo signals, so that the ambiguous distance limitation can be broken through, and high-density point cloud can be generated.

[0010] In a possible design, the device further comprises an image generating module, which is configured to generate a target image based on the echo signals received by the echo receiving module, and the target image comprises the target object.

[0011] Through the above design, the detection device outputs both point cloud data and images, and compared with a sensor combination such as a camera-radar, the detection device can realize a true radar-vision all-in-one machine.

[0012] In a possible design, the scanning direction of the i-th round of scanning is a first direction, the scanning direction of the j-th round of scanning is a second direction, and the first direction is the same as or opposite to the second direction.

[0013] Through the above design, multiple scanning modes are provided.

[0014] In a possible design, the first pair of sub-tracks are any two adjacent sub-tracks in the multiple sub-tracks.

[0015] Through the above design, since the parts of the object scanned by the two adjacent sub-tracks are closer, the accuracy of the point cloud data generated based on the two adjacent sub-tracks is higher.

[0016] In a possible design, the first echo signal and the second echo signal comprised in the first pair of echo signals are two echo signals adjacent in position in the ith sub-track and the jth sub-track.

[0017] By the above design, since two echo signals adjacent in position scan an object part closer, the point cloud data generated based on the two echo signals adjacent in position is more accurate.

[0018] In a second aspect, the present application provides a point cloud data generation method, the method comprising: obtaining echo information, the echo information comprising echo signals reflected by a target object in a plurality of rounds of scanning processes performed on the target object by using electromagnetic waves; scanning tracks of the plurality of rounds of scanning comprising a plurality of sub-tracks, one sub-track corresponding to one round of scanning, the plurality of sub-tracks comprising at least one pair of sub-tracks; a first pair of sub-tracks in the at least one pair of sub-tracks comprising an ith sub-track and a jth sub-track, the ith sub-track corresponding to an ith round of scanning, the jth sub-track corresponding to a jth round of scanning, an electromagnetic wave emission frequency of the ith round of scanning being different from an electromagnetic wave emission frequency of the jth round of scanning, wherein i and j are positive integers, the first pair of sub-tracks being any one pair of sub-tracks in the at least one pair of sub-tracks; and generating point cloud data based on echo signals on the ith sub-track and echo signals on the jth sub-track, the point cloud data being used to indicate a distance between the apparatus and the target object.

[0019] In a possible design, the method further comprises: generating a target image based on the received echo signals, the target image comprising the target object.

[0020] In a possible design, a scanning direction of the ith round of scanning is a first direction, a scanning direction of the jth round of scanning is a second direction, the first direction being the same as the second direction, or the first direction being opposite to the second direction.

[0021] In a possible design, the first pair of sub-tracks is any two adjacent sub-tracks in the plurality of sub-tracks.

[0022] In a possible design, the echo signals received on the ith sub-track comprise a plurality of first echo signals, and the echo signals received on the jth sub-track comprise a plurality of second echo signals; the plurality of first echo signals and the plurality of second echo signals form a plurality of pairs of echo signals, each pair of echo signals comprising one first echo signal and one second echo signal.

[0023] Generating point cloud data of one sub-track based on the echo signals received on the ith sub-track and the echo signals received on the jth sub-track comprises:

[0024] For a first pair of echo signals in the plurality of pairs of echo signals, performing: determining a non-ambiguous distance based on time information of the first echo signal and based on time information of the second echo signal; the first pair of echo signals is any one pair in the plurality of pairs of echo signals; the time information of the first echo signal indicates a time difference value between a transmission time of a first electromagnetic wave and a reception time of the first echo signal, the first electromagnetic wave being an electromagnetic wave transmitted before the first echo signal is received; the time information of the second echo signal indicates a time difference value between a transmission time of a second electromagnetic wave and a reception time of the second echo signal, the second electromagnetic wave being an electromagnetic wave transmitted before the second echo signal is received.

[0025] The point cloud data includes a plurality of non-ambiguous distances determined based on the plurality of pairs of echo signals.

[0026] In a possible design, the first echo signal and the second echo signal included in the first pair of echo signals are two echo signals with adjacent positions in the ith sub-track and the jth sub-track.

[0027] In a third aspect, the present application provides a vehicle including the detection apparatus of the first aspect of the present application and any possible design of the first aspect.

[0028] In a fourth aspect, the present application provides a camera including the detection apparatus of the first aspect of the present application and any possible design of the first aspect.

[0029] In a fifth aspect, the present application provides a terminal device including the detection apparatus of the first aspect of the present application and any possible design of the first aspect. The terminal device includes a robot, a drone, a wearable smart device, a vehicle-mounted terminal device, and the like.

[0030] In a sixth aspect, the present application provides an electronic device including at least one processor, at least one memory, a display screen, and a transceiver; wherein the one or more memories store one or more computer programs, the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the technical solutions of the second aspect of the present application and any possible design of the second aspect.

[0031] In a seventh aspect, another electronic device is provided, which includes: a module / unit for performing the method of the above-mentioned second aspect or any possible design of the second aspect; these modules / units can be implemented by hardware, or by hardware executing corresponding software.

[0032] The eighth aspect provides a chip coupled with a memory in an electronic device, used to invoke a computer program stored in the memory and execute the technical solution of the second aspect and any possible design of the second aspect.

[0033] The ninth aspect provides a computer readable storage medium, including a computer program, when the computer program is run on an electronic device, the electronic device executes the technical solution of the second aspect and any possible design of the second aspect.

[0034] The tenth aspect provides a computer program, including instructions, when the instructions are run on a computer, the computer executes the technical solution of the second aspect and any possible design of the second aspect.

[0035] The beneficial effects of the second aspect to the tenth aspect are described above, and the beneficial effects of the first aspect are not repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0036] Fig. 1 is one of the prior art solutions;

[0037] Fig. 2 is another prior art solution;

[0038] Fig. 3 is a third prior art solution;

[0039] Fig. 4 is a schematic diagram of a distance detection scenario;

[0040] Fig. 5 is a schematic diagram of a fuzzy distance problem;

[0041] Fig. 6 is a schematic diagram of a detection device 10 provided by an embodiment of the present application;

[0042] Fig. 7 is a schematic diagram of a scanning direction provided by an embodiment of the present application;

[0043] Fig. 8 is a schematic diagram of a single-shot single-receiver structure of a detection device 10 provided by an embodiment of the present application;

[0044] Fig. 9 is a schematic diagram of a flow corresponding to a point cloud data generation method provided by an embodiment of the present application;

[0045] Fig. 10A is a schematic diagram of a scanning method provided by an embodiment of the present application;

[0046] Fig. 10B is a schematic diagram of different pulse transmission frequencies provided by an embodiment of the present application;

[0047] Fig. 10C is a schematic diagram of a scanning path provided by an embodiment of the present application;

[0048] FIG. 11A is a schematic diagram of an echo signal numbering method according to an embodiment of the present application;

[0049] FIG. 11B is a schematic diagram of an echo signal pair composition method according to an embodiment of the present application;

[0050] FIG. 11C is a schematic diagram of a method for determining the three-dimensional coordinates of a point cloud according to an embodiment of the present application;

[0051] FIG. 12A is a schematic diagram of another scanning method according to an embodiment of the present application;

[0052] FIG. 12B is a schematic diagram of another scanning path according to an embodiment of the present application;

[0053] FIG. 12C is a schematic diagram of an echo signal numbering method according to an embodiment of the present application;

[0054] FIG. 12D is a schematic diagram of another echo signal pair composition method according to an embodiment of the present application;

[0055] FIG. 12E is a schematic diagram of another method for determining the three-dimensional coordinates of a point cloud according to an embodiment of the present application;

[0056] FIG. 13A is a schematic diagram of yet another scanning method according to an embodiment of the present application;

[0057] FIG. 13B is a schematic diagram of yet another scanning path according to an embodiment of the present application;

[0058] FIG. 14A is a schematic diagram of a sub-track shape according to an embodiment of the present application;

[0059] FIG. 14B is a schematic diagram of another sub-track shape according to an embodiment of the present application;

[0060] FIG. 15 is a schematic diagram of an image generation process according to an embodiment of the present application;

[0061] FIG. 16A is a schematic diagram of an echo signal mapping according to an embodiment of the present application;

[0062] FIG. 16B is a schematic diagram of an image generation process according to an embodiment of the present application;

[0063] FIG. 17 is a schematic diagram of a structure of an electromagnetic wave transmitting module 11 according to an embodiment of the present application;

[0064] FIG. 18 is a schematic diagram of a structure of a beam scanning module 12 according to an embodiment of the present application;

[0065] FIG. 19 is a schematic diagram of a structure of another beam scanning module 12 according to an embodiment of the present application;

[0066] Fig. 20 is a structural schematic diagram of an echo receiving module 13 provided by an embodiment of the present application;

[0067] Fig. 21 is a schematic diagram of an echo signal processing flow provided by an embodiment of the present application;

[0068] Fig. 22 is a structural schematic diagram of another detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0069] First, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0070] 1. Radar / Radio Detection And Ranging (RADAR), which is used to emit electromagnetic energy in a directional manner into space, and the direction, height, speed, relative distance of an object can be calculated by the electromagnetic waves reflected by the object existing in the space, and the shape of the object can be detected.

[0071] 2. Lidar (light laser detection and ranging), which is the abbreviation of laser detection and ranging system. Compared with traditional sensors such as cameras and ultrasonic sensors, lidar has the characteristics of high measurement accuracy, fast response speed and strong anti-interference ability. Lidar has been widely used in the fields of intelligent driving and unmanned driving.

[0072] In the working of lidar, electromagnetic waves are emitted to a target object (which can be a person, an object, an animal, which can be referred to as a target). After the lidar receives the echo signal reflected back from the target object, the lidar processes the electromagnetic wave and the echo signal to obtain a point cloud.

[0073] 3. Point cloud, which is a sampling point obtained by detecting an object by a detection device. Each point cloud corresponds to a set of three-dimensional coordinates (x, y, z). z is the depth distance between the detection device and the object.

[0074] Several existing technical solutions for improving the density of point clouds are introduced below.

[0075] Solution one: one of the multiple-receiving and multiple-transmitting solutions;

[0076] Referring to Fig. 1, in Fig. 1, ALx is a board-mounted array light source, and RLx is a light detector for receiving the echo of the electromagnetic wave reflected by the surface of the object. ALX can emit multiple electromagnetic beams at the same time to improve the density of the point cloud.

[0077] Solution two: another multiple-receiving and multiple-transmitting solution;

[0078] Referring to FIG. 2, multiple transceiver modules are used to improve the point cloud density. Scheme two is similar to scheme one, both of which increase the number of transceiver modules to improve the point cloud density, the difference is that the multiple transceiver modules of scheme two are responsible for different viewing angles, while the multiple transceiver modules of scheme one are in the same viewing angle.

[0079] The disadvantages of scheme one and scheme two include: multiple transceiver uses many light sources and detectors or multiple light paths, the number of components is large, the material cost and assembly cost will increase accordingly, the cost is high, and the product size is large, the application scene is limited.

[0080] Scheme three: surface transmits multiple receivers scheme

[0081] Another scheme for improving the point cloud density of laser radar is flash laser radar, as shown in FIG. 3, a surface light source irradiates a target, and a surface array detector measures the distance. The density of the flash laser radar point cloud depends on the density of the surface array detector, and the point cloud density can be improved by increasing the number of pixels of the surface array detector. However, this scheme is limited by the volume of the surface array detector element, and the current point cloud density is still relatively low.

[0082] Scheme four: improve the electromagnetic wave transmission frequency.

[0083] As shown in FIG. 4, the point cloud density can be improved by increasing the laser transmission frequency, that is, the number of laser pulses transmitted per unit time M. One pulse signal corresponds to the transmission of one electromagnetic wave, which can be understood as, for a single receiver and single transmitter module, the more the number of points per unit time, the greater the point cloud density.

[0084] However, when scheme four is used to improve the point cloud density, the problem of distance ambiguity will be encountered, which leads to the shortening of the radar measurement distance. As shown in FIG. 5, for a pulse radar, in order to avoid aliasing, the time interval between the echo signal and the transmitted pulse is less than the pulse transmission time interval. When the repetition frequency of the transmitted pulse is increased, the pulse period, that is, the transmission time interval of the pulse, is shortened. At this time, when the distance of the target to be measured (such as a person, a vehicle, or an object) is greater than the maximum distance corresponding to the pulse repetition period, the target echo does not fall within the current period, that is, aliasing occurs, and the distance measured between the target is not the true distance, which is called ambiguous distance.

[0085] Those skilled in the art can know that the modulation frequency of the electromagnetic wave is related to the point cloud density and the detection distance, and the higher the modulation frequency of the electromagnetic wave, the higher the point cloud density, but the maximum detection distance of the surrounding environment is shorter. Because, in order to ensure that aliasing does not occur, the next electromagnetic wave should be transmitted after receiving the echo reflected by the electromagnetic wave flying a detection distance, so as to ensure that aliasing does not occur. It can be understood that the farther the detection distance, the longer the flight time required by the electromagnetic wave / echo, and therefore, the modulation frequency of the electromagnetic wave cannot be changed at will, resulting in that the point cloud density is limited by the blur distance and cannot be further improved.

[0086] Next, the technical solutions provided in the present application are introduced.

[0087] The embodiment of the present application provides a method for generating high-density point cloud data, which can break through the blur distance limit and realize high-density point cloud data output on the basis of a single-transmission-single-reception module (single transmitter-single receiver) hardware. Optionally, the device can also output images. Compared with the combination of the hardware device of camera output image + radar and other sensor output point cloud data, the present application can realize a real radar visual integrated sensor.

[0088] The product type of the detection device can include but is not limited to radar sensors, laser sensors, cameras, etc.

[0089] In one application scenario, the detection device is applied to a vehicle, such as a vehicle navigation system or an unmanned driving system, and can be used to generate vehicle driving images and measure the distance between the vehicle and surrounding objects, thereby assisting the vehicle to avoid obstacles.

[0090] In one application scenario, the detection device is applied to a smart terminal, wherein the smart terminal device includes but is not limited to a mobile phone, a wearable device (such as a watch, a bracelet, etc.), an augmented reality (AR) device, a virtual reality (VR) device, etc.

[0091] In one application scenario, the detection device is applied to a monitoring device, such as a security monitoring device or an electronic traffic police device, and the generated point cloud data and images can be used to assist the traffic police in determining responsibility.

[0092] In one application scenario, the detection device is applied to a robot, such as a medical robot, a carrying robot, a production robot, an entertainment robot, etc., and is used to detect the distance between the robot and surrounding objects, thereby assisting the robot to avoid obstacles, positioning, etc. The detection device can also be applied to a machine vision scene and used for robot material recognition.

[0093] In an application scenario, the detection device is applied to aerospace equipment such as an airplane, a drone, and the like, and can be specifically applied to an aerospace navigation system and the like.

[0094] FIG. 6 is a structural schematic diagram of a detection device provided by an embodiment of the present application. As shown in FIG. 6, the detection device 10 includes an electromagnetic wave emitting module 11, a beam scanning module 12, a return wave receiving module 13, an image generating module 15, a point cloud generating module 14, and a clock control module 15.

[0095] 1) The electromagnetic wave emitting module 11 is configured to emit electromagnetic waves. The electromagnetic waves can include, but are not limited to, a laser source, a radio frequency source, or a terahertz source, and the like. In the present application, the electromagnetic wave emitting module 11 has at least two different emission frequencies. Different emission frequencies can be used to emit electromagnetic waves when performing different rounds of scanning. In an example, the electromagnetic wave emitting module 11 receives a scanning trajectory signal sent by the beam scanning module 12 and switches the emission frequency according to the scanning trajectory signal.

[0096] 2) The beam scanning module 12 is configured to deflect the electromagnetic waves to scan and irradiate an object. The scanning manner includes multiple rounds of scanning. In the multiple rounds of scanning, the electromagnetic wave emitting module 11 uses at least two different emission frequencies to emit electromagnetic waves. For example, in the first round of scanning, the emission frequency is a first emission frequency. In the second round of scanning, the emission frequency is a second emission frequency. In the third round of scanning, the emission frequency is the first emission frequency. In the second round of scanning, the emission frequency is the second emission frequency. This cycle is repeated until a set number of scanning rounds is reached.

[0097] In an example, the multiple rounds of scanning are performed in units of rows. The beam scanning module 12 can deflect the electromagnetic beam incident on the electromagnetic wave emitting module 11 by using a scanning mirror, so as to change the outgoing direction of the electromagnetic beam. In hardware, the scanning mirror is driven and controlled to produce horizontal rotation and vertical rotation. The horizontal rotation can realize scanning and irradiation of the target in the horizontal direction by deflecting the outgoing electromagnetic waves. The vertical rotation can change the optical path of the incident electromagnetic waves to different rows for scanning. If the vertical direction remains unchanged, the beam scanning module 12 will repeatedly scan the same row back and forth.

[0098] Based on this, the beam scanning module 12 can also be configured to output a scanning trajectory signal and an angle signal. The scanning trajectory signal is used to indicate different scanning rounds. The angle signal can include a horizontal rotation angle (denoted as an angle signal x) and a vertical rotation angle (denoted as an angle signal y).

[0099] 3) an echo receiving module 13, configured to receive echo signals reflected by electromagnetic waves scanned to the object. The echo receiving module 13 is configured to generate a plurality of depth distances of point clouds by processing the received echo signals according to the point cloud data generation method provided in the embodiments of the present application.

[0100] From the hardware layer, in one example, referring to FIG. 7, the detection device 10 is a single-transmit-single-receive detector, i.e., the electromagnetic wave transmitting module 11 includes a single transmitter, and the echo receiving module 13 includes a single receiver. In another example, the detection device 10 is a multi-transmit-multi-receive detector, i.e., the electromagnetic wave transmitting module 11 includes a plurality of transmitters, and the echo receiving module 13 includes a plurality of receivers. The detection device 10 provided in the present application can realize high-density point cloud output on the basis of single-transmit-single-receive. In another example, referring to FIG. 8, the detection device 10 is a multi-transmit-multi-receive detector, as shown in FIG. 8, the detection device 10 includes a plurality of transmitters and a plurality of receivers. It can be understood that N-transmit-N-receive can increase the point cloud density by N times relative to single-transmit-single-receive. Therefore, the following will be introduced by taking single-transmit-single-receive as an example.

[0101] 4) a point cloud generation module 14, configured to generate point cloud data based on the angle signal and the depth distances generated by the echo receiving module 13. Optionally, the point cloud data can also be generated in combination with the scanning track signal.

[0102] 5) an image generation module 15, configured to generate an image based on the echo signal, and the image contains the target object.

[0103] a clock control module 15, configured to output a clock signal to each module for control, synchronization, triggering, etc.

[0104] The above introduces the structure of the detection device 10, and it needs to be explained that FIG. 6 is only an example, and the detection device 10 designed in the present application can include more or fewer components relative to FIG. 6, for example, in another implementation, the detection device 10 does not include the point cloud generation module 14, but only includes the electromagnetic wave transmitting module 11-echo receiving module 13, the image generation module 15 and the clock control module 15.

[0105] The following will take the detection device 10 shown in FIG. 6 as an example to introduce a method for generating point cloud data provided in the embodiments of the present application.

[0106] FIG. 9 is a flowchart of a method for generating point cloud data provided in the embodiments of the present application. The method can include the following steps:

[0107] Step 901, transmitting electromagnetic waves to perform multiple rounds of scanning on the target object, and receiving echo signals reflected by the target object in each round of scanning.

[0108] In the embodiments of the present application, the scanning mode can include multi-round scanning, and at least two frequencies are used in the multi-round scanning. Each round of scanning corresponds to a sub-track, and the scanning path of the multi-round scanning includes the sub-tracks corresponding to each round of scanning.

[0109] Referring to FIG. 10A, scanning is performed with the scanning parameters shown in FIG. 10A, and taking one row of scanning as one round of scanning as an example, it can be understood in combination with FIG. 10B that the scanning process includes: when performing the first row of scanning, the electromagnetic wave emitting module 11 emits electromagnetic waves at the frequency f1. After completing the first row of scanning, the angle signal x is reset quickly, and the next row (for example, the angle signal y+1) is moved to perform the second row of scanning. When performing the second row of scanning, the electromagnetic wave emitting module 11 emits electromagnetic waves at the frequency f2. After completing the second row of scanning, the angle signal is reset quickly, and the next row is moved to perform the third row of scanning. And so on.

[0110] Each row of scanning corresponds to a sub-track, as shown in FIG. 10C, the scanning track of the first row is sub-track 1, the scanning track of the second row is sub-track 2, the scanning track of the third row is sub-track 3, and so on. Taking laser pulses as an example, each black dot in FIG. 10C represents a laser pulse. As can be seen, different frequencies have different emission intervals between laser pulses. The higher the emission frequency, the shorter the emission interval. Conversely, the lower the emission frequency, the longer the emission interval.

[0111] At step 902, for each pair of sub-tracks in the plurality of sub-tracks, a depth distance of a plurality of sampling points corresponding to each pair of echo signals included in the pair of sub-tracks is generated.

[0112] In one example, the two sub-tracks in a pair of sub-tracks are adjacent. Taking the plurality of sub-tracks shown in FIG. 10C as an example, the pairs of sub-tracks are: the first pair: (sub-track 1, sub-track 2), the second pair: (sub-track 2, sub-track 3), the third pair: (sub-track 3, sub-track 4), the fourth pair: (sub-track 4, sub-track 5), and so on.

[0113] In one example, the two sub-tracks in a pair of sub-tracks are adjacent. Taking the plurality of sub-tracks shown in FIG. 10C as an example, the pairs of sub-tracks are: the first pair: (sub-track 1, sub-track 2), the second pair: (sub-track 2, sub-track 3), the third pair: (sub-track 3, sub-track 4), the fourth pair: (sub-track 4, sub-track 5), and so on.

[0114] In one example, the two sub-tracks in a pair of sub-tracks are adjacent. Taking the plurality of sub-tracks shown in FIG. 10C as an example, the pairs of sub-tracks are: the first pair: (sub-track 1, sub-track 2), the second pair: (sub-track 2, sub-track 3), the third pair: (sub-track 3, sub-track 4), the fourth pair: (sub-track 4, sub-track 5), and so on.

[0115] For example, referring to Figure 11A(a), each black dot represents a laser pulse signal, and the white circle represents a received echo signal. The echo signals are aligned according to the preceding adjacent laser pulse signal. It is worth noting that if aliasing occurs, such as receiving two echo signals in a certain transmission interval in sub-track 3 in Figure 11A(a), the two echo signals are arranged according to the receiving time sequence. See Figure 11A(b). Alternatively, one echo signal can be retained while the other is discarded. The rearranged echo signals are then numbered by row and column according to the scanning order. For example, multiple echo signals in sub-track 1 are sorted and numbered as d. 11 d 12 , ...,d 1m The multiple echo signals in sub-trajectory 2 are sorted and numbered as d. 21 d 22 , ..., d 2m Among them, d 11 This represents the first echo signal in the first row, i.e., the first echo signal on sub-track 1. d 12 This represents the second echo signal in the first row, d. 1m This represents the m-th echo signal in the first row. d 21 This indicates the first echo signal in the second row, i.e., the first echo signal on sub-track 2. d 22 d represents the second echo signal in the second row, and d2m represents the m-th echo signal in the second row. That is, d nm In this context, n represents the row number and m represents the column number.

[0116] It should be noted that the echo signal sorting method shown in Figure 11A is only one example. This application can also sort the echo signals according to the receiving time order, without any specific limitation.

[0117] Based on the sorting method shown in Figure 11A(b), the multiple pairs of echo signals included in the first pair of sub-trajectories include, see Figure 11B: the first pair of echo signals (d) 11 d 21 ), the second pair of echo signals (d 12 d 22 ), the third pair of echo signals (d 13 d 23 ), and so on.

[0118] In step 902, each pair of echo signals is processed to obtain the depth distance corresponding to each pair of echo signals.

[0119] For example, as shown in Figure 11B, based on (d 11 d 21 Generate depth distance D 11 Based on (d)12 , d 22 ) to generate a depth distance D 12 based on (d 13 , d 23 ) to generate a depth distance D 13 , and so on. Wherein, the depth distance represents the distance from the detection device 10 to the target. The depth distances of different point clouds represent the distances from the detection device 10 to different sampling points on the target.

[0120] Taking a pair of echo signals (including a first echo signal and a second echo signal) as an example, the processing manner of the echo signal pair can include: determining the unambiguous distance based on the time information of the first echo signal and the time information of the second echo signal.

[0121] Wherein, the time information is used to indicate the time of flight of the echo signal, for example, the time information of the first echo signal indicates the time of flight of the first echo signal Δt, Δt = the time difference between the receiving time of the first echo signal and the transmitting time of the first pulse signal (as shown in (a) of FIG. 11A), the first pulse signal is the adjacent laser pulse signal in front of the first echo signal. Similarly, the time information of the second echo signal indicates the time of flight of the second echo signal.

[0122] The process of determining the depth distance based on the time information of the echo signal is introduced as follows:

[0123] Firstly, the concept of solving the ambiguous distance problem provided by the present application is introduced based on FIG. 5. As shown in FIG. 5, for the ambiguous distance caused by aliasing, the true distance (i.e. the unambiguous distance, denoted as D) D = n * d per + ambiguous distance; wherein, d per represents the distance corresponding to the pulse transmitting time interval, and the calculation manner is d per represents the distance corresponding to the pulse transmitting time interval, wherein T per represents the pulse interval corresponding to the pulse transmitting frequency f per , and c represents the speed of light.

[0124] It is assumed that in the present embodiment, f1> f2, and satisfies wherein f r is the greatest common divisor of f1and f2, and N and a are both positive integers. For example, in an example, let a be equal to 1, such as when f1= 10 MHz, f2= 9.9 MHz, f r = 10 5 Hz, a = 1.

[0125] D 11 is generated based on the time of flight of the echo signal represented by d 21 , d 11, as understood in conjunction with FIG. 10B, the calculation process is as follows: ΔT 11 = Δt 11 + n1*T1 = Δt 21 + n2*T2 Equation (2)

[0126] Where, Δt 11 represents the time difference of the echo signal identified by d 11 ; Δt 21 represents the time difference of the echo signal identified by d 21 ; 0 ≤ Δt 11 < T1, 0 ≤ Δt 21 < T2; n1, n2 are integers greater than or equal to 0.

[0127] Since it can be obtained that (N + 1)*T1 = N*T2, therefore, Equation (2) can be transformed as follows:

[0128] In one example, when Δt 11 > Δt 21 based on Equation (3), it can be obtained that n1 > n2. When n1 and n2 are both integers, it can be obtained that n1 = n2 + 1. Substituting this into Equation (2) gives:

[0129] Substituting Equation (4) into Equation (2) and Equation (1), D can be obtained 11 .

[0130] In another example, when Δt 11 ≥ Δt 21 it can be obtained that n1 = n2. Substituting this into Equation (2) gives:

[0131] Substituting Equation (5) into Equation (2) and Equation (1), D can be obtained 11 .

[0132] Similarly, D 12 , D 13 , …, D 1m .

[0133] It should be noted that in the embodiments of the present application, the relationships and values of the above formulas and calculation parameters can be configured as preset values according to the above derivation results. Alternatively, the above formulas and calculation parameters can be obtained by a program executing the derivation process, and the embodiments of the present application do not limit this.

[0134] Similarly, the multiple pairs of echo information included in the second pair of sub-trajectories include: (d21 , d 31 ), (d 22 , d 32 ), (d 23 , d 33 ), …, (d 2m , d 3m ). The third pair of sub-trajectories includes a plurality of pairs of echo information: (d 31 , d 41 ), (d 32 , d 42 ), (d 33 , d 43 ), …, (d 3m , d 4m ). And so on. Based on the above manner, each pair of echo signals is processed to obtain D 21 , D 22 , D 23 , …, D 2m . And so on. Based on the angle signal of each echo signal in the plurality of echo signals and the depth distance (determined in step 902), the point cloud data of the target object is generated in step 903.

[0135] As described above, a point cloud includes three-dimensional coordinates (x, y, z) of sampling points. In the embodiments of the present application, the x coordinate can be the angle signal x, the y coordinate can be the angle signal y, and the z coordinate is the depth distance D.

[0136] The point cloud generation module 14 receives the angle signals generated by the beam scanning module 12. As shown in FIG. 11C, the angle signals include the angle signal x and the angle signal y. It should be noted that the signals shown in FIG. 11C are time-synchronized, and the angle signal x and the angle signal y corresponding to each echo signal are determined based on the reception time of each echo signal (or according to the transmission time of the pulse signal corresponding to each echo signal), and the depth distance D nm is generated based on the depth distance D 21 of the echo signal.

[0137] As shown in FIG. 11C, (x 11 , y 11 , D 11 ), (x 12 , y 12 , D 12 ), … (x 1m , y 1m , D 1m ) constitute the first row of the point cloud, (x 21 , y 21 , D 21 ), (x 22 , y 22 , D22 ),…(x 2m ,y 2m ,D 2m ) constitute the second row of point cloud, and so on.

[0138] The above Figs. 11A to 11C are examples with the same scanning direction, and the following examples are introduced with different scanning directions.

[0139] Taking the scanning parameters shown in Fig. 12A as an example, the scanning process includes: when performing the first row scanning, the transmitting frequency f1 is adopted, and the scanning direction is forward, after the first row scanning is completed, the return motion is performed, the horizontal direction is reversed, the angle signal y+1, that is, moving to the next row, and performing the second row scanning. When performing the second row scanning, the transmitting frequency f2 is adopted, and the scanning direction is reverse. After the second row scanning is completed, the third row scanning is performed. And so on. Correspondingly, the scanning path is shown in Fig. 12B, and the scanning path is a snake shape. Based on the above-mentioned sorting and numbering method, the received echo signal is numbered, as shown in Fig. 12C.

[0140] In Fig. 12C, the multiple pairs of sub-trajectories can include: (sub-trajectory 1, sub-trajectory 2), (sub-trajectory 2, sub-trajectory 3), (sub-trajectory 3, sub-trajectory 4), and so on. Taking a pair of sub-trajectories as an example, the multiple pairs of echo signals in the sub-trajectory pair can include: (d 11 , d 2m ), (d 12 , d 2(m-1) ), (d 1m , d 21 ), and so on. Based on (d 11 , d 2m ), the depth distance D 11 is generated, based on (d 12 , d 2(m-1) ), the depth distance D 12 is generated, based on (d 13 , d 2(m-2) ), the depth distance D 13 is generated, and so on. Then, based on the angle signal and the depth distance D nm , the point cloud is generated, as shown in Fig. 12E. The specific method flow can be referred to the above introduction, which will not be repeated here.

[0141] It should be noted that:

[0142] (1) The above examples are illustrated with non-overlapping sub-trajectories. In the embodiments of the present application, the multiple sub-trajectories can be overlapping or non-overlapping. For example, in one example, the sub-trajectories partially overlap. For example, in combination with the scanning parameters shown in FIG. 13A, the scanning process can include: using the transmission frequency f1 when performing the first round of scanning along the first row. After completing the first round of scanning, quickly resetting (i.e., the angle signal x is reset), i.e., the angle signal y is unchanged, and then performing the second round of scanning along the first row using the transmission frequency f2. After completing the second round of scanning, quickly resetting, i.e., the angle signal y+1 (i.e., moving to the next row), performing the third round of scanning along the second row using the transmission frequency f1. After completing the third round of scanning, quickly resetting, i.e., the angle signal y is unchanged, and then performing the fourth round of scanning along the second row using the transmission frequency f2. And so on. As shown in FIG. 13B, the sub-trajectory 1 corresponding to the first round of scanning and the sub-trajectory 2 corresponding to the second round of scanning overlap. The sub-trajectory 3 corresponding to the third round of scanning and the sub-trajectory 4 corresponding to the fourth round of scanning overlap. That is, the sub-trajectories partially overlap.

[0143] In another example, all the sub-trajectories overlap. For example, in the example of FIG. 13A described above, when the multiple rounds of scanning only include two rounds of scanning for the same row, all the sub-trajectories overlap, for example, all the sub-trajectories only include the sub-trajectory 1 and the sub-trajectory 2 in FIG. 13B, or only include the sub-trajectory 3 and the sub-trajectory 4, and so on.

[0144] (2) In the above examples, the shape of the sub-trajectory is a straight line segment, i.e., the angle signal y remains unchanged during the scanning process of each row. In addition to the straight line segment, the embodiments of the present application also support forming sub-trajectories of different shapes. For example, FIG. 14A exemplarily shows another scanning mode, in which each sub-trajectory is in a wave shape. For another example, FIG. 14B exemplarily shows another scanning mode, in which each sub-trajectory is in a zigzag shape. The embodiments of the present application do not limit this.

[0145] (3) The scanning rule in the above examples is only an example. The embodiments of the present application also support other scanning modes, for example, the same transmission frequency is used for multiple consecutive rows. For example, the first row, the second row, and the third row are scanned using the transmission frequency f1, the fourth row is scanned using the transmission frequency f2, the fifth row and the sixth row are scanned using the transmission frequency f1, and the seventh row and the eighth row are scanned using the transmission frequency f2. In summary, at least two different transmission frequencies are used in the multiple rounds of scanning.

[0146] (4) The number of exit points of each round of scanning in the above examples is the same. The embodiments of the present application also support different numbers of exit points in multiple rounds of scanning, including: different numbers of exit points in some rounds, for example, the number of exit points of the first row of scanning is different from the number of exit points of the second round of scanning, the numbers of exit points of the first row, the third row, and the fifth row, etc. n+1 rows are the same, the numbers of exit points of the second row, the fourth row, and the sixth row, etc. 2n rows are the same, and n is an integer. Or, the numbers of exit points of all rounds are different, that is, the number of exit points of each round is different. When the number of exit points is different, only pairs of echo signals can be processed.

[0147] (5) The above is introduced by taking two transmission frequencies in multiple rounds of scanning as an example. In the embodiments of the present application, two or more transmission frequencies can also be used in multiple rounds of scanning, for example, the first row of scanning uses transmission frequency f1, the second row of scanning uses transmission frequency f2, the third row of scanning uses transmission frequency f3, the fourth row of scanning uses transmission frequency f1, the fifth row of scanning uses transmission frequency f2, the sixth row of scanning uses transmission frequency f3, and so on.

[0148] The various scanning modes introduced above can be combined in any way, for example, multiple rounds of scanning can be performed at two or more scanning frequencies, the sub-trajectories can be any shape such as a straight line segment, a wavy line, or a polyline, the number of exit points of each row of scanning can be the same or different, the electromagnetic wave transmission frequencies of adjacent rows can be the same or different, etc. The first paragraph can be scanned in any combination. The embodiments of the present application do not limit this.

[0149] (6) The above is an example of dividing a pair of sub-trajectories by position. The embodiments of the present application do not limit this, for example, in another example, the two sub-trajectories in a pair of sub-trajectories are not adjacent. For example, in Figure 11A (a), the first and second rows of sub-trajectories are a pair, and the third and fourth rows of sub-trajectories are a pair. For another example, in Figure 11A (a), the first and fourth rows of sub-trajectories are a pair, and the second and fifth rows of sub-trajectories are a pair. For another example, in Figure 13B, sub-trajectory 1 and sub-trajectory 2 are a pair, sub-trajectory 3 and sub-trajectory 4 are a pair, and so on. Or, sub-trajectory 1 and sub-trajectory 3 are a pair, sub-trajectory 2 and sub-trajectory 4 are a pair, etc. The embodiments of the present application do not limit the division of the sub-trajectory pair. Regardless of the way the sub-trajectory pair is divided, the processing method of the echo signal pair included in each pair of sub-trajectories is the same in the embodiments of the present application. This will not be described here.

[0150] Through the above design, multiple rounds of scanning are performed using different transmission frequencies, and the depth distance between the detection device and the target object is determined based on the echo signals of two sub-trajectories scanned using different transmission frequencies, rather than calculating the depth distance by the time of flight of the echo signal. This can break through the ambiguity distance limit and generate a high-density point cloud.

[0151] Step 904, generating a target image based on the echo signal, the target image comprising the target object.

[0152] In one embodiment, the target image is generated based on a gray scale signal of the echo signal, the target image being a gray scale image for indicating the brightness of the object. Illustratively, the image generating module 15 can rearrange the received echo signal into a two-dimensional image based on the pulse time sequence, and then generate the target image based on the depth signal of the echo signal corresponding to each pixel in the two-dimensional image.

[0153] Referring to FIG. 15, the method for generating the target image can comprise the following steps:

[0154] Step 1501: mapping the echo signal received by the beam scanning module 12 based on one or more parameters such as the pulse time sequence, the angle signal, the scanning trajectory signal, etc.

[0155] As shown in (a) of FIG. 16A, each black dot represents a transmitted laser pulse signal, and the white circle represents a received echo signal. The echo signal is aligned with the previous adjacent pulse signal. It is worth noting that due to the blurring distance problem, there can be many overlapping echo signals, for example, two or more echo signals are received within one transmission interval. If overlapping occurs, one of the echo signals can be discarded, for example, the echo signal closest to the pulse signal is retained for alignment. In the figure, d indicates that there is no echo signal at this position.

[0156] Based on the angle signal or the scanning trajectory signal, the row and column of each echo signal can be determined. (b) of FIG. 16A illustratively shows the rearranged echo signal, which is numbered by row number and column number to generate an index value of each echo signal, which is used to indicate the position of the echo signal in the two-dimensional image, for example, d 11 indicates the first echo signal in the first row, i.e., the first echo signal on the sub-track 1. 12 indicates the second echo signal in the first row. 1m indicates the mth echo signal in the first row. 21 indicates the first echo signal in the second row, i.e., the first echo signal on the sub-track 2. 22 indicates the second echo signal in the second row, and d nm wherein n represents the row number, and m represents the column number.

[0157] Step 1502: gridizing the spatial position indicated by the scanning parameter, and integerizing the coordinate value to generate an image array.

[0158] For example, the number of rows of the two-dimensional image is determined based on the angle signal y, the number of columns of the two-dimensional image is determined based on the pulse time sequence, and the image array includes the integerized coordinates of each pixel, as shown in (a) of FIG. 16B. It should be understood that the integerized coordinate values shown in (a) of FIG. 16B are only an example.

[0159] Step 1503: Assign the corresponding echo signal to the pixel (or coordinate point) based on the index value of the echo signal.

[0160] For example, as shown in (b) of FIG. 16B, the echo signal is assigned to the pixel at the corresponding position according to the index value. For example, assuming that the position indicated by the index value of the echo signal is the first row and the first column (as shown in (d) of FIG. 16B), the echo signal is assigned to the pixel corresponding to the coordinate (0, 0). 11 If the position indicated by the index value is the first row and the second column (as shown in (e) of FIG. 16B), the echo signal is assigned to the pixel corresponding to the coordinate (0, 1). 12 If the position indicated by the index value is the first row and the second column (as shown in (e) of FIG. 16B), the echo signal is assigned to the pixel corresponding to the coordinate (0, 1).

[0161] Step 1504: Generate a target image based on the gray-scale signal of the echo signal corresponding to each pixel.

[0162] It should be noted that the arrangement of the coordinate values of each pixel shown in (a) of FIG. 16B is only an example, and the specific arrangement is not limited. In addition, the mapping manner of the echo signal and the pulse signal when generating the image and the sorting manner of the echo signal based on the pulse signal when calculating the depth distance D can be independent of each other, can each adopt a different manner, or can adopt the same manner, and the specific arrangement is not limited.

[0163] In another embodiment, the gray-scale signal of the echo signal corresponding to each pixel in the above embodiment is replaced by the depth signal of the echo signal, and another target image can be generated.

[0164] Through the above design, the detection device 10 outputs both point cloud data and images, and compared with a sensor combination such as a camera-radar, the detection device can realize a true radar-vision all-in-one machine.

[0165] The hardware control manner of some modules in the detection device 10 is introduced as follows in combination with the above method embodiment.

[0166] Electromagnetic wave emission module 11: used for receiving a scanning trajectory signal (such as a line number), and setting an electromagnetic wave emission frequency (or a laser pulse repetition frequency) according to the scanning trajectory signal. Referring to FIG. 17, in an example, the electromagnetic wave emission module 11 includes a laser driver, a laser diode, and an optical path. The laser driver receives the scanning trajectory signal, i.e., the line signal, in the method embodiment of FIG. 9, and outputs an electrical signal of a certain frequency to control the laser diode to emit light. The line signal here is used to trigger the change of the frequency of the electrical signal driving the laser, so as to change the laser pulse repetition frequency. The laser diode emits laser light, which is input into the beam scanning module 12 after passing through the optical path. The optical path is used for collimating the laser light emitted by the laser diode, and in specific embodiments, the optical path can be a collimating lens or a collimating lens group.

[0167] Beam scanning module 12: used for receiving an electromagnetic wave beam and scanning the electromagnetic wave beam, and outputting scanning trajectory information and an angle signal. Optionally, a pulse time sequence can also be output.

[0168] The beam scanning module 12 can deflect the laser beam to scan the target object from the forward direction or the reverse direction by using a MEMS scanning mirror, a voice coil scanning mirror, an optical phased array (OPA), etc. Taking the MEMS scanning mirror as an example, referring to FIG. 18, in this embodiment, the beam scanning module 12 is composed of a 2-dimensional MEMS scanning mirror and a driving circuit thereof. In addition, the beam scanning module 12 can also be composed of a galvanometer type 2-dimensional scanning mirror and a driving circuit thereof, or a 2-dimensional voice coil type scanning mirror and a driving circuit thereof, or a 2-dimensional optical phased array and a driving circuit thereof. Among them, 2-dimensional includes an angle signal x and an angle signal y, i.e., the driving can control the rotation of the MEMS scanning mirror in the x direction and the rotation in the y direction. Referring to FIG. 19, the scanning light beam can also be emitted after passing through a suitable shaping optical path. FIGS. 18 and 19 are illustrated by taking a single-transmit-single-receive module as an example, and when the detection device 10 includes a multi-receive-multi-transmit module, multiple light beams can also be scanned simultaneously.

[0169] Echo receiving module 13: used for receiving an echo signal. Referring to FIG. 20(a), the echo signal entering the echo receiving module 13 can be focused on a light detector by a lens. Alternatively, referring to FIG. 20(b), the echo signal can be directly irradiated onto the light detector. The light detector is matched with the light source, and the light detector includes but is not limited to a photoelectric detector (PD), an avalanche photo diode (APD), a single photon avalanche diode (SPAD), a photo multiplier tube (PMT), a multi-pixel photon counter (MPPC), etc. Optionally, a filter of a corresponding wave band can be added in front of the light detector. The specific implementation is not limited.

[0170] The echo receiving module 13 is also configured to output the depth distance information. Referring to FIG. 21, in an embodiment, the echo signal can be a laser beam. The laser beam is received by a detector via a shaping optical path. In a specific implementation, the shaping optical path is a focusing lens, and the detector can be an APD and its circuit, or a SiPM and its circuit, or a PD and its circuit, or a PMT and its circuit. The time-to-digital conversion unit is composed of a time-to-digital converter (TDC). In a specific implementation, the electric pulse signal of the electromagnetic wave transmitting module and the electric pulse signal generated by the detector unit from the echo signal are input to an ADC circuit. The ADC circuit is configured to extract the reflectivity intensity carried by the echo signal, thereby obtaining a gray signal. One of the signals is input to a TDC, and the TDC outputs a time vector. The depth distance (D nm ) is generated based on the time vectors of adjacent rows of echo signals according to the method shown in step 902 of the method embodiment of FIG. 9, and output to the point cloud generation module.

[0171] The application further provides a detection device 2400. FIG. 22 is a structural schematic diagram of a detection device 2400 according to an embodiment of the application. The detection device 2400 can be applied to the vehicle, the camera, or various terminal devices described above. Referring to FIG. 22, the detection device 2400 can include a transceiver 2401 and a chip 2402.

[0172] In terms of functions, in an optional design, the transceiver 2401 is configured to perform the functions of the electromagnetic wave transmitting module 11 to the echo receiving module 13 of the device 10 shown in FIG. 6, and the chip 2402 is configured to perform the functions of the point cloud generation module 14 to the image generation module 15. In terms of hardware, in a design, the transceiver 2401 can include the electromagnetic wave transmitting module 11 to the echo receiving module 13. The chip 2402 can be, but is not limited to, any one or more of a central processing unit (CPU) data processing unit (DPU), a system on chip (SOC), a field programmable gate array (FPGA), a graphics processing unit (GPU), a special application integrated circuit (ASIC), and the like.

[0173] Based on the above embodiments, the embodiments of the present application further provide a computer program, which, when executed on a computer, causes the computer to perform the point cloud data generation method provided by the embodiment shown in FIG. 9.

[0174] Based on the above embodiments, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program, when executed on a computer, causes the computer to perform the point cloud data generation method provided by the embodiment shown in FIG. 9. The storage medium can be any available medium that can be accessed by a computer. Examples of the computer readable medium include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0175] Based on the above embodiments, the embodiments of the present application further provide a chip for reading a computer program stored in a memory, and implementing the point cloud data generation method provided by the embodiment shown in FIG. 9.

[0176] Based on the above embodiments, the embodiments of the present application provide a chip system, which includes a processor for supporting a computer device to implement the point cloud data generation method provided by the embodiment shown in FIG. 9. In a possible design, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0177] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0179] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0181] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for generating point cloud data, the method comprising: The method is applied to a detection device, and the method comprises: Performing multiple rounds of scanning on a target object by emitting electromagnetic waves, and receiving echo signals reflected by the electromagnetic waves of each round of scanning on the target object; the scanning tracks of the multiple rounds of scanning comprise multiple sub-tracks, one sub-track corresponds to one round of scanning, and the multiple sub-tracks comprise at least one pair of sub-tracks; a first pair of sub-tracks in the at least one pair of sub-tracks comprises an ith sub-track and a jth sub-track, the ith sub-track corresponds to an ith round of scanning, the jth sub-track corresponds to a jth round of scanning, and the electromagnetic wave emission frequency of the ith round of scanning is different from the electromagnetic wave emission frequency of the jth round of scanning, wherein i and j are positive integers, and the first pair of sub-tracks is any one pair of sub-tracks in the at least one pair of sub-tracks; Generating point cloud data based on the echo signals on the ith sub-track and the echo signals on the jth sub-track, the point cloud data being used to indicate the distance between the device and the target object.

2. The method of claim 1, wherein, The method further comprises: Generating a target image based on the received echo signals, the target image comprising the target object.

3. The method of claim 1 or 2, wherein, The scanning direction of the ith round of scanning is a first direction, the scanning direction of the jth round of scanning is a second direction, the first direction is the same as the second direction, or the first direction is opposite to the second direction.

4. The method according to any one of claims 1 to 3, characterized in that, The first pair of sub-tracks is any two adjacent sub-tracks in the multiple sub-tracks.

5. The method according to any one of claims 1 to 4, characterized in that, The echo signals received on the ith sub-track comprise multiple first echo signals, and the echo signals received on the jth sub-track comprise multiple second echo signals; the multiple first echo signals and the multiple second echo signals form multiple pairs of echo signals, each pair of echo signals comprising one first echo signal and one second echo signal; Generating the point cloud data of one sub-track based on the echo signals received on the ith sub-track and the echo signals received on the jth sub-track comprises: For a first pair of echo signals in the multiple pairs of echo signals, determining a non-ambiguous distance based on time information of the first echo signal and based on time information of the second echo signal; the first pair of echo signals is any one pair in the multiple pairs of echo signals; the time information of the first echo signal indicates a time difference value between the emission time of a first electromagnetic wave and the reception time of the first echo signal, the first electromagnetic wave being an electromagnetic wave emitted before the first echo signal is received; the time information of the second echo signal indicates a time difference value between the emission time of a second electromagnetic wave and the reception time of the second echo signal, the second electromagnetic wave being an electromagnetic wave emitted before the second echo signal is received; The point cloud data comprises multiple non-ambiguous distances determined based on the multiple pairs of echo signals.

6. The method of claim 5, wherein, The first echo signal and the second echo signal comprised in the first pair of echo signals are two echo signals adjacent in position in the ith sub-track and the jth sub-track.

7. A detection device, characterized in that The device comprises: An electromagnetic wave emission module, configured to emit electromagnetic waves at a set emission frequency corresponding to a scanning round. The beam scanning module is configured to deflect electromagnetic waves to perform multiple rounds of scanning on the target object. The scanning track of the multiple rounds of scanning includes multiple sub-tracks, one sub-track corresponds to one round of scanning, and the multiple sub-tracks include at least one pair of sub-tracks. The first pair of sub-tracks in the at least one pair of sub-tracks includes an ith sub-track and a jth sub-track, the ith sub-track corresponds to an ith round of scanning, the jth sub-track corresponds to a jth round of scanning, the electromagnetic wave emission frequency of the ith round of scanning is different from the electromagnetic wave emission frequency of the jth round of scanning, where i and j are positive integers, and the first pair of sub-tracks is any pair of sub-tracks in the at least one pair of sub-tracks. The echo receiving module is configured to receive an echo signal reflected by the target object in each round of scanning, and generate a plurality of point cloud corresponding depth distances based on the echo signals on the two sub-tracks included in each pair of sub-tracks. The point cloud generating module is configured to generate point cloud data based on the point cloud corresponding depth distances.

8. The apparatus of claim 7, wherein, The device further includes an image generating module. The image generating module is configured to generate a target image based on the echo signal, and the target image includes the target object.

9. The apparatus of claim 7 or 8, wherein, The scanning direction of the ith round of scanning is a first direction, the scanning direction of the jth round of scanning is a second direction, the first direction is the same as the second direction, or the first direction is opposite to the second direction.

10. The device of any one of claims 7-9, wherein, The first pair of sub-tracks is any two adjacent sub-tracks in the multiple sub-tracks.

11. The device of any one of claims 7-10, wherein, The first echo signal and the second echo signal included in the first pair of echo signals are two echo signals adjacent in position in the ith sub-track and the jth sub-track.

12. A vehicle characterized by comprising: The device includes a detection device configured to perform the method of any one of claims 1-6.

13. A camera, characterized by The device includes a detection device configured to perform the method of any one of claims 1-6.

14. A terminal device, comprising: The device includes a detection device configured to perform the method of any one of claims 1-6.

Images