METHOD FOR PROCESSING RADAR POINT CLOUD DATA

Abstract

A processing method for radar point cloud data is provided, comprising the following: receiving multiple targets generated by acquisition using radar point cloud data; identifying a first target and one or more corresponding second targets from the multitude of targets exhibiting similar motion characteristics and overlapping with each other; and recognizing the first target and the one or more second targets as corresponding to the same acquisition object. In this way, multiple targets generated by the same acquisition object during the radar point cloud data acquisition process can be identified, thus facilitating the subsequent accurate detection and classification of the acquisition object.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of data processing, in particular the processing of radar point cloud data. STATE OF THE ART

[0002] Due to their adaptability to complex environments and their favorable price-performance ratio, millimeter-wave radar products are widely used in autonomous driving for a variety of automated driving functions, such as automatic emergency braking (AEB), adaptive cruise control (ACC), etc.

[0003] In the sensing process based on millimeter-wave radar point cloud data, targets are generated from the point cloud data of the millimeter-wave radar sensor, for example, through clustering. These targets are used as sensing objects for subsequent target detection and classification. Because millimeter-wave radar point cloud data is relatively sparse, it is difficult to describe the entire outline of a sensing object (e.g., a vehicle), and therefore it is difficult to distinguish different parts of the object under investigation. For example, millimeter-wave radar point cloud data of the same sensing object may be detected as multiple targets, which affects the accuracy of the driving assistance function and results in a poor user experience. DISCLOSURE OF THE INVENTION

[0004] It is desirable to provide a method and apparatus for processing radar point cloud data generated from the same sensing object during the sensing process from radar point cloud data, which facilitates the subsequent accurate detection and classification of the sensing object.

[0005] According to one aspect, there is provided a radar point cloud data processing method, comprising: obtaining a plurality of targets generated by detection from radar point cloud data; identifying a first target and one or more corresponding second targets from the plurality of targets that have similar motion characteristics and overlap with each other; and identifying the first target and the one or more second targets as corresponding to the same detection object.According to one aspect, there is provided a processing method for radar point cloud data, comprising: a detection unit that obtains the plurality of targets generated by detection from radar point cloud data; and a processing unit that identifies a first target and one or more corresponding second targets from the plurality of targets that have similar movement characteristics and overlap with each other; and recognizing the first target and the one or more second targets as corresponding to the same detection object.

[0006] According to one aspect, a radar point cloud data processing apparatus is provided, comprising: a memory and a processor that performs a method of processing radar point cloud data according to various embodiments of the present invention.

[0007] According to a further aspect, a computer-readable medium is provided that stores a computer program, wherein the computer program, when executed by a processor, initiates the processor to perform a method according to the method described in various embodiments of the present invention.

[0008] According to various embodiments of aspects of the present invention, different targets generated by the same detection object are detected based on both the similarity of the motion characteristics and the overlap of the targets. Specifically, different targets corresponding to the same detection object can be accurately identified by collectively detecting common correlation points of the different targets and their overlap. Based on the detection results, the target detection results can be adjusted, thereby making subsequent target detection and classification more accurate and improving the accuracy of the driving assistance functions. DESCRIPTION OF THE CHARACTERS

[0009] In the figures, the embodiments are explained only by way of example and not by way of limitation, and in the figures, like reference numerals indicate like elements. Fig. 1 shows a scene where a truck is in the front area of ​​a vehicle; Fig. Figure 2 shows a schematic representation of the target acquisition results for the Fig. 1 depicted scene; Fig. 3 shows a flowchart of a method for processing image data according to an embodiment of the present invention; Fig. 4 shows the detection result after removing the first in Fig. 2 according to an embodiment of the present invention; Fig. 5 shows a flowchart of a method for detecting a first target and corresponding one or more second targets from a plurality of targets having similar motion characteristics and overlapping each other, according to an embodiment of the present invention; Fig. 6 shows a scenario for determining correlation points according to an embodiment of the present invention; Fig. 7 shows a block diagram of a radar point cloud data processing apparatus according to an embodiment of the present invention.

[0010] The individual aspects and features of the individual embodiments of the present invention are described with reference to the figures mentioned. These figures are merely schematic and not limiting. Furthermore, not all parts of the devices of the embodiments of the present invention are identified by reference numerals in the figures mentioned, and in some figures, only relevant elements are identified, which does not limit the individual elements to those illustrated in the figures of the description. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] In a scenario like Fig. 1, in which the radar sensor detects the surroundings of a vehicle and performs assistive driving functions, when a truck appears in front of the vehicle, the radar sensor detects signals reflected from various reflection points on the truck and forms point cloud data. With the help of data processing, the presence of a target in front of the vehicle can be detected from point cloud data, and the target can then be identified and classified, thereby determining that a vehicle (truck) is in front of the current vehicle. Based on the detection of the preceding vehicle, various assistive driving functions such as emergency braking can be performed.

[0012] The point cloud data acquired by millimeter-wave radar sensors is relatively sparse, so during the target detection process based on such sparse point cloud data, each point in the point cloud data corresponding to a detection object may be detected as multiple targets. For example, during clustering of multiple points, the relatively large distance between some points and another point range may result in detection as different targets. As shown in Fig. 2 (only as a schematic diagram) is detected as Target 1 and Target 2 (represented by two bounding boxes). This may result in different targets being identified and classified during subsequent detection and classification, resulting in one detection object being recognized as multiple detection objects. This may affect the accuracy of assisted driving and lead to a poor user experience. For example, in high-speed scenarios, detecting the wrong object may cause the vehicle to falsely decelerate even though, from the user's perspective, there is no reason for deceleration, thus negatively impacting the user experience.

[0013] The present invention detects various targets generated from the same detection object, both in terms of similarity of motion characteristics and overlap of targets. Based on the detection results, the target detection results can be adjusted, thereby making subsequent target detection and classification more accurate and improving the accuracy of driving assistance functions.

[0014] The motion characteristics of multiple targets detected from radar point cloud data corresponding to the same detection object are similar to each other, as the motion characteristics of the individual reflection points corresponding to the same detection object should be fundamentally similar. Furthermore, multiple targets corresponding to the same detection object often overlap (for example, represented as bounding boxes during detection). Based on both, different targets obtained by detecting the same detection object can be identified.

[0015] In particular, a first target and one or more second targets that have similar motion characteristics to the first target and overlap each other can be identified by identifying corresponding targets from the plurality of targets that have common correlation points and overlap each other. The said targets with common correlation points refer to two or more targets that have similar motion characteristics to the same point in the radar point cloud data, wherein the same point is the common correlation point of the two or more targets, wherein one point may be contained within the two target ranges or outside the two target ranges. In one embodiment, this point may be a point within a predetermined range including the two targets, but outside the range of the two targets.The present invention has discovered that different targets detected by the same detection object typically share one or more common correlation points. Multiple targets corresponding to the same detection object can be accurately identified by detecting common correlation points and overlaps between them.

[0016] Fig. 3 shows a flowchart of a method 100 for processing radar point cloud data according to an embodiment of the present invention, wherein, as in Fig. 3, in step 110, a plurality of targets (as indicated by the bounding boxes in Fig. 2) generated by acquiring the radar point cloud data, said plurality of targets comprising a first target and one or more second targets (such as the one shown in Fig. 2 shown first target 1 and the second target 2)

[0017] In step 120, one or more second targets are identified from the plurality of targets that have similar motion characteristics and overlap with each other. The aforementioned similar motion characteristics refer to the difference in motion characteristics between two or more compared objects being within a predetermined threshold. If the difference between the motion characteristic of the first target and the motion characteristic of every second target of the one or more second targets is within a predetermined threshold, this means that the first target and the one or more second targets have similar motion characteristics. The user can preset a specific threshold based on experience. In one embodiment, multiple targets can be multiple targets within a predetermined range. The predetermined range can, for example, be preset by a user.In one scenario, the predetermined area may be a specified front area of ​​the current vehicle. In another scenario, once the target has been detected, the specified area may be a specified area around a detected target.

[0018] Motion properties refer to parameters that characterize the motion characteristics of the object, which can include at least one of the following: distance, speed, and radar cross-sectional area. Each point in the radar point cloud data is detected due to the reflection of radar waves from reflection points on the object, where each point has several attributes, including size, speed, and radar cross-sectional area. The detected motion properties of the target can be determined based on the motion characteristics of each point contained in the target or can be directly selected to represent the motion characteristics of points at specific positions on the target.Considering that in practice, users attach more importance to the positional relationship between the rear end of the preceding vehicle and their own vehicle, in one embodiment, the movement characteristics of a certain point at the rear end of the preceding target may be determined based on the movement characteristics of each point included in the target to represent the movement characteristics of the target, and this point may be used as a reference point for the target.

[0019] Various methods can be used to identify a first target and one or more corresponding second targets from the plurality of targets, both of which (1) have similar motion characteristics and (2) overlap each other. It is expected that targets with similar motion characteristics are first identified among several targets, and then a decision is made as to whether these targets with similar motion characteristics overlap; the subsequent Fig. The description relating to the method shown in Figure 5 is not restrictive; rather, it is conceivable that overlapping targets are first identified among several targets and then targets with similar movement properties are identified among these overlapping targets.

[0020] The present invention has discovered that different targets detected by the same detection object typically have one or more common correlation points. The motion characteristics of the correlation points of a particular target are similar to the motion characteristics of the target, meaning that the difference between the motion characteristics of the two is within a permissible, predetermined threshold, e.g., below the predetermined threshold. The differences between the motion characteristics of the common correlation points of the two targets and the motion characteristics of the two targets are each within a permissible, predetermined threshold.The correlation points of a target must not only lie within the target's range (as represented by the bounding box), but can also lie outside the target's range (as represented by the bounding box). Similarly, the common correlation points of two targets can also lie outside the range of both targets. By detecting whether different objects have similar motion characteristics, it is possible to determine whether the different targets have the same correlation points. The following describes in detail how the first target and the corresponding one or more second targets are identified.

[0021] In step 130, after determining the first target and one or more corresponding second targets that have similar motion characteristics and overlap with each other, it may be determined that these first targets and second targets correspond to the same detection object. The detection result may be used as a basis for adjusting the detection result (detected target) for the detection object, thereby achieving a more accurate detection result and facilitating subsequent target recognition and classification. In one embodiment, the detection result of the present invention may be a target with added attributes. For example, the attributes of the first target may include the identifiers of one or more second targets designated as a corresponding detection object, and the score of each second target and the first target corresponds to the same detection object.

[0022] This value may be determined based on the similarity and overlap of motion characteristics. In particular, during the execution of said step 120, the similarity between the motion characteristics of the first moving target and the motion characteristics of each of the one or more second moving targets may be determined; an overlap rate between the first moving target and each of the one or more second moving targets may be determined; based on the similarity and the overlap rate, a score indicating that the first moving target and each of the one or more second targets correspond to the same detection object is determined; thereafter, the first target, the one or more second targets, and the corresponding scores are output, wherein the detection of the second targets and the corresponding scores may be used as attributes of the first targets.

[0023] After determining that the first target and one or more second targets correspond to the same detection object, it can be determined how the detection result should be adjusted based on the aforementioned attributes of the first target and the relative position of the first target and the current vehicle. In one case, the first target can be removed. In another case, the first target can be merged with the second target. Fig. 4 Fig. 4 shows an example after removing the first in Fig. 2. The target indicated by the arrow in Fig. The box displayed shows the only remaining target after removal. Based on the detection results after adjustment, the targets can be identified and classified, providing more accurate and reliable results and thereby increasing the accuracy of assisted driving functions.

[0024] The following describes in detail how one or more second targets are identified among several targets that have similar movement characteristics to the first target and overlap in the aforementioned step 120. Fig. 5 shows a flowchart of a method 200 for identifying a first target and one or more corresponding second targets among a plurality of targets having similar motion characteristics and overlapping each other according to an embodiment of the present invention.

[0025] According to method 200, a first reference point on the first target is obtained in step 210, which may correspond to a specific position at the rear of the target. The motion characteristics of the first reference point may be derived based on the motion characteristics of each point in the first target.

[0026] In step 220, one or more points in the radar point cloud data that have similar motion characteristics to the first reference point are identified as correlation points of the first target within a preset range (e.g., a range of 10 meters from the current vehicle) including the first target. If the difference between the motion characteristics of the first reference point and the motion characteristics of a point within a predetermined range is within a first predetermined threshold, this means that the first target and this point have similar motion characteristics; conversely, they do not have similar motion characteristics. One or more points that are correlation points of the first target can be points within the range of the first target or points within a predetermined range but outside the range of the first target. Fig. Figure 6 shows a situation where the correlation points of the first target are outside the range of the first target. As in Fig. As shown in Figure 6, the first reference point P1 of the first target is associated with a point P in the point cloud data outside the area of ​​the first target. For clarity, the points P1, P2 and P are Fig. 6, shown as black dots, which is purely schematic.

[0027] After the correlation points of the first target have been identified within the predetermined range in step 220, it can be determined in further steps whether these correlation points of the first target are also correlation points of other targets, wherein two targets that have correlation points in common can be assumed to have similar movement properties.

[0028] Specifically, in step 230, corresponding second reference points of other targets from the plurality of targets except for the first target are obtained, wherein the second reference points are also positions at the rear of the corresponding targets. As shown in Fig. As shown in Figure 6, P2 is the reference point of the second target.

[0029] In step 240, it is determined whether any of the one or more points that are correlation points of the first target has similar motion characteristics as at least one of the corresponding second reference points. That is, it is determined whether one of the one or more points that are correlation points of the first target is also a correlation point of the other target. If the difference between the motion characteristics of any of the one or more points and the motion characteristics of at least one second reference point is within the second predetermined threshold, then this indicates that the second reference point and any point have similar motion characteristics, that is,Among the one or more points that are correlation points for the first target, there is a point that has similar motion characteristics to at least one second reference point among the corresponding second reference points and can equally be a correlation point for other targets to which the at least one second reference point corresponds. On the other hand, no point among the one or more points that is the correlation point of the first target is equally the correlation point of other targets. As shown in FIG. Fig. As shown in Figure 6, the associated point P of the first target is also associated with the second reference point P2. Said first predetermined threshold, second predetermined threshold, and predetermined threshold may be the same or different predetermined thresholds. It may be set manually.

[0030] If it is determined in step 240 that any point of the one or more points that are correlation points of the first target has similar motion properties as at least one of the corresponding second reference points, this means that each point can serve as the same correlation point shared by other targets that correspond to the first target and at least one second reference point, for example, if point P in Fig. is a common correlation point of two targets, other targets corresponding to the first target and at least one second reference point can be identified as targets with similar motion characteristics in step 250. If it is determined in step 240 that no such point exists, the process continues with step 280 and the detection fails.

[0031] For the identified first target and other targets with similar motion characteristics, a decision is made in step 260 as to whether they overlap. In one embodiment, it may be determined whether the first target and the corresponding other target overlap based on the distance between the first target and the corresponding other target with similar motion characteristics and the size of the other target and the first target.

[0032] If the first target and any other target with similar motion characteristics overlap, it is determined in step 270 that the first target and one or more corresponding second targets with similar motion characteristics that overlap each other are identified, and the identified other targets that overlap with the first target and have similar motion characteristics serve as the second targets. In the future, in step 130, which is described in Fig. 3, such a first target and the corresponding one or more second targets are identified as corresponding to the same detection object. If the first target and any other target corresponding to at least one second reference point do not overlap, this means that the detection has failed, and in step 280, it is determined that the first target and the corresponding one or more second targets with similar motion characteristics and mutually overlapping cannot be identified.

[0033] The said reference first determines correlation points of the first target (ie, one or more of the said points) and then determines whether any point among the correlation points of the first target is associated with other targets to find the first target and other targets sharing the same correlation points, which is not limitative, the correlation points of multiple targets may also be determined simultaneously, and then targets having common correlation points may be identified as targets having similar motion characteristics, and the identified targets may include the first target.

[0034] Specifically, in one embodiment, a reference point is obtained for each of the plurality of targets within a predetermined range; a corresponding point or points in the radar point cloud data within this predetermined range that have similar motion characteristics to the reference point of each of the targets are identified as a correlation point for that target. After identifying the corresponding point or points that are the correlation points for each target, it is possible to perform comparisons between the corresponding point or points that are the correlation points for each target to find the points that collectively serve as correlation points.For example, if the correlation points of one target are assumed to be A and B, and the correlation points of another target are assumed to be B and C, then the associated points A and B can be compared with the correlation points B and C to find the common correlation point B. After the common correlation points are found, multiple targets with at least one common correlation point can be identified as having similar motion characteristics. The identified targets with similar motion characteristics include the first target and the one or more second targets with similar motion characteristics that overlap with each other.

[0035] In this case, it is possible to identify further overlapping targets from the targets with similar movement characteristics, whereby the identified targets are a first target and one or more second targets with similar movement characteristics that overlap each other.

[0036] A method for processing radar point cloud data according to the present invention, which is described with reference to Fig. The method described in sections 1-6 is particularly suitable for processing point cloud data from millimeter-wave radar. It is understood that the described processing method does not represent a limitation, and that modifications / changes to one or more steps of the described methods can also achieve the desired effect.

[0037] Fig. Figure 7 shows a block diagram of a radar point cloud data processing device 10 according to an embodiment of the present invention. The device 10 includes an acquisition unit 11 for obtaining various data types, such as point cloud data, multiple targets detected from point cloud data, reference points of each target, etc.

[0038] The device 10 also includes a processing unit 12 used to perform various steps of the processing method according to various embodiments of the present invention, which may be one or more units with processing capabilities. Specifically, the processing unit 12 identifies a first target and one or more corresponding second targets with similar motion characteristics that overlap among multiple targets, and determines that the first target and the one or more second targets correspond to the same detection object. In particular, the processing unit identifies a first target and one or more corresponding second targets by identifying, among a plurality of targets, individual targets that have a common correlation point and overlap with each other.

[0039] Various embodiments of the method and apparatus of the present invention are described with reference to Fig. 1-6, whereby these embodiments can be combined with each other to achieve different effects without being limited by the nature of the subject matter. Furthermore, the described units / steps / processes are not to be understood as limiting. The functions of the described units / steps / processes can be merged / combined / changed / modified to achieve the corresponding effect.

[0040] The functions of each unit of the radar point cloud data processing method and apparatus of the present invention may be realized by software or corresponding hardware, or the functions of each said unit may be realized by means of a processor, wherein the processor may read computer programs from a memory and execute these computer programs to realize the functions of each of the described units.

[0041] It is understood that the methods of the individual embodiments of the present disclosure may be implemented by computer programs / software. This software may be loaded into the memory of a processor and, when executed, may perform a method according to the embodiments of the present disclosure.

[0042] It is understood that the methods and devices according to the present invention can be executed by any device with processing capabilities, even from a remote location. The exemplary embodiments of the present disclosure include both the creation / use of the computer program / computer software of the present disclosure from scratch and the conversion of existing programs / software to use the computer program / computer software of the present disclosure via updates.

[0043] According to further embodiments of the present disclosure, a computer program product is provided, for example, a machine-readable medium (e.g., a computer) such as a CD-ROM containing computer program code. Upon execution of this computer program code, the computer or processor is initiated to perform a method according to the embodiments of the present disclosure. This machine-readable medium is, for example, an optical storage medium or a solid-state storage medium that is integrated into other hardware or delivered as part of other hardware.

[0044] According to further embodiments of the present disclosure, a system for packaging is provided, the system comprising a three-dimensional camera arranged above the packaging container and a device according to various embodiments of the present disclosure.

[0045] Specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that in the embodiments and may still achieve the desired result. Moreover, the process depicted in the figures does not necessarily require a particular order shown or a continuous sequence to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0046] Above, the present disclosure has been described with reference to specific embodiments. Those skilled in the art should understand that the technical solutions of the present disclosure can be implemented in various ways without departing from the spirit and basic features of the disclosure. The specific embodiments are merely schematic and not limiting. Furthermore, these embodiments can be combined in any way to achieve the objectives of the present disclosure. The scope of the present disclosure is defined by the appended claims.

[0047] The term "comprising" in the description and claims does not exclude the presence of additional elements or steps. Expressions such as "first," "second," and "steps" in connection with the individual steps, as well as the sequence of the individual steps shown in the figures, do not define their order or number. The functions of the individual elements explained in the description or listed in the claims may also be performed separately or combined, and may be implemented by multiple elements or a single element.

Claims

[1] A method for processing radar point cloud data, comprising: Acquiring multiple targets generated by acquisition from radar point cloud data; Detecting a first target and one or more corresponding second targets from the plurality of targets that have similar motion characteristics and overlap with each other; and the first target and the one or more second targets are determined as corresponding to the same detection object. [2] The method according to claim 1, wherein by identifying targets having similar movement characteristics from the plurality of targets and identifying targets that overlap from the targets having similar movement characteristics and each other, the first target and the corresponding one or more second targets are identified; or by identifying overlapping targets from the plurality of targets and identifying targets with similar movement characteristics from the overlapping targets, the first target and the corresponding one or more second targets are identified. [3] A processing method according to claim 1 or 2, wherein the one or more second targets are identified from the plurality of targets within a predetermined range. [4] The processing method according to claim 3, wherein by identifying each target from the plurality of targets having a common correlation point and overlapping each other, the first target and the corresponding one or more second targets are identified. [5] A processing method according to claim 4, further comprising: Obtaining a first reference point at the first target; Identifying one or more points in the radar point cloud data with similar motion characteristics to the first reference point within the specified area as correlation points of the first target; Obtaining corresponding second reference points of other targets from the plurality of targets except the first target; Determining whether one of the one or more points exists with similar motion characteristics to at least one second reference point of the corresponding second reference points; If present, targets that include at least one second reference point from the other targets are identified as targets with similar motion properties to the first target. [6] A processing method according to claim 4, further comprising: Obtaining a reference point for each of the multiple targets; Identifying one or more corresponding points in the radar point cloud data with similar motion characteristics to the reference point of each target within the specified area as correlation points of the target; Identifying several targets with at least one identical correlation point as targets with similar movement properties from the plurality of targets and Identifying targets, wherein the targets with similar movement characteristics include the first target and the one or more second targets. [7] A processing method according to any one of claims 5 to 6, further comprising: Identifying targets having similar motion characteristics that overlap with the first target having similar motion characteristics and with each other as the one or more second targets. [8] The processing method according to claim 1, wherein based on the distance between the first target and the other targets among the plurality of targets except the first target, the size of the other targets and the first target is determined whether the first target and the other targets overlap. [9] A processing method according to any one of claims 1 to 6 and 8, further comprising: Determining a degree of similarity between the motion characteristics of the first moving target and the motion characteristics of each of the one or more second moving targets; Determining an overlap rate of the first moving target and with one or more second moving targets; Determining, based on the degree of similarity and the overlap rate, a score indicating that the first target and the one or more second targets correspond to the same detection object; and Output of the first target and one or more second targets as well as the score. [10] A processing method according to any one of claims 1 to 6 and 8, wherein the motion characteristics comprise at least one of the following: distance, speed and cross-sectional area of ​​the radar scatter. [11] A processing method according to any one of claims 1 to 6 and 8, wherein the radar point cloud data is acquired by a millimeter wave radar. [12] Apparatus for processing radar point cloud data, comprising: Detection unit of multiple targets generated by detection from radar point cloud data; Processing unit that selects from the plurality of targets a first target and one or identifies a plurality of corresponding second targets having similar motion characteristics and overlapping each other; and determines the first target and the one or more second targets as corresponding to the same detection object. [13] The processing apparatus of claim 12, wherein the processing unit identifies the first target and the corresponding one or more second targets by identifying individual targets from the plurality of targets that have a common correlation point and overlap each other. [14] Processing device comprising: A memory and A processing apparatus that carries out the radar point cloud data processing method according to any one of claims 1 to 11. [15] A computer-readable medium having stored thereon a computer program, wherein upon execution of the computer program by a processor or a computer, the processor or computer is initiated to execute the radar point cloud data processing method according to any one of claims 1 to 11.

Images