Point cloud denoising method and device, electronic equipment and storage medium

Abstract

The invention discloses a point cloud denoising method and device, electronic equipment and a storage medium, and the denoising method comprises the steps: carrying out the multi-sensor fusion processing in response to a sensing result of a suspected noise attribute, and obtaining an obstacle tracking trajectory; in response to a de-noising process, detecting whether the obstacle tracking trajectory has a suspected noise attribute and the obstacle type is unknown; if yes, whether the obstacle tracking trajectory is in a common-view area of the first sensor and the second sensor is judged; if yes, whether the ratio of the number of times of matching the second sensor to the life cycle meets a set condition or not is judged; and if yes, judging the perception result of the suspected noise attribute as noise. According to the invention, the filtering of noise false detection generated by the laser radar is realized through the characteristics of the millimeter wave radar.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a point cloud denoising method and device, electronic equipment and storage medium. BACKGROUND

[0002] The automatic driving perception suite is divided into two suite schemes, one of which is a laser radar as the main sensor, and the other of which is a camera as the main sensor.

[0003] For the laser-based perception suite scheme, due to the property of the laser radar itself, the laser radar is very sensitive to water mist, dust, rain and snow, and is easy to produce point cloud information. Thus, false detection is caused, which leads the planning module to produce brake and obstacle avoidance strategies, brings passengers an uncomfortable feeling, and even needs to be manually taken over. However, when a human driver drives a car, these are not obstacles, and thus will not cause braking or avoidance, so these are unacceptable for automatic driving. SUMMARY

[0004] The embodiments of the present application provide a point cloud denoising method, device, electronic equipment and storage medium, so as to realize the denoising method of the laser-based main sensor in the automatic driving perception module, that is, to realize the filtering of the noise false detection of the laser radar by the characteristics of the millimeter wave radar.

[0005] The embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a point cloud denoising method, wherein the denoising method comprises:

[0007] In response to the perception result of the suspected noise attribute, a multi-sensor fusion processing is performed to obtain an obstacle tracking trajectory, and the fusion comprises counting a life cycle of each obstacle tracking trajectory in a common view area of a first sensor and a second sensor, a number of times of matching the second sensor, and a number of times of matching other sensors;

[0008] In response to the denoising process, it is detected whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown;

[0009] If yes, it is judged whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor;

[0010] If yes, it is judged whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set condition; and

[0011] If yes, the perception result of the suspected noise attribute is judged as noise.

[0012] In some embodiments, in response to the sensing result of the suspected noise attribute, a multi-sensor fusion process is performed to obtain an obstacle tracking trajectory, and the fusion includes counting a lifetime of each obstacle tracking trajectory in a common view area of the first sensor and the second sensor, a number of times of matching the second sensor, and a number of times of matching other sensors, including:

[0013] In response to the sensing result of the suspected noise attribute, in the process of the multi-sensor fusion, the lifetime tracker_frames of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor is counted;

[0014] The number of times of matching the second sensor match_radar_frames is counted;

[0015] The number of times of matching other sensors match_else_frames is counted;

[0016] The number of times of each fusion process lifetime is added by 1, and the number of times of each successful matching is counted by 1.

[0017] In some embodiments, before the process of responding to the denoising, detecting whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown, further includes:

[0018] Detecting whether the second sensor is normal;

[0019] If the second sensor is normal, starting the process of denoising;

[0020] If the second sensor is abnormal, stopping the process of denoising.

[0021] In some embodiments, in response to the process of denoising, detecting whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown, includes:

[0022] According to the process of denoising, detecting whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown;

[0023] If not, judging the sensing result of the suspected noise attribute as a non-noise attribute.

[0024] In some embodiments, if yes, judging whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor, includes:

[0025] If detecting whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown, judging whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor;

[0026] if the judgment is that the obstacle tracking trajectory is not in the common view area of the first sensor and the second sensor, judging whether a last frame obstacle tracking trajectory is judged as a noise attribute;

[0027] if yes, judging the sensing result of the suspected noise attribute as a noise attribute;

[0028] if no, judging the sensing result of the suspected noise attribute as a non-noise attribute.

[0029] In some embodiments, if yes, judging whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set condition, including:

[0030] judging whether match_radar_frames / tracker_frames is less than a set value;

[0031] if match_radar_frames / tracker_frames is judged as less than the set value, or match_radar_frames < 2, the sensing result of the suspected noise attribute is pending;

[0032] if all the above are no, judging the sensing result of the suspected noise attribute as a non-noise attribute.

[0033] In some embodiments, after the sensing result of the suspected noise attribute is pending, the method further includes:

[0034] judging whether match_else_frames is 0;

[0035] if yes, judging the sensing result of the suspected noise attribute as a noise attribute;

[0036] if no, judging the sensing result of the suspected noise attribute as a non-noise attribute.

[0037] In a second aspect, the embodiments of the present application further provide a point cloud denoising device, wherein the device includes:

[0038] a multi-sensor fusion module, configured to perform multi-sensor fusion processing to obtain an obstacle tracking trajectory in response to a sensing result of a suspected noise attribute, the fusion including counting a life cycle of each obstacle tracking trajectory in a common view area of a first sensor and a second sensor, a number of times of matching the second sensor, and a number of times of matching other sensors;

[0039] a first judging module, configured to detect whether the obstacle tracking trajectory is a suspected noise attribute and an obstacle type is unknown in response to a denoising process;

[0040] a second determining module, configured to determine whether the obstacle tracking trajectory is in a common view area of the first sensor and the second sensor if yes;

[0041] a third determining module, configured to determine whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set value if yes; and

[0042] a fourth determining module, configured to determine the sensing result of the suspected noise attribute as noise if yes.

[0043] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the above method.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores one or more programs, which, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the above method.

[0045] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects: in response to a sensing result of a suspected noise attribute, a multi-sensor fusion processing is performed to obtain an obstacle tracking trajectory; then in response to a noise removal process, it is detected whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown. If yes, it is determined whether the obstacle tracking trajectory is in a common view area of a first sensor and a second sensor; if the determination result is yes, it is determined whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set value; if the determination result is yes, the sensing result of the suspected noise attribute is determined as noise. Through the above method, different sensing characteristics of a noise type obstacle are identified by using a laser radar and a millimeter wave radar, and the accuracy of identifying noise is improved by using multi-frame matching results of the obstacle tracking trajectory for determination. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate embodiments of the present application and the description thereof, and do not constitute improper limitations on the present application. In the drawings:

[0047] Figure 1 a flowchart of a point cloud denoising method in an embodiment of the present application;

[0048] Figure 2 a schematic diagram of an implementation principle of a point cloud denoising method in an embodiment of the present application;

[0049] Figure 3Fig. 1 is a structural schematic diagram of a point cloud denoising device in an embodiment of the present application;

[0050] Figure 4 Fig. 1 is a structural schematic diagram of a point cloud denoising device in an embodiment of the present application; DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in detail with reference to the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0052] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0053] The embodiments of the present application provide a point cloud denoising method, as shown in Figure 1 The embodiments of the present application provide a point cloud denoising method, as shown in

[0054] Step S110, in response to the perception result of the suspected noise attribute, a multi-sensor fusion processing is performed to obtain an obstacle tracking trajectory, and the fusion includes statistics of the life cycle of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor, the number of matching the second sensor, and the number of matching other sensors.

[0055] The perception module of the autonomous driving system needs to give a suspected noise noise attribute according to the point cloud density, point cloud intensity, point cloud distribution characteristics and other information, and then pass the perception result and the noise noise attribute to the fusion module. It should be noted that the point cloud density, point cloud intensity and point cloud distribution characteristics belong to the basic attributes of the point cloud, which can be obtained by methods in various related technologies, and are not specifically limited in the present application.

[0056] It can be understood that the noise, noise attribute, etc. can be distinguished and determined by a laser point cloud recognition AI model or a laser point cloud recognition rule model.

[0057] Step S120, in response to the denoising process, detecting whether the obstacle tracking trajectory is of a suspected noise attribute and the obstacle type is unknown.

[0058] According to the denoising process, it is further detected whether the obstacle tracking trajectory is of a suspected noise attribute, and at the same time, the type of the obstacle belongs to the unknown unknown type attribute.

[0059] It should be noted that the purpose of judging whether it is a suspected noise attribute is to judge whether the suspected noise belongs to noise or non-noise.

[0060] Step S130, if yes, judge whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor.

[0061] If the obstacle tracking trajectory is detected as a suspected noise attribute and the obstacle type is unknown, it is further judged whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor.

[0062] It should be noted that the common view area of the first sensor and the second sensor can be understood as the range of area in which both sensors can perceive the same obstacle. Generally, the interval of the common view area of the first sensor and the second sensor is determined by pre-calibration. The common view area may change during the driving of the vehicle.

[0063] Step S140, if yes, judge whether the ratio between the number of times of matching the second sensor and the life cycle satisfies the set condition.

[0064] If it is judged that the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor, it is further judged whether the ratio between the number of times of matching the second sensor and the life cycle is greater than or less than the set value. The set value can be set according to experience or specific scene.

[0065] Step S150, if yes, judge the sensing result of the suspected noise attribute as noise.

[0066] If it is judged that the ratio between the number of times of matching the second sensor and the life cycle satisfies the set value, the sensing result of the suspected noise attribute can be finally judged as noise.

[0067] Since the laser radar is very sensitive to water mist, dust, rain and snow, etc., it is easy to produce point cloud information and false detection. However, the millimeter wave radar is not sensitive to water mist, dust, rain and snow, etc., and it is not easy to perceive such noise, so the reverse characteristics of the two can be used to identify the false detection of such noise.

[0068] The above method is performed in a fusion module in an automatic driving system, that is, after multi-sensor fusion, the obstacle tracking trajectory is post-processed. First, before the fusion module, the first perception module gives a suspected noise attribute according to the characteristics of the noise point cloud, such as point cloud density, point cloud intensity, and point cloud distribution characteristics; then, after fusion into a track, on the basis of the suspected noise attribute, the second perception module gives the final result of whether it is a noise attribute in the detection of the noise position in the historical multi-frame detection.

[0069] Through the above method, the confidence of the point cloud recognition result is improved through the multi-frame information of the obstacle tracking trajectory and the pre-laser point cloud information.

[0070] In an embodiment of the present application, in response to the perception result of the suspected noise attribute, a multi-sensor fusion process is performed to obtain an obstacle tracking trajectory, and the fusion includes counting the life cycle of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor, the number of matches with the second sensor, and the number of matches with other sensors, including: in response to the perception result of the suspected noise attribute, counting the life cycle tracker_frames of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor in the process of multi-sensor fusion; counting the number of matches with the second sensor match_radar_frames; counting the number of matches with other sensors match_else_frames; the number of times of life cycle of each fusion process is added by 1, and the statistical result is added by 1 for each successful match.

[0071] The first sensor is a laser radar, and the second sensor is a millimeter wave radar.

[0072] Specifically, in the fusion module of the automatic driving system, a multi-sensor fusion is performed to generate an obstacle tracking trajectory track, and in the fusion process, it is necessary to:

[0073] (a) Count the life cycle of each obstacle tracking trajectory track in the common view area of the laser radar and the millimeter wave radar, set as tracker_frames, and add 1 each time the fusion process is performed.

[0074] (b) Count the number of matches with the millimeter wave radar in the common view area, set as match_radar_frames, and add 1 each time the millimeter wave radar is matched.

[0075] (c) In addition, the number of matches of other sensors in the common view area, such as Vidar and the like, is also needed, which is set as match_else_frames, and +1 is performed once per match (if there is no other 3D perception sensor, it can be ignored). If other sensors also perceive, it is not noise. Only if it is perceived in the laser radar, but not perceived by other sensors, it is probably noise.

[0076] It can be understood that Vidar, (Visual Point Cloud Forecasting) is a pre-trained model applied to automatic driving, which predicts future point clouds through historical visual input, realizes joint learning of semantics, 3D geometry and time dynamics, and thus improves the accuracy of perception, prediction and planning and the like.

[0077] In an embodiment of the present application, before the de-noising process, the process of detecting whether the obstacle tracking trajectory is of a suspected noise attribute and the obstacle type is unknown further comprises: detecting whether the second sensor is normal; if the second sensor is normal, starting the de-noising process; if the second sensor is abnormal, closing the de-noising process.

[0078] In order to avoid abnormal de-noising function caused by millimeter wave perception failure, before the de-noising process, the fusion module needs to detect whether the millimeter wave perception is normal (judge whether the millimeter wave radar is working normally), if normal, open the de-noising switch to start the de-noising process, if abnormal, close it.

[0079] In an embodiment of the present application, the process of responding to de-noising detects whether the obstacle tracking trajectory is of a suspected noise attribute and the obstacle type is unknown, comprising: according to the de-noising process, detecting whether the obstacle tracking trajectory is of a suspected noise attribute and the obstacle type is unknown; if not, judging the perception result of the suspected noise attribute as a non-noise attribute.

[0080] As shown in Figure 3 According to the de-noising process, it is detected whether the obstacle tracking trajectory track is of a suspected noise attribute and the type is unknown, if yes, the next specific judgment process is entered. If not, a non-noise noise attribute is directly given, and the judgment is ended.

[0081] In one embodiment of this application, the step of determining whether the obstacle tracking trajectory is within the common field of view of the first sensor and the second sensor includes: if the obstacle tracking trajectory is detected to be a suspected noise attribute and the obstacle type is unknown, then determining whether the obstacle tracking trajectory is within the common field of view of the first sensor and the second sensor; if the obstacle tracking trajectory is determined not to be within the common field of view of the first sensor and the second sensor, then determining whether the obstacle tracking trajectory in the previous frame was determined to be a noise attribute; if yes, then determining the perception result of the suspected noise attribute as a noise attribute; if no, then determining the perception result of the suspected noise attribute as a non-noise attribute.

[0082] like Figure 3 As shown, it determines whether the obstacle tracking trajectory is within the shared field of view of both the LiDAR and millimeter-wave radar. If it is, it proceeds to the next specific judgment process. If it is not within the shared field of view, it checks whether the obstacle tracking trajectory in the previous frame was determined to be noise (i.e., the previous judgment result / confidence level is used). If the obstacle tracking trajectory in the previous frame was determined to be noise, the noise attribute is directly given; otherwise, a non-noise attribute is given.

[0083] In one embodiment of this application, if yes, determining whether the ratio between the number of times the second sensor is matched and the lifecycle meets a set condition includes: determining whether match_radar_frames / tracker_frames is less than a set value; if match_radar_frames / tracker_frames is less than the set value, or match_radar_frames < 2, then the perception result of the suspected noise attribute is pending; if neither of the above applies, then the perception result of the suspected noise attribute is determined to be a non-noise attribute.

[0084] like Figure 3 As shown, one possible scenario is to check the ratio of the number of times the signal matches the millimeter wave (match_radar_frames) to the total number of frames (tracker_frames). If this ratio is less than a certain threshold (rate), say 0.3, it is likely a noise value, indicating a problem with the accuracy of the millimeter wave.

[0085] like Figure 3 As shown, another possible scenario is (for example, when calculating the number of times it matches the millimeter wave in the first few frames, if the number of times it matches the millimeter wave is relatively small), if the number of times it matches the millimeter wave match_radar_frames < 2, then it is necessary to enter the next specific judgment process (the fallback process), otherwise a non-noise attribute is given.

[0086] It can be understood that if the ratio of match_radar_frames / tracker_frames is less than the threshold rate, it means that the lifetime of each track in the laser radar and the millimeter wave radar common view area match_radar_frames is not perceived by the millimeter wave radar. And match_radar_frames is not 0 may be due to false matching or millimeter wave false detection.

[0087] It can be understood that if it is not a noise noise attribute, the ratio of the number of times of matching the millimeter wave match_radar_frames / the total number of frames tracker_frames should be close to 1.

[0088] In an embodiment of the present application, after the perception result of the suspected noise attribute is pending, the method further comprises: determining whether match_else_frames is 0; if yes, determining that the perception result of the suspected noise attribute is a noise attribute; if no, determining that the perception result of the suspected noise attribute is a non-noise attribute.

[0089] As shown in Figure 3 match_else_frames is 0 (the number of times of perception of other sensors), if yes, it means that the obstacle tracking track is not perceived by other sensors, and then the noise noise attribute is given, otherwise the non-noise noise attribute is given.

[0090] The above steps are used as a bottom solution to ensure the consistency and accuracy of the final result.

[0091] The present application also provides a point cloud denoising device 200, as shown in Figure 2 The structure schematic diagram of the point cloud denoising device in the embodiment of the present application is provided, and the point cloud denoising device 200 at least comprises: a multi-sensor fusion module 210, a first judgment module 220, a second judgment module 230, a third judgment module 240, and a fourth judgment module 250, wherein:

[0092] In an embodiment of the present application, the multi-sensor fusion module 210 is specifically used for: in response to the perception result of the suspected noise attribute, performing multi-sensor fusion processing to obtain the obstacle tracking track, and the fusion includes counting the lifetime of each obstacle tracking track in the common view area of the first sensor and the second sensor, the number of times of matching the second sensor, and the number of times of matching other sensors.

[0093] The perception module of the automatic driving system needs to give a suspected noise attribute according to the point cloud density, point cloud intensity, point cloud distribution characteristics and other information, and then pass the perception result and the noise attribute to the fusion module. It should be noted that the point cloud density, point cloud intensity and point cloud distribution characteristics belong to the basic attributes of the point cloud, which can be obtained by various methods in related technologies, and are not specifically limited in the present application.

[0094] It can be understood that the noise, noise attribute, etc. can be distinguished and determined by a laser point cloud recognition AI model or a laser point cloud recognition rule model.

[0095] In an embodiment of the present application, the first judgment module 220 is specifically configured to: in response to the de-noising process, detect whether the obstacle tracking trajectory is a suspected noise attribute and the obstacle type is unknown.

[0096] According to the de-noising process, it is further detected whether the obstacle tracking trajectory is a suspected noise attribute, and at the same time, the type of the obstacle belongs to the unknown type attribute.

[0097] It should be noted that the purpose of judging whether it is a suspected noise attribute is to judge whether the suspected noise belongs to noise or non-noise.

[0098] In an embodiment of the present application, the second judgment module 230 is specifically configured to: if yes, judge whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor.

[0099] If it is detected that the obstacle tracking trajectory is a suspected noise attribute and the obstacle type is unknown, it is further judged whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor.

[0100] It should be noted that the common view area of the first sensor and the second sensor can be understood as the range of the area where both sensors can perceive the same obstacle. Generally, the interval of the common view area of the first sensor and the second sensor is determined by pre-calibration. The common view area may change during the driving of the vehicle.

[0101] In an embodiment of the present application, the third judgment module 240 is specifically configured to: if yes, judge whether the ratio between the number of times of matching the second sensor and the life cycle satisfies the set condition.

[0102] If it is judged that the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor, it is further judged whether the ratio between the number of times of matching the second sensor and the life cycle is greater than or less than the set value. The set value can be set according to experience or specific scene.

[0103] In an embodiment of the present application, the fourth judging module 250 is specifically configured to: if yes, judge the sensing result of the suspected noise attribute as noise.

[0104] If the ratio between the number of times of matching the second sensor and the life cycle meets a set value, the sensing result of the suspected noise attribute can be finally judged as noise.

[0105] Since the laser radar is sensitive to water mist, dust, rain and snow, etc., it is easy to produce point cloud information and false detection. However, the millimeter wave radar is not sensitive to water mist, dust, rain and snow, etc., and it is not easy to perceive such noise. Therefore, the reverse characteristics of the two can be used to identify the false detection of such noise.

[0106] In an embodiment of the present application, the multi-sensor fusion module 210 is further configured to: in response to the sensing result of the suspected noise attribute, count the life cycle tracker_frames of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor in the process of multi-sensor fusion; count the number of times of matching the second sensor match_radar_frames; count the number of times of matching other sensors match_else_frames; count the number of times of each fusion processing life cycle by 1, and count the number of times of each successful matching by 1.

[0107] In an embodiment of the present application, the first judging module 220 is further configured to:

[0108] detect whether the second sensor is normal; if the second sensor is normal, start the noise removal process; if the second sensor is abnormal, close the noise removal process.

[0109] In an embodiment of the present application, the first judging module 220 is further configured to: according to the noise removal process, detect whether the obstacle tracking trajectory is of a suspected noise attribute and an unknown obstacle type; if not, judge the sensing result of the suspected noise attribute as a non-noise attribute.

[0110] In an embodiment of the present application, the second judging module 230 is further configured to:

[0111] If it is detected that the obstacle tracking trajectory is of a suspected noise attribute and the obstacle type is unknown, it is determined whether the obstacle tracking trajectory is within a common view area of the first sensor and the second sensor; if it is determined that the obstacle tracking trajectory is not within the common view area of the first sensor and the second sensor, it is determined whether a previous frame obstacle tracking trajectory is determined to be of a noise attribute; if yes, the perception result of the suspected noise attribute is determined to be of a noise attribute; if no, the perception result of the suspected noise attribute is determined to be of a non-noise attribute.

[0112] In an embodiment of the present application, the third determining module 240 is further configured to determine whether match_radar_frames / tracker_frames is less than a set value; if match_radar_frames / tracker_frames is less than the set value, or match_radar_frames < 2, the perception result of the suspected noise attribute is pending; if not, the perception result of the suspected noise attribute is determined to be of a non-noise attribute.

[0113] In an embodiment of the present application, the fourth determining module 250 is further configured to:

[0114] determine whether match_else_frames is 0; if yes, the perception result of the suspected noise attribute is determined to be of a noise attribute; if no, the perception result of the suspected noise attribute is determined to be of a non-noise attribute.

[0115] It can be understood that the point cloud denoising device described above can realize each step of the point cloud denoising method provided in the foregoing embodiments, and the related explanations about the point cloud denoising method are all applicable to the point cloud denoising device, which will not be repeated here.

[0116] Figure 4 is a structural schematic diagram of an electronic device of an embodiment of the present application. Please refer to Figure 4 At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a random-access memory (RAM), and can also include a non-volatile memory such as at least one disk memory. Of course, the electronic device can also include other hardware required by the business.

[0117] The processor, the network interface and the memory can be connected with each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0118] The memory is used to store programs. Specifically, the program can include program code including computer operation instructions. The memory can include an internal memory and a non-volatile memory, and provide instructions and data for the processor.

[0119] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs, and forms a point cloud denoising device at a logical level. The processor executes the program stored in the memory, and is specifically used for executing the following operations:

[0120] In response to the perception result of the suspected noise attribute, a multi-sensor fusion processing is performed to obtain an obstacle tracking trajectory, and the fusion includes counting a life cycle of each obstacle tracking trajectory in a common view area of the first sensor and the second sensor, a number of times of matching the second sensor, and a number of times of matching other sensors;

[0121] In response to the denoising process, it is detected whether the obstacle tracking trajectory is of the suspected noise attribute and the obstacle type is unknown;

[0122] If yes, it is judged whether the obstacle tracking trajectory is in the common view area of the first sensor and the second sensor;

[0123] If yes, it is judged whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set condition; and

[0124] If yes, the perception result of the suspected noise attribute is judged as noise.

[0125] The above as described in the present application Figure 1The method performed by the point cloud denoising apparatus disclosed in the embodiments shown can be applied in a processor or implemented by the processor. The processor can be an integrated circuit chip with signal processing capability. In the implementation, each step of the method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the method.

[0126] The electronic device can also perform the method performed by the point cloud denoising apparatus Figure 1 and implement the functions of the point cloud denoising apparatus in the embodiments shown. The embodiments of the present application will not be described here. Figure 1

[0127] The embodiments of the present application also propose a computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device including a plurality of applications, can cause the electronic device to perform the method performed by the point cloud denoising apparatus Figure 1 in the embodiments shown, and specifically for performing:

[0128] In response to the perception result of the suspected noise attribute, a multi-sensor fusion processing is performed to obtain an obstacle tracking trajectory, and the fusion includes counting the life cycle of each obstacle tracking trajectory in the common view area of the first sensor and the second sensor, the number of matches of the second sensor, and the number of matches of other sensors.

[0129] ​In response to the de-noising procedure, it is detected whether the obstacle tracking trajectory is of a suspected noise attribute and an obstacle type is unknown;

[0130] If yes, it is determined whether the obstacle tracking trajectory is within a common view area of the first sensor and the second sensor;

[0131] If yes, it is determined whether a ratio between the number of times of matching the second sensor and the life cycle satisfies a set condition; and

[0132] If yes, the perception result of the suspected noise attribute is determined as noise.

[0133] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0134] The present application is described in reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The flow or flows and / or blocks in a flowchart and / or block diagram represent one or more of any flow processes that can be embodied in computer program instructions. Figure 1 The flow or flows and / or blocks in a flowchart and / or block diagram represent one or more of any flow processes that can be embodied in computer program instructions.

[0135] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a particular manner, so that the instructions stored in the computer readable memory produce an article of manufacture including instruction means that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The flow or flows and / or blocks in a flowchart and / or block diagram represent one or more of any flow processes that can be embodied in computer program instructions. Figure 1 The flow or flows and / or blocks in a flowchart and / or block diagram represent one or more of any flow processes that can be embodied in computer program instructions.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operational steps are performed on the computer or other programmable data processing apparatus to generate a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 steps of a function specified in one or more blocks.

[0137] In one typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0138] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0139] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0140] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0141] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code thereon.

[0142] The above embodiments are only used to illustrate the present application, but not to limit it. Various modifications and changes can be made by those skilled in the art. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A point cloud denoising method, wherein, The noise reduction method includes: In response to the perception results of suspected noise attributes, multi-sensor fusion processing is performed to obtain obstacle tracking trajectories. The fusion includes statistically analyzing the lifetime of each obstacle tracking trajectory within the common field of view of the first and second sensors, the number of times it matches the second sensor, and the number of times it matches other sensors. In response to the denoising process, it is detected whether the obstacle tracking trajectory has suspected noise attributes and the obstacle type is unknown; If so, determine whether the obstacle tracking trajectory is within the common field of view of the first sensor and the second sensor; If so, determine whether the ratio between the number of times the second sensor is matched and the lifespan meets a set condition; and If so, the perceived result of the suspected noise attribute will be judged as noise.

2. The method as described in claim 1, wherein, In response to the perception result of suspected noise attributes, multi-sensor fusion processing is performed to obtain obstacle tracking trajectories. The fusion includes statistically analyzing the lifetime of each obstacle tracking trajectory within the shared field of view of the first and second sensors, the number of matches with the second sensor, and the number of matches with other sensors, including: In response to the perception results of suspected noise attributes, during the multi-sensor fusion process, the lifecycle tracker_frames of each obstacle tracking trajectory within the common field of view of the first and second sensors are statistically analyzed. Count the number of times the second sensor is matched (match_radar_frames); Count the number of times the data is matched against other sensors (match_else_frames); The number of times the fusion process is incremented by 1 for each lifecycle, and the statistical result is incremented by 1 for each successful match.

3. The method as described in claim 2, wherein, Before detecting whether the obstacle tracking trajectory has suspected noise attributes and the obstacle type is unknown, the denoising process further includes: Check if the second sensor is functioning properly; If the second sensor is working properly, then the noise reduction process is initiated. If the second sensor malfunctions, the noise reduction process is shut down.

4. The method as described in claim 2, wherein, In response to the denoising process, detecting whether the obstacle tracking trajectory has suspected noise attributes and the obstacle type is unknown includes: According to the denoising process, it is detected whether the obstacle tracking trajectory has suspected noise attributes and the obstacle type is unknown; If not, the perceived result of the suspected noise attribute is judged as a non-noise attribute.

5. The method as described in claim 4, wherein, If so, determining whether the obstacle tracking trajectory is within the common field of view of the first and second sensors includes: If the obstacle tracking trajectory is detected to be of suspected noise attribute and the obstacle type is unknown, then it is determined whether the obstacle tracking trajectory is within the common field of view of the first sensor and the second sensor; If it is determined that the obstacle tracking trajectory is not within the common field of view of the first sensor and the second sensor, then it is determined whether the obstacle tracking trajectory of the previous frame was determined to be noise. If so, the perception result of the suspected noise attribute is judged as a noise attribute; If not, the perceived result of the suspected noise attribute is judged as a non-noise attribute.

6. The method as described in claim 2, wherein, If so, determine whether the ratio between the number of times the second sensor is matched and the lifespan meets the set conditions, including: Check if match_radar_frames / tracker_frames is less than the set value; If it is determined that match_radar_frames / tracker_frames is less than the set value, or match_radar_frames < 2, then the perception result of the suspected noise attribute will be pending. If none of the above apply, the perceived result of the suspected noise attribute will be judged as a non-noise attribute.

7. The method of claim 6, wherein, After determining the perception result of the suspected noise attribute, the method further includes: Check if match_else_frames is 0; If so, the perception result of the suspected noise attribute is judged as a noise attribute; If not, the perceived result of the suspected noise attribute is judged as a non-noise attribute.

8. A point cloud denoising device, wherein, The device includes: A multi-sensor fusion module is used to perform multi-sensor fusion processing to obtain obstacle tracking trajectories in response to the perception results of suspected noise attributes. The fusion includes statistically analyzing the lifecycle of each obstacle tracking trajectory within the common field of view of the first and second sensors, the number of times it matches the second sensor, and the number of times it matches other sensors. The first judgment module is used to detect, in response to the noise reduction process, whether the obstacle tracking trajectory is a suspected noise attribute and the obstacle type is unknown. The second judgment module is used to determine whether the obstacle tracking trajectory is within the common field of view of the first sensor and the second sensor if the condition is met. The third judgment module is used to determine, if yes, whether the ratio between the number of times the second sensor is matched and the life cycle meets a set condition; and The fourth judgment module is used to judge the perception result of the suspected noise attribute as noise if it is true.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 7.

Images