Point cloud processing method and related apparatus

Abstract

A point cloud processing method and a related apparatus. The point cloud processing method comprises: determining noise point information in a first frame of point cloud, and one or more perceived targets in the first frame of point cloud; and on the basis of the noise point information and the one or more perceived targets, determining whether there is a first perceived target among the one or more perceived targets, wherein the first perceived target is determined on the basis of the noise point information corresponding to the first perceived target. The objectivity of point cloud quality assessment can be improved.

Description

Point cloud processing methods and related devices Technical Field

[0001] This application relates to the field of lidar technology, specifically to a point cloud processing method and related apparatus. Background Technology

[0002] With the development and advancement of technology, vehicle intelligence has gradually become a research hotspot in the automotive field. Intelligent vehicles (referred to as smart vehicles) can bring people a safer and more comfortable driving experience. For example, smart vehicles can be equipped with various sensors to collect point clouds and achieve intelligent driving based on these point clouds. The quality of the point clouds affects the reliability of intelligent driving.

[0003] Currently, in real-world vehicle environments, noise identification processing can be performed on real-time generated point cloud data. By visually observing the noise distribution in the point cloud image, a subjective assessment of the data quality—such as poor, relatively poor, good, or fairly good—can be derived. However, current point cloud quality assessment methods lack objectivity. Summary of the Invention

[0004] This application provides a point cloud processing method and related apparatus, which can improve the objectivity of point cloud quality assessment.

[0005] In a first aspect, embodiments of this application provide a point cloud processing method, which can perform steps including but not limited to the following:

[0006] Determine the noise information in the first frame point cloud and one or more perceived targets in the first frame point cloud;

[0007] Based on noise information and one or more sensing targets, determine whether a first sensing target exists among the one or more sensing targets. The first sensing target is determined based on the noise information corresponding to the first sensing target.

[0008] In this embodiment, the first frame of point cloud data can be understood as one frame of point cloud data. Point cloud data is data collected by sensors, which may include, but are not limited to, sensors capable of collecting point cloud data such as lidar, millimeter-wave radar, and ultrasonic radar.

[0009] The first perceived target can be understood as a falsely detected target that does not exist in the physical world but is generated due to noise information. The presence of a first perceived target in the first frame of the point cloud can be used to evaluate the quality of that frame. If a first perceived target is present, it indicates that the quality of the first frame is relatively poor; if no first perceived target is present, it indicates that the quality of the first frame is relatively good. The quality of the first frame of the point cloud can be objectively evaluated based on the presence and quantity of first perceived targets.

[0010] The noise information corresponding to the first perceived target can be understood as the noise information overlapping with the position of the first perceived target. Optionally, the noise information corresponding to the first perceived target is the noise information within the three-dimensional bounding box or two-dimensional bounding box of the first perceived target.

[0011] Compared to the noise information in the first frame of the point cloud that does not form a perceived target, the noise information corresponding to the first perceived target has a more severe impact on subsequent intelligent driving algorithms. The noise information corresponding to the first perceived target can also characterize the sensor's performance.

[0012] The embodiments of this application can identify a first perceptual target in a first frame point cloud. The first perceptual target is determined based on the noise information corresponding to the first perceptual target. The quality of the first frame point cloud can be evaluated by whether the first perceptual target exists in the first frame point cloud, thereby improving the objectivity of the point cloud quality evaluation.

[0013] In one possible implementation, the noise information includes the location information of at least one noise point, and the first sensing target is the sensing target whose corresponding number of noise points is greater than or equal to a first threshold among one or more sensing targets.

[0014] The first threshold can be a pre-set threshold or a threshold that can be flexibly set according to the size of the perceived target.

[0015] In this embodiment, the first sensing target can be accurately determined based on the number of noise points corresponding to one or more sensing targets. The first sensing target may have a significant impact on the decisions of subsequent intelligent driving algorithms. Accurately determining the first sensing target allows for an accurate assessment of its impact on the decisions of subsequent intelligent driving algorithms within the first frame point cloud.

[0016] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0017] Based on one or more of the noise information corresponding to the first sensing target and the position information of the first sensing target, the noise quantization index of the first sensing target is obtained; the noise quantization index of the first sensing target includes any one or more of the following: the noise type of the first sensing target, the number of noise points corresponding to the first sensing target, the lateral distance between the first sensing target and the vehicle, the longitudinal distance between the first sensing target and the vehicle, the size of the first sensing target, and the height of the first sensing target.

[0018] In this embodiment, the noise quantization index of the first perceived target can accurately assess the quality of the point cloud in the first frame.

[0019] The noise type of the first sensing target can indicate what type of noise caused the first sensing target, thus facilitating subsequent noise analysis.

[0020] The number of noise points within the first perceived target indicates the sensor's road scene performance. A higher number of noise points within the first perceived target indicates a greater number of noise points posing a perceived risk, and consequently, a lower level of road scene performance. The sensor's road scene performance reflects its ability to perceive the road environment.

[0021] The lateral and longitudinal distances between the primary sensing target and the vehicle indicate the degree of influence the primary sensing target has on the vehicle. A larger lateral distance indicates a greater distance between the primary sensing target and the vehicle, and a smaller influence on the vehicle. Similarly, a larger longitudinal distance indicates a greater distance between the primary sensing target and the vehicle, and a smaller influence on the vehicle.

[0022] The size of the first perceived target indicates its magnitude. For subsequent decisions in autonomous driving algorithms, the focus is primarily on larger perceived risk targets.

[0023] The height of the first perceived target indicates its distance from the ground. For subsequent decisions in intelligent driving algorithms, the height of the first perceived target needs to be considered, which is related to the vehicle model. The scope of attention should extend beyond the vehicle's height to include noise and perceived targets, in order to ensure driving safety.

[0024] In one possible implementation, the noise information also includes the noise type of at least one noise point;

[0025] The noise types of the first sensing target include the noise type with the highest number of noise points corresponding to the first sensing target; and / or,

[0026] The lateral distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the horizontal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or,

[0027] The longitudinal distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the longitudinal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or,

[0028] The height of the first perceived target includes: the distance between the projection point of the center of the first perceived target onto the vertical axis of the vehicle-related coordinate system and the origin of the coordinate system; and / or,

[0029] The dimensions of the first perceived target include any one or more of its length, width, and height in the vehicle-related coordinate system.

[0030] In this embodiment of the application, the noise type of the first sensing target is the noise type with the most noise points corresponding to the first sensing target. The noise type of the most common noise point in the noise information corresponding to the first sensing target can be used as the noise type of the first sensing target.

[0031] The vehicle-related coordinate system may include one of the following: the vehicle body coordinate system (the origin is the projection point of the center of the rear axle of the vehicle onto the ground, or the origin is the center of mass of the vehicle), the vehicle coordinate system, or the vehicle body coordinate system.

[0032] This application provides definitions for various noise quantification indicators of the first sensing target (noise type of the first sensing target, lateral and longitudinal distance between the first sensing target and the vehicle, height of the first sensing target, and size of the first sensing target), which can accurately determine various noise quantification indicators of the first sensing target.

[0033] In one possible implementation, the noise type includes any of the following: image noise, cluster noise, multipath, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghosting, and distance blur.

[0034] In this embodiment, noise types can be classified more finely, resulting in more refined noise classifications. The industry typically categorizes point cloud noise into several main types, including cluster noise, crosstalk, and ghosting. This embodiment defines a more complete and detailed set of noise classifications, enabling more precise noise categorization and covering cases where various noise types affect intelligent driving functions.

[0035] In one possible implementation, before determining the noise information in the first frame point cloud and one or more perceived targets in the first frame point cloud, the above point cloud processing method may also perform the following steps, including but not limited to:

[0036] Acquire the first point cloud data, which includes at least the first frame of point cloud data;

[0037] After obtaining the noise quantization index of the first sensing target based on one or more of the noise information and the position information of the first sensing target, the above point cloud processing method further includes, but is not limited to, the following steps:

[0038] Based on the noise quantification index of the first perceived target in the first point cloud data, the noise assessment result of the first point cloud data is obtained.

[0039] The noise assessment results of the first point cloud data include any one or more of the following: the number of instances corresponding to each noise type, the number of instances of the first sensing target corresponding to each noise type, the number of identity documents (IDs) of the first sensing target corresponding to each noise type, the average number of noise points of the first sensing target corresponding to each noise type, the average position of the first sensing target corresponding to each noise type, the average size of the first sensing target corresponding to each noise type, and the number of first sensing targets formed per unit mileage. Among these, consecutive frame point clouds in the first point cloud data that contain noise points of the same noise type are considered as one instance corresponding to that noise type, and consecutive frame point clouds in the first point cloud data that contain first sensing targets of the same noise type and the same ID are considered as one instance of the first sensing target corresponding to that noise type.

[0040] In this embodiment of the application, the first point cloud data may include a series of consecutive point clouds, including the first frame point cloud.

[0041] The noise quantization index of the first perceived target in the first point cloud data is the noise quantization index of the first perceived target in the point cloud of the frame containing the first perceived target in the continuous multi-frame point cloud. Among them, at least one frame of the point cloud contains the first perceived target.

[0042] The noise assessment result of the first point cloud data can be obtained by statistically analyzing the noise quantization index of the first sensing target in the frame point cloud containing the first sensing target. The noise assessment result of the first point cloud data can be used to evaluate the quality of the first point cloud data.

[0043] The number of IDs of the first perceived target corresponding to each noise type: This indicates the number of first perceived targets caused by each noise type. The larger the number, the greater the impact of that noise type on the point cloud data.

[0044] Average number of noise points in the first perceived target for each noise type: This indicates the average number of noise points in the first perceived target caused by each noise type. A higher average number of noise points indicates poorer sensor performance for that type of noise in a road scene.

[0045] The average position of the first perceived target for each noise type indicates the position of the first perceived target relative to the vehicle for each noise type. For subsequent intelligent driving algorithms, the farther the first perceived target is relative to the vehicle, the smaller its impact on the vehicle.

[0046] The average size of the first perceived target for each noise type indicates the size of the first perceived target for each noise type. For subsequent decisions in autonomous driving algorithms, larger perceived-level risk targets have a greater impact on the vehicle.

[0047] Number of first sensing targets formed per unit mileage: For example, this can indicate the number of first sensing targets formed per 10,000 kilometers. The smaller the value of this indicator, the less noise impact and the better the sensor performance. This indicator is strongly correlated with the type of failure scenario and can be used to evaluate sensor performance by combining the generalized distribution of failure scenarios and the amount of data. The type of failure scenario refers to the type of scenario that causes noise in the point cloud data.

[0048] In one possible implementation, the average number of noise points in the first sensing target corresponding to each noise type satisfies the following condition:

[0049] Ca = B1 / A1, or Ca = B1 / A2;

[0050] Where Ca is the average number of noise points of the first sensing target corresponding to each noise type in the first point cloud data, B1 is the sum of the number of noise points of the first sensing target corresponding to each noise type in the first point cloud data, A1 is the number of IDs of the first sensing target corresponding to each noise type in the first point cloud data, and A2 is the number of frames in the first point cloud data that contain the first sensing target corresponding to each noise type.

[0051] In this embodiment, a method for calculating the average number of noise points on the first sensing target corresponding to each noise type is provided, which can accurately obtain the average number of noise points on the first sensing target corresponding to each noise type. There are two calculation methods: target counting method (Ca = B1 / A1) and frame counting method (Ca = B1 / A2). The target counting method focuses on evaluating the performance of the sensing side, while the frame counting method focuses on evaluating the performance of the point cloud side.

[0052] In one possible implementation, the average position of the first sensing target corresponding to each noise type includes any one or more of the following: the average lateral distance, average longitudinal distance, and average height of the first sensing target and the vehicle corresponding to each noise type;

[0053] The average lateral distance between the first perceived target and the vehicle for each noise type satisfies the following condition:

[0054] Da = D1 / A1; and / or,

[0055] The average longitudinal distance between the first perceived target and the vehicle for each noise type satisfies the following condition:

[0056] Db = D2 / A1; and / or,

[0057] The average height of the first perceived target for each noise type satisfies the following condition:

[0058] Dc = D3 / A1;

[0059] Where Da is the average lateral distance between the first sensing target and the vehicle for each noise type, D1 is the sum of the lateral distances between the first sensing target and the vehicle for each noise type in the first point cloud data, A1 is the number of IDs of the first sensing target for each noise type in the first point cloud data, Db is the average longitudinal distance between the first sensing target and the vehicle for each noise type, D2 is the sum of the longitudinal distances between the first sensing target and the vehicle for each noise type in the first point cloud data, Dc is the average height of the first sensing target for each noise type, and D3 is the sum of the heights of the first sensing targets for each noise type in the first point cloud data.

[0060] In this embodiment, a method for calculating the average position of the first sensing target corresponding to each noise type is provided, which can accurately obtain the average position of the first sensing target corresponding to each noise type. The average position of the first sensing target corresponding to each noise type may include: the average lateral distance between the first sensing target and the vehicle, the average longitudinal distance between the first sensing target and the vehicle, and the average height of the first sensing target.

[0061] In one possible implementation, the average size of the first sensing target corresponding to each noise type includes any one or more of the following: the average length, average width, and average height of the first sensing target corresponding to each noise type in the vehicle-related coordinate system;

[0062] The average length of the first perceived target for each noise type in the vehicle-related coordinate system satisfies the following condition:

[0063] Ea = E1 / A1; and / or,

[0064] The average width of the first perceived target for each noise type in the vehicle-related coordinate system satisfies the following condition:

[0065] Eb = E2 / A1; and / or,

[0066] The average height of the first perceived target in the vehicle-related coordinate system corresponding to each noise type satisfies the following condition:

[0067] Ec = E3 / A1;

[0068] Wherein, Ea is the average length of the first sensing target corresponding to each noise type in the vehicle-related coordinate system, E1 is the sum of the lengths of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system in the first point cloud data, A1 is the number of IDs of the first sensing targets corresponding to each noise type in the first point cloud data, Eb is the average width of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system, E2 is the sum of the widths of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system in the first point cloud data, Ec is the average height of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system, and E3 is the sum of the heights of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system in the first point cloud data.

[0069] In this embodiment, a method for calculating the average size of the first sensing target corresponding to each noise type is provided, which can accurately obtain the average size of the first sensing target corresponding to each noise type. The average size of the first sensing target corresponding to each noise type may include: the average length of the first sensing target corresponding to each noise type in the vehicle-related coordinate system, the average width of the first sensing target corresponding to each noise type in the vehicle-related coordinate system, and the average height of the first sensing target corresponding to each noise type in the vehicle-related coordinate system.

[0070] In one possible implementation, the number of first sensing targets formed per unit mileage satisfies the following condition:

[0071] Fa = F1 / F2;

[0072] Where Fa is the number of first sensing targets formed per unit mileage, F1 is the number of IDs of first sensing targets formed in the first point cloud data, and F2 is the mileage corresponding to the first point cloud data.

[0073] In this embodiment of the application, a method for calculating the number of first sensing targets formed per unit mileage is provided, which can accurately obtain the number of first sensing targets formed per unit mileage.

[0074] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0075] Identify the second sensing target in the first point cloud data that triggers the alarm. The second sensing target that triggers the alarm is a second sensing target used to trigger the security system alarm. The second sensing target has the same attributes as the first sensing target.

[0076] The safety system includes one or more of the following: Emergency Braking (AEB), Lane Keeping Assist (LKA), and Adaptive Cruise Control (ACC).

[0077] In this embodiment, a second sensing target for triggering an alarm in the security system can be determined from the first point cloud data. The noise performance of the sensor can be quantitatively evaluated based on the number of second sensing targets for triggering the alarm in the first point cloud data. The second sensing target has the same attributes as the first sensing target, and the method for determining the second sensing target can be the same as that for determining the first sensing target.

[0078] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0079] Acquire the second point cloud data and determine the noise assessment result of the second point cloud data; the first point cloud data and the second point cloud data are data collected under the first condition or data collected under the second condition; the first condition is: the software versions used to collect the first point cloud data and the software versions used to collect the second point cloud data are different; the second condition is: the sensors used to collect the first point cloud data and the second point cloud data are different.

[0080] Based on the noise assessment results of the first point cloud data and the second point cloud data, a noise comparison result between the first point cloud data and the second point cloud data is obtained.

[0081] In this embodiment, the noise evaluation results of two point cloud data can be compared, and the noise comparison result of the first point cloud data and the second point cloud data can be obtained based on the noise evaluation results of the first point cloud data and the second point cloud data.

[0082] The noise comparison results have the following two application scenarios:

[0083] The first application scenario is comparing the noise results of two point cloud datasets collected by different software versions. Various noise quantification metrics can be compared between different software versions (the first and second point cloud datasets can be collected by the same sensor using different software versions) to evaluate the performance of different software versions.

[0084] The second application scenario involves comparing the noise results of two point cloud datasets collected by different sensors. This allows for the comparison of various noise quantification metrics between different hardware (where the first and second point cloud datasets were collected by different sensors), thus evaluating the performance of different sensors.

[0085] In one possible implementation, after obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the above point cloud processing method may further perform the following steps, including but not limited to:

[0086] Identify the first type of frame point cloud in the first point cloud data. The first type of frame point cloud is the frame point cloud in the first point cloud data that contains the first perceived target.

[0087] Output one or more of the following: the frame index value of the first type of frame point cloud, the position of the first perceived target in the frame point cloud corresponding to the frame index value, the noise type of the first perceived target in the frame point cloud corresponding to the frame index value, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, the first image acquired synchronously with the frame point cloud corresponding to the frame index value, and the second image acquired synchronously with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

[0088] In this embodiment, the first type of frame point cloud is the frame point cloud where a first sensing target is detected. The first type of frame point cloud can also be called a sensing-level problem frame. The first type of frame point cloud can be identified, and its relevant information can be output. Visual analysis of the first type of frame point cloud can be performed, allowing for rapid manual assessment of the severity of the sensing-level problem. A sensing-level problem may include any one or more of the following: the number of frames of the first type of frame point cloud in the first point cloud data, the proportion of the first type of frame point cloud in the first point cloud data, the position of the first sensing target in the first type of frame point cloud, the noise type of the first sensing target in the first type of frame point cloud, the number of noise points in the first type of frame point cloud, the noise type of the noise points in the first type of frame point cloud, and the position of the noise points in the first type of frame point cloud. The noise points in the frame point cloud corresponding to the frame index value and the non-noise points in the frame point cloud corresponding to the frame index value can be displayed separately. The first sensing target in the frame point cloud corresponding to the frame index value can be highlighted (e.g., marked with a striking color).

[0089] The first and second images can assist in the analysis of the reasons for the appearance of point clouds in the first type of frame.

[0090] In one possible implementation, after obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the above point cloud processing method may further perform the following steps, including but not limited to:

[0091] Identify the second type of frame point cloud in the first point cloud data. The second type of frame point cloud is the frame point cloud in the first point cloud data that contains noise.

[0092] Output one or more of the following: the frame index value of the second type of frame point cloud, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, the noise type of the noise point in the frame point cloud corresponding to the frame index value, the first image acquired synchronously with the frame point cloud corresponding to the frame index value, and the second image acquired synchronously with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

[0093] In this embodiment, the second type of frame point cloud is a frame point cloud containing detected noise; it can also be referred to as a point cloud-level problem frame. The second type of frame point cloud can be identified, and its relevant information can be output. Visual analysis of the second type of frame point cloud is possible, allowing for rapid manual assessment of the severity of point cloud-level problems. Point cloud-level problems may include any one or more of the following: the number of frames of the second type of frame point cloud in the first point cloud data, the proportion of the second type of frame point cloud in the first point cloud data, the number of noise points in the second type of frame point cloud, the noise type of the noise points in the second type of frame point cloud, and the location of the noise points in the second type of frame point cloud. The noise points in the frame point cloud corresponding to the frame index value and the non-noise points in the frame point cloud corresponding to the frame index value can be displayed separately.

[0094] The first and second images can assist in the analysis of the reasons for the appearance of the second type of frame point cloud.

[0095] Secondly, embodiments of this application provide a point cloud processing apparatus, which includes a unit for performing the method as described in any of the first aspects.

[0096] In one possible design, the device includes:

[0097] The processing unit is used to determine noise information in the first frame point cloud and one or more sensing targets in the first frame point cloud.

[0098] The processing unit is further configured to determine, based on the noise information and the one or more sensing targets, whether there is a first sensing target among the one or more sensing targets, wherein the first sensing target is determined based on the noise information corresponding to the first sensing target.

[0099] In one possible implementation, the device further includes a communication unit;

[0100] The processing unit is specifically used to acquire the first frame point cloud through the communication unit.

[0101] Regarding the processing unit and communication unit described in the second aspect and any possible implementation, the steps performed thereon can be referred to the corresponding implementations in the first aspect.

[0102] For the technical effects of the second aspect and any possible implementation, please refer to the description of the technical effects corresponding to the first aspect and the corresponding implementation.

[0103] Optionally, in the point cloud processing apparatus described in the second aspect above and any possible implementation:

[0104] In one implementation, the point cloud processing device is a point cloud processing equipment. When the point cloud processing device is a point cloud processing equipment, the communication unit can be a transceiver or an input / output interface; the processing unit can be at least one processor. Optionally, the transceiver can be a transceiver circuit. Optionally, the input / output interface can be an input / output circuit.

[0105] In another implementation, the point cloud processing device is a chip (system) or circuit used in a point cloud processing device. When the point cloud processing device is a chip (system) or circuit used in a point cloud processing device, the communication unit can be a communication interface (input / output interface), interface circuit, output circuit, input circuit, pin, or related circuit on the chip (system) or circuit; the processing unit can be at least one processor, processing circuit, or logic circuit.

[0106] Thirdly, embodiments of this application provide a point cloud processing apparatus, which includes a processor. The processor is coupled to a memory and can be used to execute instructions in the memory to implement the methods described in the first aspect and any of the possible implementations. Optionally, the point cloud processing apparatus further includes a memory. Optionally, the point cloud processing apparatus further includes a communication interface, and the processor is coupled to the communication interface.

[0107] Fourthly, embodiments of this application provide a chip, including: logic circuitry and an interface. The interface is used to receive or transmit information; the logic circuitry is used to receive or transmit information through the interface, causing the chip to execute the methods described in the first aspect and any of the possible implementations.

[0108] Fifthly, embodiments of this application provide a computer-readable storage medium for storing a computer program (also referred to as code or instructions); when the computer program is run on a computer, the methods described in the first aspect and any possible implementation are implemented.

[0109] Sixthly, embodiments of this application provide a computer program product, the computer program product comprising: a computer program (also referred to as code or instructions); and, when the computer program is run, causing a computer to perform the methods described in the first aspect and any possible implementation thereof.

[0110] In a seventh aspect, embodiments of this application provide a terminal that includes at least one point cloud processing device as described in the second aspect, or the point cloud processing device as described in the third aspect, or the chip as described in the fourth aspect.

[0111] Optionally, the terminal can be a means of transportation, such as a car, truck, aircraft, drone, slow transport vehicle, spacecraft, or ship, or any other possible means of transportation used in any possible scenario. This application embodiment does not limit this.

[0112] Optionally, the terminal is used to implement the method described in the first aspect and any possible implementation.

[0113] Furthermore, in the process of performing the methods described in any of the first to second aspects and any possible embodiments described above, the processes related to sending and / or receiving information in the above methods can be understood as the process of the processor outputting information, and / or the process of the processor receiving input information. When outputting information, the processor can output the information to a transceiver (or communication interface, or transmitting module) so that the transceiver can transmit it. After the information is output by the processor, it may need to undergo other processing before reaching the transceiver. Similarly, when the processor receives input information, the transceiver (or communication interface, or transmitting module) receives the information and inputs it to the processor. Furthermore, after the transceiver receives the information, the information may need to undergo other processing before being input to the processor.

[0114] Based on the above principles, for example, the information sent mentioned in the aforementioned method can be understood as information output by the processor. Similarly, the information received can be understood as information received by the processor from input.

[0115] Optionally, unless otherwise specified, or unless they contradict their actual function or internal logic in the relevant description, the operations of the processor, such as transmitting, sending, and receiving, can be more generally understood as processor output and receiving, input, and other operations.

[0116] Optionally, in performing the methods described in the first aspect and any possible implementation above, the processor may be a processor specifically designed to perform these methods, or it may be a processor that performs these methods by executing computer instructions stored in memory, such as a general-purpose processor. The memory may be a non-transitory memory, such as read-only memory (ROM), which may be integrated with the processor on the same chip or disposed on separate chips. This application does not limit the type of memory or the arrangement of the memory and processor.

[0117] In one possible implementation, at least one of the aforementioned memories is located outside the device.

[0118] In yet another possible implementation, at least one of the aforementioned memories is located within the device.

[0119] In another possible implementation, a portion of the memory of the at least one memory is located inside the device, while another portion is located outside the device.

[0120] In this application, the processor and memory may also be integrated into a single device, that is, the processor and memory can be integrated together. Attached Figure Description

[0121] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0122] Figure 1 is a schematic diagram of a sensor on a vehicle provided in an embodiment of this application;

[0123] Figure 2 is a schematic diagram of noise point cloud caused by vehicle exhaust in a road driving scenario.

[0124] Figure 3 is a schematic diagram of the technical framework of an intelligent driving system provided in an embodiment of this application;

[0125] Figure 4 is a flowchart illustrating a point cloud processing method provided in an embodiment of this application;

[0126] Figure 5 is a schematic diagram of a vehicle coordinate system provided in an embodiment of this application;

[0127] Figure 6 is a schematic flowchart of a point cloud processing method provided in an embodiment of this application;

[0128] Figure 7 is a schematic diagram of the positional relationship of a sensor installed on a vehicle according to an embodiment of this application;

[0129] Figure 8 is a schematic diagram of the positional relationship between a sensing target and a point cloud provided in an embodiment of this application;

[0130] Figure 9 is a schematic diagram of the positional relationship between a perceived target, a real point cloud, and a noisy point cloud provided in an embodiment of this application.

[0131] Figure 10 is a schematic diagram of a visual interface provided in an embodiment of this application;

[0132] Figure 11 is a schematic diagram of another visual interface provided in an embodiment of this application;

[0133] Figure 12 is a schematic diagram of a point cloud processing device provided in an embodiment of this application;

[0134] Figure 13 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0135] Figure 14 is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0136] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0137] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0138] The term "embodiment" as used herein means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that, unless otherwise specified or logically conflicting, the terminology and / or descriptions between the various embodiments of this application are consistent and can be mutually referenced, and technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0139] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0140] It should be noted that, in this application, "instruction" can include direct instruction, indirect instruction, explicit instruction, and implicit instruction. When describing a certain instruction information for the purpose of instructing A, it can be understood that the instruction information carries A, directly instructs A, or indirectly instructs A.

[0141] In this application, the information indicated by the instruction information is called the information to be instructed. In specific implementations, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is a correlation between the other information and the information to be instructed. It can also indicate only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction of specific information can be achieved by using a pre-agreed (e.g., protocol-defined) arrangement of various information, thereby reducing instruction overhead to some extent. The information to be instructed can be sent as a whole or divided into multiple sub-information units, and the sending period and / or timing of these sub-information units can be the same or different. This application does not limit the specific sending method. The sending period and / or timing of these sub-information units can be predefined, for example, according to a protocol, or configured by the transmitting device by sending configuration information to the receiving device.

[0142] It should be noted that in this application, "send" can be understood as "output" and "receive" can be understood as "input". "Send information to A", where "to A" simply indicates the direction of information transmission, and A is the destination, does not limit "send information to A" to a direct transmission over the air interface. "Send information to A" includes sending information directly to A, as well as sending information indirectly to A through a transmitter. Therefore, "send information to A" can also be understood as "outputting information destined for A". Similarly, "receive information from A" indicates that the source of the information is A, including receiving information directly from A, as well as receiving information indirectly from A through a receiver. Therefore, "receive information from A" can also be understood as "inputting information from A".

[0143] This application provides a point cloud processing method and related apparatus, which are applied in the field of vehicle technology, and specifically in the field of lidar technology.

[0144] Currently, vehicle intelligence has gradually become a research hotspot in the automotive field. Intelligent vehicles can bring people a safer and more comfortable driving experience. For example, various sensors can be installed on vehicles to acquire information about their surrounding environment and enable intelligent driving based on this information. Sensors can include cameras, LiDAR, millimeter-wave radar, ultrasonic sensors, etc.

[0145] Please refer to Figure 1, which is a schematic diagram of a sensor on a vehicle according to an embodiment of this application. As shown in Figure 1, various sensors, such as cameras, lidar, millimeter-wave radar, and ultrasonic sensors, can be installed on a vehicle to acquire surrounding environmental information and achieve assisted driving or autonomous driving based on this information. The sector-shaped areas in Figure 1 represent the detection range of the corresponding sensors. It should be noted that the number and position of the various sensors in Figure 1 can be adjusted as needed.

[0146] LiDAR uses laser as the detection medium. It uses a laser emitting component to emit a laser beam within a certain field of view, and a receiving component to receive the reflected light within the range. Using the known and acquired information about the emitted and reflected light, it directly calculates or derives the information of the reflection point (distance, angle, reflection intensity, etc.), and finally outputs point cloud data of the environment.

[0147] Compared to manual driving, intelligent driving relies significantly more heavily on environmental perception. Point cloud data output from sensors such as LiDAR, millimeter-wave radar, and ultrasonic sensors, with their high precision and density, have become a crucial information source for intelligent driving algorithms. The quality of the point cloud data directly impacts the reliability of intelligent driving. Simultaneously, the issue of point cloud noise during road driving is becoming increasingly prominent. Figure 2 illustrates a schematic diagram of noise point clouds caused by vehicle exhaust in a road driving scenario. As shown in Figure 2, the left side is an image captured by a camera containing the preceding vehicle, and the right side is the corresponding frame point cloud image. The rectangle represents the noise point cloud caused by the preceding vehicle's exhaust.

[0148] Currently, most point cloud testing for LiDAR in the industry focuses on objective specifications of the point cloud itself, such as ranging accuracy, ranging precision, and angular resolution, which are not tested under real-world automotive environments. Testing for point cloud noise quality in real-world automotive environments currently relies solely on subjective evaluation through human observation of a limited number of constructed scenarios. This subjective assessment, categorized as poor, relatively poor, good, or fairly good based on visual observation of the point cloud data, is not only costly but also difficult to quantify and scale, lacking objectivity and persuasiveness.

[0149] Please refer to Figure 3, which is a schematic diagram of the technical framework of an intelligent driving system provided in an embodiment of this application. As shown in Figure 3, the intelligent driving system may include an environmental perception module, a multi-sensor data fusion module, a decision planning module, and a control execution module. The environmental perception module can collect information about the external environment through sensing devices and output the data to the multi-sensor data fusion module. Sensing devices may include one or more of the following: lidar, millimeter-wave radar, camera, inertial measurement unit (IMU), global positioning system (GPS), BeiDou navigation satellite system (BDS), ultrasonic sensors, etc. Information about the external environment may include one or more of the following: road information, traffic information, obstacle information, location information, weather information, vehicle information, etc. The multi-sensor data fusion module can fuse data output from multiple sensors of the sensing devices and output it to the decision planning module. The decision planning module can execute one or more of the following decisions: lane keeping or lane changing, constant speed or variable speed adjustment, path planning, trajectory planning. The control execution module can execute one or more of the following controls based on the decisions made by the decision planning module: speed control, steering control.

[0150] Figure 3 illustrates a possible technical framework for an intelligent driving system. LiDAR, millimeter-wave radar, and ultrasonic sensors collect information about the external environment and output point cloud data. This point cloud data is then fused and transmitted to subsequent decision-making and planning modules and control execution modules to achieve intelligent driving. During this process, if noise appears in the point cloud data due to road scene factors, it will affect the reliability of the environmental perception module, thus affecting the vehicle's ability to perform unexpected driving actions, ultimately rendering the entire intelligent driving system unusable.

[0151] The point cloud processing method provided in this application embodiment can automatically and objectively evaluate the noise information in the identified point cloud, avoiding errors caused by subjective evaluation by the human eye. It can effectively improve efficiency, reduce costs, and is suitable for large-scale promotion and implementation.

[0152] Please refer to Figure 4, which is a flowchart illustrating a point cloud processing method provided in an embodiment of this application. As shown in Figure 4, this point cloud processing method is applied in the field of vehicle technology. The point cloud processing method includes, but is not limited to, the following steps:

[0153] 401, The point cloud processing device determines the noise information in the first frame point cloud and one or more sensing targets in the first frame point cloud;

[0154] 402, The point cloud processing device determines whether a first sensing target exists among the one or more sensing targets based on noise information and one or more sensing targets, and the first sensing target is determined based on the noise information corresponding to the first sensing target.

[0155] It is understood that the point cloud processing device in this application embodiment can be a device equipped with a processor / chip that can execute computer execution instructions, or it can be a processor / chip that can execute computer execution instructions. Optionally, the point cloud processing device can be an electronic device, or it can be a processor / chip within an electronic device, used to execute the point cloud processing method in this application embodiment to improve the objectivity of point cloud quality assessment. The electronic device can be an intelligent driving vehicle or terminal device, or it can be a device that communicates with an intelligent driving vehicle or terminal device, such as a computer, mobile phone, tablet computer, etc.

[0156] Optionally, the point cloud processing device and point cloud processing method in the embodiments of this application can be applied to, but are not limited to, vehicle systems. The vehicle equipped with the vehicle system is an intelligent driving vehicle and can be replaced by a terminal device. The terminal device can be, but is not limited to, vehicles such as commercial vehicles, passenger cars, trains, industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), robots, etc. The embodiments of this application do not specifically limit this.

[0157] The first frame of point cloud in this embodiment can be understood as one frame of point cloud data. Point cloud data can be data collected by sensors, including but not limited to LiDAR, millimeter-wave radar, ultrasonic radar, and other sensors capable of collecting point cloud data. The point cloud processing device can acquire the first frame of point cloud from the point cloud data collected by the sensors.

[0158] The noise information in the first frame of the point cloud may include any one or more of the following: the location information of the noise points in the first frame of the point cloud, the noise type of the noise points, and the number of noise points. The point cloud processing device can determine the noise information in the first frame of the point cloud using a noise recognition algorithm. This noise recognition algorithm may include: supervised learning-based ensemble learning algorithms, neural network algorithms, etc. The noise recognition algorithm can be implemented using a noise recognition model.

[0159] A point cloud processing device can determine one or more perceived targets in a first frame of point cloud using a perception and recognition algorithm. This algorithm can include ground segmentation, point cloud clustering, target detection and tracking, lane line and road edge fitting, and other similar algorithms. The perception and recognition algorithm can be implemented using a perception and recognition model.

[0160] The first perceived target in this embodiment can be understood as a false target that does not exist in the physical world and is generated due to noise information. It can also be understood as a spurious target. The first perceived target in the first frame point cloud can be used to evaluate the quality of the first frame point cloud. If the first perceived target exists in the first frame point cloud, it indicates that the quality of the first frame point cloud is relatively poor; if the first perceived target does not exist in the first frame point cloud, it indicates that the quality of the first frame point cloud is relatively good. The quality of the first frame point cloud can be objectively evaluated based on whether the first perceived target exists and the number of existing first perceived targets. Those skilled in the art will readily understand that if spurious targets that do not actually exist appear in the point cloud, then the quality of the point cloud is questionable.

[0161] The noise information corresponding to the first perceived target can be understood as noise information overlapping with the position of the first perceived target. Optionally, the noise information corresponding to the first perceived target is the noise information within the three-dimensional or two-dimensional bounding box of the first perceived target. The first perceived target is determined based on the number of noise points in the noise information corresponding to the first perceived target. For example, the number of noise points in the noise information corresponding to the first perceived target is greater than or equal to 1.

[0162] The first perceived target can be a three-dimensional target or a two-dimensional target. When the first perceived target is a three-dimensional target, the noise information corresponding to the first perceived target is the noise information within the three-dimensional bounding box of the first perceived target; when the first perceived target is a two-dimensional target, the noise information corresponding to the first perceived target is the noise information within the two-dimensional bounding box of the first perceived target.

[0163] Compared to the noise information in the first frame of the point cloud that does not form a perceived target, the noise information corresponding to the first perceived target has a more severe impact on subsequent intelligent driving algorithms. The noise information corresponding to the first perceived target can also characterize the sensor's performance.

[0164] Optionally, the point cloud processing method in this application embodiment can be applied to scenarios where sensors output point cloud data in real time, such as road driving scenarios. This application embodiment does not limit this.

[0165] The embodiments of this application can identify a first perceptual target in a first frame point cloud. The first perceptual target is determined based on the noise information corresponding to the first perceptual target. The quality of the first frame point cloud can be evaluated through the first perceptual target, thereby improving the objectivity of the point cloud quality evaluation.

[0166] In one possible implementation, the noise information includes the location information of at least one noise point, and the first sensing target is the sensing target whose corresponding number of noise points is greater than or equal to a first threshold among one or more sensing targets.

[0167] Optionally, the first threshold can be a pre-set threshold. For example, the first threshold can be set to an integer greater than or equal to 2. In this embodiment, the same threshold is used to determine whether all perceived targets are the first perceived target, which facilitates software settings and improves the speed of judgment. This embodiment is applicable to scenarios where the size difference between targets is small.

[0168] Optionally, the first threshold can be a threshold that is flexibly set according to the size of the sensing target. For example, the first threshold can be positively correlated with the size of the sensing target; the larger the size of the sensing target, the larger the first threshold can be set. In this embodiment, different thresholds can be used to determine whether a sensing target of different sizes is the first sensing target. By determining whether a sensing target is the first sensing target based on the noise density within the sensing target, the first sensing target can be accurately identified. This embodiment is applicable to scenarios where the size of the target varies significantly.

[0169] The number of noise points corresponding to the first perceived target can be understood as the number of noise points within the first perceived target. The number of noise points within the first perceived target can be the number of noise points located within the first perceived target. Taking a 3D target as an example, the number of noise points within the first perceived target can be determined based on the position of the 3D bounding box of the first perceived target and the noise points in the same coordinate system. For example, the number of noise points within the 3D bounding box of the first perceived target can be determined based on the coordinate range of the 3D bounding box of the first perceived target in the vehicle coordinate system and the coordinates of the noise points in the first frame point cloud in the vehicle coordinate system.

[0170] In this embodiment, the first sensing target can be accurately determined based on the number of noise points corresponding to one or more sensing targets. The first sensing target may have a significant impact on the decisions of subsequent intelligent driving algorithms. Accurately determining the first sensing target allows for an accurate assessment of its impact on the decisions of subsequent intelligent driving algorithms within the first frame point cloud.

[0171] Among them, the intelligent driving algorithm can be understood as the intelligent driving algorithm used by the decision planning module and / or control execution module shown in Figure 3.

[0172] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0173] The point cloud processing device obtains a noise quantization index of the first sensing target based on one or more of the noise information corresponding to the first sensing target and the position information of the first sensing target; the noise quantization index of the first sensing target includes any one or more of the following: the noise type of the first sensing target, the number of noise points corresponding to the first sensing target, the lateral distance between the first sensing target and the vehicle, the longitudinal distance between the first sensing target and the vehicle, the size of the first sensing target, and the height of the first sensing target.

[0174] In this embodiment, the noise quantization index of the first perceived target can accurately assess the quality of the point cloud in the first frame.

[0175] The noise type of the first sensing target can indicate what type of noise caused the first sensing target, thus facilitating subsequent noise analysis.

[0176] The number of noise points corresponding to the first sensing target indicates the sensor's road scene performance. The more noise points within the first sensing target, the more noise points pose a sensing-level risk, and the worse the sensor's road scene performance. The sensor's road scene performance refers to its ability to perceive the road environment. Noise points posing a sensing-level risk can be understood as noise points located within the first sensing target.

[0177] The lateral and longitudinal distances between the first perceived target and the vehicle indicate the degree of influence the first perceived target has on the vehicle. A larger lateral distance means the first perceived target is farther from the vehicle laterally, and the smaller its impact. For example, referring to the lane width in national standards, attention can be paid to noise and perceived targets within a lateral distance of 5.6 meters. A larger longitudinal distance means the first perceived target is farther from the vehicle longitudinally, and the smaller its impact. For example, the scope of longitudinal distance attention depends on the vehicle's speed; for subsequent intelligent driving algorithm decisions, the faster the vehicle's speed, the more distant the perceived targets need to be.

[0178] The size of the first perceived target indicates its magnitude. For subsequent decisions in autonomous driving algorithms, the focus is primarily on larger perceived risk targets.

[0179] The height of the first perceived target indicates its distance from the ground. For subsequent decisions in intelligent driving algorithms, the height of the first perceived target needs to be considered, which is related to the vehicle model. The scope of attention should extend beyond the vehicle's height to include noise and perceived targets, in order to ensure driving safety.

[0180] In this embodiment, the noise quantification index of the first perceived target can provide a quantitative analysis of the point cloud quality of the first frame point cloud, which can improve the objectivity of the point cloud quality assessment. It can also clarify whether the noise affects the decision-making of the subsequent intelligent driving algorithm based on the noise quantification index of the first perceived target, thus improving reliability.

[0181] In one possible implementation, the noise information also includes the noise type of at least one noise point;

[0182] The noise types of the first sensing target include the noise type with the highest number of noise points corresponding to the first sensing target; and / or,

[0183] The lateral distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the horizontal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or,

[0184] The longitudinal distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the longitudinal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or,

[0185] The height of the first perceived target includes: the distance between the projection point of the center of the first perceived target onto the vertical axis of the vehicle-related coordinate system and the origin of the coordinate system; and / or,

[0186] The dimensions of the first perceived target include any one or more of its length, width, and height in the vehicle-related coordinate system.

[0187] In this embodiment, the noise type of the first sensing target is the noise type with the most noise points corresponding to the first sensing target. The noise type of the most frequent noise point in the noise information corresponding to the first sensing target can be used as the noise type of the first sensing target. For example, if there are three noise points in the first sensing target, two of which have the noise type of spike crosstalk and the other has the noise type of cluster noise, then the mode is 2, and the noise type of the most frequent noise point is spike crosstalk. Therefore, the noise type of the first sensing target is spike crosstalk.

[0188] The center of the first perceived target can be understood as the volume center or surface center of the first perceived target. When the first perceived target is a three-dimensional target, the center of the first perceived target is the volume center of the first perceived target; when the first perceived target is a two-dimensional target, the center of the first perceived target is the surface center of the first perceived target.

[0189] The vehicle-related coordinate system may include one of the following: a vehicle body coordinate system (with the origin being the projection of the center of the rear axle onto the ground, or the origin being the vehicle's center of mass), a vehicle coordinate system, or a body coordinate system. Please refer to Figure 5, which is a schematic diagram of a vehicle body coordinate system provided in an embodiment of this application. As shown in Figure 5, the origin of the vehicle body coordinate system is the projection of the center of the rear axle onto the ground. The positive direction of the vertical axis (X-axis) is the forward direction of the vehicle during forward movement. The positive directions of the horizontal axis (Y-axis), the vertical axis (Z-axis), and the vertical axis (X-axis) form a right-handed coordinate system. The vehicle body coordinate system defines the translation in the X, Y, and Z directions, as well as the roll angle (rotation around the X-axis), the pitch angle (rotation around the Y-axis), and the yaw angle (rotation around the Z-axis).

[0190] For example, if the first perceived target is a three-dimensional target, and the vehicle-related coordinate system is the vehicle body coordinate system shown in Figure 5, and the coordinates of the volume center of the three-dimensional bounding box of the first perceived target in the vehicle body coordinate system are (x1, y1, z1), then the distance between the projection point of the center of the first perceived target on the horizontal axis (Y-axis) of the vehicle body coordinate system and the origin can be understood as: the absolute value of the horizontal axis of the coordinate system of the volume center of the three-dimensional bounding box of the first perceived target (the absolute value of y1). The distance between the projection point of the center of the first perceived target on the vertical axis (X-axis) of the vehicle body coordinate system and the origin can be understood as: the absolute value of the vertical axis of the coordinate system of the volume center of the three-dimensional bounding box of the first perceived target (the absolute value of x1). The distance between the projection point of the center of the first perceived target on the vertical axis (Z-axis) of the vehicle body coordinate system and the origin can be understood as: the absolute value of the vertical axis of the coordinate system of the volume center of the three-dimensional bounding box of the first perceived target (the absolute value of z1).

[0191] The length, width, and height of the first perceived target in the vehicle coordinate system can be understood as the length, width, and height of the cuboid corresponding to the 3D bounding box of the first perceived target in the vehicle coordinate system. The length, width, and height directions of the cuboid corresponding to the 3D bounding box of the first perceived target are the Y-axis, X-axis, and Z-axis directions of the vehicle coordinate system, respectively.

[0192] This application provides definitions for various noise quantification indicators of the first sensing target (noise type of the first sensing target, lateral and longitudinal distance between the first sensing target and the vehicle, height of the first sensing target, and size of the first sensing target), which can accurately determine various noise quantification indicators of the first sensing target.

[0193] In one possible implementation, the noise type includes any of the following: image noise, cluster noise, multipath, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghosting, and distance blur.

[0194] The noise type in this application embodiment can be the type of noise in the point cloud data output by the lidar.

[0195] Please refer to Table 1, which is a comparison table of various failure scenarios and noise types provided in this application.

[0196] Table 1

[0197] As shown in Table 1, the point cloud data output by the LiDAR is easily affected by the failure scenario, resulting in corresponding noise types. Table 1 defines the failure scenarios and their corresponding noise types. In failure scenarios caused by smooth or wet ground, the corresponding noise type is mirror noise; in failure scenarios caused by exhaust fumes, water splashes, dust, or smoke, the corresponding noise type is patch noise; in failure scenarios caused by multipath propagation, the corresponding noise type is multipath propagation; in failure scenarios caused by adhesion, the corresponding noise type is adhesion; in failure scenarios caused by road signs, the corresponding noise type is road sign crosstalk; in failure scenarios caused by vehicles, the corresponding noise type is vehicle crosstalk; in failure scenarios caused by road studs, the corresponding noise type is road stud crosstalk; in failure scenarios caused by wheel chocks, the corresponding noise type is wheel chock crosstalk; in failure scenarios caused by ghosting, the corresponding noise type is ghosting; and in failure scenarios caused by highly reflective objects, the corresponding noise type is distance blurring.

[0198] 1. Smooth or wet surfaces – mirror image. On smooth or wet surfaces, the surface has a high reflectivity, causing the laser to be reflected twice to other objects before being received by the lidar. The resulting noise is called a mirror image.

[0199] 2. Exhaust gas / water splash / dust / smoke – cluster noise. Cluster noise refers to the clusters of noise formed when suspended solid or liquid particles in the air reflect laser light due to exhaust gas, water splash, dust, or smoke.

[0200] 3. Multipath – Multipath. The presence of highly reflective objects such as car windows and spherical mirrors in the road causes laser light to be reflected twice to other objects before being received by the lidar. The resulting noise is classified as multipath.

[0201] 4. Adhesion – Adhesion. Typical scenarios include objects such as fences, where the emitted light spot hits different objects, creating adhesion noise.

[0202] 5. Road signs / vehicles / road studs / wheel chocks – Crosstalk noise. High-reflectivity objects in various typical scenarios reflect laser light, resulting in a large number of laser photons returning. This can easily crosstalk to adjacent single-photon avalanche diodes (SPADs), generating crosstalk noise. This is particularly noticeable when the viewport is dirty. Crosstalk noise can be further categorized according to scene type: road sign crosstalk, vehicle crosstalk, road stud crosstalk, and wheel chock crosstalk noise.

[0203] 6. Ghost Images – Due to the structural design and viewing window characteristics of lidar, highly reflective objects appearing in specific areas will create noise points along a certain optical path, forming what is known as ghost images.

[0204] 7. Highly reflective objects – distance blurring. Highly reflective objects beyond a certain distance can cause confusion within the time window of lasers emitted at different times, resulting in noise of the type called distance blurring.

[0205] In this embodiment, noise types can be classified more finely, resulting in more refined noise classifications. The industry typically categorizes point cloud noise into several main types, including cluster noise, crosstalk, and ghosting. This embodiment defines a more complete and detailed set of noise classifications, enabling more precise noise categorization and covering cases where various noise types affect intelligent driving functions.

[0206] In one possible implementation, before determining the noise information in the first frame point cloud and one or more perceived targets in the first frame point cloud, the above point cloud processing method may also perform the following steps, including but not limited to:

[0207] The point cloud processing device acquires first point cloud data, which includes at least a first frame of point cloud.

[0208] After the point cloud processing device obtains the noise quantization index of the first sensing target based on one or more of the noise information corresponding to the first sensing target and the position information of the first sensing target, the above point cloud processing method further includes, but is not limited to, the following steps:

[0209] The point cloud processing device obtains the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data.

[0210] The noise assessment results of the first point cloud data include any one or more of the following: the number of instances corresponding to each noise type, the number of instances with a first sensing target corresponding to each noise type, the number of IDs of the first sensing target corresponding to each noise type, the average number of noise points of the first sensing target corresponding to each noise type, the average position of the first sensing target corresponding to each noise type, the average size of the first sensing target corresponding to each noise type, and the number of first sensing targets formed per unit mileage. Among these, consecutive frame point clouds in the first point cloud data that have noise points of the same noise type are considered as one instance corresponding to that noise type, and consecutive frame point clouds in the first point cloud data that have first sensing targets of the same noise type and the same ID are considered as one instance with a first sensing target corresponding to that noise type.

[0211] Each noise type can be any one of the following: image noise, cluster noise, multipath noise, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghost image, and distance ambiguity.

[0212] In this embodiment, the first point cloud data may include multiple consecutive point cloud frames, including the first point cloud frame. The point cloud processing device can acquire the first point cloud data output by the sensor.

[0213] The noise quantization index of the first perceived target in the first point cloud data is the noise quantization index of the first perceived target in the point cloud of the consecutive multi-frame point cloud where the first perceived target exists. Specifically, at least one frame of the consecutive multi-frame point cloud contains the first perceived target. For example, if the first point cloud data includes 50 consecutive frames of point clouds, and if the first perceived target exists in frames 1 to 3, frames 11 to 15, frames 33 to 35, and frames 45 to 49, then the noise quantization index of the first perceived target in frames 1 to 3, frames 11 to 15, frames 33 to 35, and frames 45 to 49 can be statistically analyzed.

[0214] The noise assessment result of the first point cloud data can be obtained by statistically analyzing the noise quantization index of the first sensing target in the frame point cloud containing the first sensing target. The noise assessment result of the first point cloud data can be used to evaluate the quality of the first point cloud data.

[0215] The number of IDs of the first perceived target corresponding to each noise type: This indicates the number of first perceived targets caused by each noise type. A larger number indicates a greater impact of that noise type on the point cloud data. Across multiple point cloud frames, the counts are merged based on the IDs of the first perceived targets; identical IDs are counted only once. For example, if the first point cloud data includes 50 consecutive frames, the IDs of the first perceived targets in frames 1 to 3 are all ID1, and the noise type is clutter; the IDs of the first perceived targets in frames 11 to 15 are ID2, and the noise type is spike crosstalk; the IDs of the first perceived targets in frames 12 to 15 are ID3, and the noise type is clutter; the IDs of the first perceived targets in frames 33 to 35 are ID4, and the noise type corresponding to ID4 is mirror noise; the IDs of the first perceived targets in frames 45 to 49 are all ID5, and the noise type is clutter. The number of IDs for the first sensing target corresponding to the cluster noise is 3, the number of IDs for the first sensing target corresponding to the spike crosstalk is 1, and the number of IDs for the first sensing target corresponding to the mirror image is 1. It should be noted that within the same frame of the point cloud, there may be no first sensing target, or there may be one or more first sensing targets. As in the example above, the 4th frame of the point cloud does not have a first sensing target, the 1st frame of the point cloud has one first sensing target (ID1), and the 12th frame of the point cloud has two first sensing targets (ID2 and ID3).

[0216] Average number of noise points in the first perceived target for each noise type: This indicates the average number of noise points in the first perceived target caused by each noise type. A higher average number of noise points indicates poorer sensor performance for that type of noise in a road scene.

[0217] The average position of the first perceived target for each noise type indicates the position of the first perceived target relative to the vehicle for each noise type. For subsequent intelligent driving algorithms, the farther the first perceived target is relative to the vehicle, the smaller its impact on the vehicle.

[0218] The average size of the first perceived target for each noise type indicates the size of the first perceived target for each noise type. For subsequent decisions in autonomous driving algorithms, larger perceived-level risk targets have a greater impact on the vehicle.

[0219] Number of first-sensory targets formed per unit mileage: For example, this can indicate the number of first-sensory targets formed per 10,000 kilometers. The smaller the value of this indicator, the less noise impact and the better the sensor performance. This indicator is strongly correlated with the failure scenario type and can be used to evaluate sensor performance by combining the generalized distribution of failure scenarios and the amount of data. The failure scenario type refers to the type of scenario that causes noise in the point cloud data. Failure scenario types can include the following: failure scenarios caused by smooth or wet ground, failure scenarios caused by exhaust fumes, water splashes, dust, or smoke, failure scenarios caused by multipath propagation, failure scenarios caused by adhesion, failure scenarios caused by road signs, failure scenarios caused by vehicles, failure scenarios caused by road studs, failure scenarios caused by wheel chocks, failure scenarios caused by ghosting, and failure scenarios caused by highly reflective objects.

[0220] In one possible implementation, the average number of noise points in the first sensing target corresponding to each noise type satisfies the following condition:

[0221] Ca = B1 / A1, or Ca = B1 / A2;

[0222] Where Ca is the average number of noise points of the first sensing target corresponding to a certain type of noise in the first point cloud data, B1 is the sum of the number of noise points of the first sensing target corresponding to the same type of noise in the first point cloud data, A1 is the number of IDs of the first sensing target corresponding to the same type of noise in the first point cloud data, and A2 is the number of frames in the first point cloud data that contain the first sensing target corresponding to the same type of noise.

[0223] One of the noise types can be any one of the following: image noise, cluster noise, multipath noise, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghost image, and distance ambiguity.

[0224] This application provides a method for calculating the average number of noise points on the first sensing target corresponding to each noise type, which can accurately obtain the average number of noise points on the first sensing target corresponding to each noise type. There are two calculation methods: target counting (Ca = B1 / A1) and frame counting (Ca = B1 / A2). The target counting method focuses on evaluating the performance of the sensing side, while the frame counting method focuses on evaluating the performance of the point cloud side. The performance of the sensing side can include the performance of the sensing and recognition model. The performance of the point cloud side can include the performance of the point cloud data output by the sensor.

[0225] As in the example above, the number of IDs (such as ID1, ID3, and ID5) of the first perceived target corresponding to the cluster noise is 3. If the number of noise points for ID1 is 5, the number of noise points for ID3 is 6, and the number of noise points for ID5 is 7, according to the target counting method, the average number of noise points of the first perceived target corresponding to the cluster noise is (5+6+7) / 3 = 6; according to the frame counting method, the average number of noise points of the first perceived target corresponding to the cluster noise is (5+6+7) / (3+4+5) = 1.5.

[0226] In one possible implementation, the average position of the first sensing target corresponding to each noise type includes any one or more of the following: the average lateral distance, average longitudinal distance, and average height of the first sensing target and the vehicle corresponding to each noise type;

[0227] The average lateral distance between the first perceived target and the vehicle for each noise type satisfies the following condition:

[0228] Da = D1 / A1; and / or,

[0229] The average longitudinal distance between the first perceived target and the vehicle for each noise type satisfies the following condition:

[0230] Db = D2 / A1; and / or,

[0231] The average height of the first perceived target for each noise type satisfies the following condition:

[0232] Dc = D3 / A1;

[0233] Where Da is the average lateral distance between the first sensing target and the vehicle corresponding to a certain noise type, D1 is the sum of the lateral distances between the first sensing target and the vehicle corresponding to that noise type in the first point cloud data, A1 is the number of IDs of the first sensing target corresponding to that noise type in the first point cloud data, Db is the average longitudinal distance between the first sensing target and the vehicle corresponding to that noise type, D2 is the sum of the longitudinal distances between the first sensing target and the vehicle corresponding to that noise type in the first point cloud data, Dc is the average height of the first sensing target corresponding to that noise type, and D3 is the sum of the heights of the first sensing targets corresponding to that noise type in the first point cloud data.

[0234] One of the noise types can be any one of the following: image noise, cluster noise, multipath noise, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghost image, and distance ambiguity.

[0235] The lateral distance between the first perceived target and the vehicle can be understood as the coordinates of the center of the target's 3D bounding box within the vehicle's coordinate system (where the origin is the projection of the vehicle's rear axle center onto the ground) and the lateral distance between the target and the vehicle. The longitudinal distance between the first perceived target and the vehicle can be understood as the longitudinal distance between the center of the target's 3D bounding box and the vehicle. The height of the first perceived target can be understood as the distance between the center of the target's 3D bounding box and the ground.

[0236] As in the example above, if the number of IDs (such as ID1, ID3, and ID5) of the first sensing target corresponding to the cluster noise is 3, and the lateral distance between ID1 and the vehicle is y11, the lateral distance between ID3 and the vehicle is y13, and the lateral distance between ID5 and the vehicle is y15, then the average lateral distance between the first sensing target corresponding to the cluster noise and the vehicle is: (y11 + y13 + y15) / 3. If the longitudinal distance between ID1 and the vehicle is x11, the longitudinal distance between ID3 and the vehicle is x13, and the longitudinal distance between ID5 and the vehicle is x15, then the average longitudinal distance between the first sensing target corresponding to the cluster noise and the vehicle is: (x11 + x13 + x15) / 3. If the height of ID1 is z11, the height of ID3 is z13, and the height of ID5 is z15, then the average height of the first sensing target corresponding to the cluster noise is: (z11 + z13 + z15) / 3.

[0237] In this embodiment of the application, a method for calculating the average position of the first sensing target corresponding to each noise type is provided, which can accurately obtain the average position of the first sensing target corresponding to each noise type.

[0238] In one possible implementation, the average size of the first sensing target corresponding to each noise type includes any one or more of the following: the average length, average width, and average height of the first sensing target corresponding to each noise type in the vehicle-related coordinate system;

[0239] The average length of the first perceived target for each noise type in the vehicle-related coordinate system satisfies the following condition:

[0240] Ea = E1 / A1; and / or,

[0241] The average width of the first perceived target for each noise type in the vehicle-related coordinate system satisfies the following condition:

[0242] Eb = E2 / A1; and / or,

[0243] The average height of the first perceived target in the vehicle-related coordinate system corresponding to each noise type satisfies the following condition:

[0244] Ec = E3 / A1;

[0245] Wherein, Ea is the average length of the first perceived target corresponding to a certain noise type in the vehicle-related coordinate system, E1 is the sum of the lengths of the first perceived targets corresponding to that noise type in the first point cloud data in the vehicle-related coordinate system, A1 is the number of IDs of the first perceived targets corresponding to that noise type in the first point cloud data, Eb is the average width of the first perceived targets corresponding to that noise type in the vehicle-related coordinate system, E2 is the sum of the widths of the first perceived targets corresponding to that noise type in the first point cloud data in the vehicle-related coordinate system, Ec is the average height of the first perceived targets corresponding to that noise type in the vehicle-related coordinate system, and E3 is the sum of the heights of the first perceived targets corresponding to that noise type in the first point cloud data in the vehicle-related coordinate system.

[0246] One of the noise types can be any one of the following: image noise, cluster noise, multipath noise, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghost image, and distance ambiguity.

[0247] As in the example above, if the number of IDs of the first perceived target corresponding to the cluster noise (such as ID1, ID3, and ID5) is 3, and the length, width, and height of ID1 in the vehicle coordinate system are L1, W1, and H1 respectively, the length, width, and height of ID3 in the vehicle coordinate system are L3, W3, and H3 respectively, and the length, width, and height of ID5 in the vehicle coordinate system are L5, W5, and H5 respectively, then the average length of the first perceived target corresponding to the cluster noise in the vehicle coordinate system is (L1+L3+L5) / 3, the average width of the first perceived target corresponding to the cluster noise in the vehicle coordinate system is (W1+W3+W5) / 3, and the average height of the first perceived target corresponding to the cluster noise in the vehicle coordinate system is (H1+H3+H5) / 3.

[0248] In this embodiment of the application, a method for calculating the average size of the first sensing target corresponding to each noise type is provided, which can accurately obtain the average size of the first sensing target corresponding to each noise type.

[0249] In one possible implementation, the number of first sensing targets formed per unit mileage satisfies the following condition:

[0250] Fa = F1 / F2;

[0251] Where Fa is the number of first sensing targets formed per unit mileage, F1 is the number of IDs of first sensing targets formed in the first point cloud data, and F2 is the mileage corresponding to the first point cloud data.

[0252] The mileage corresponding to the first point of cloud data can be understood as the distance the vehicle travels while the sensor is outputting the first point of cloud data. The unit of mileage can be kilometers, hundreds of kilometers, tens of thousands of kilometers, etc.

[0253] For example, the unit of F2 can be 10,000 kilometers. If the number of IDs forming the first sensing target in the first point cloud data is 3, then Fa means that 3 first sensing targets are formed per 10,000 kilometers.

[0254] In this embodiment of the application, a method for calculating the number of first sensing targets formed per unit mileage is provided, which can accurately obtain the number of first sensing targets formed per unit mileage.

[0255] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0256] The point cloud processing device determines the second sensing target that triggers the alarm in the first point cloud data. The second sensing target that triggers the alarm is a second sensing target used to trigger the security system alarm. The second sensing target has the same attributes as the first sensing target.

[0257] Safety systems include one or more of the following: autonomous emergency braking (AEB), lane keeping assist (LKA), and adaptive cruise control (ACC).

[0258] In this embodiment, a second sensing target used to trigger an alarm in the security system can be identified in the first point cloud data. The noise performance of the sensor can be quantitatively evaluated based on the number of such targets in the first point cloud data, thereby allowing for the evaluation and comparison of sensor performance at higher risk levels. The number of second sensing targets in the first point cloud data that trigger an alarm is negatively correlated with sensor performance; a smaller number of such targets indicates better sensor performance.

[0259] The second sensing target has the same attributes as the first sensing target, which can be understood as the method of determining the second sensing target being the same as the method of determining the first sensing target. For example, the first sensing target is a sensing target in the first point cloud data whose corresponding noise number is greater than or equal to a first threshold, and the second sensing target is a sensing target in the first point cloud data whose corresponding noise number is greater than or equal to the first threshold.

[0260] In one possible implementation, the point cloud processing method described above may further perform the following steps, including but not limited to:

[0261] The point cloud processing device acquires the second point cloud data and determines the noise assessment result of the second point cloud data; the first point cloud data and the second point cloud data are data collected under the first condition or data collected under the second condition; the first condition is that the software versions used to collect the first point cloud data and the software versions used to collect the second point cloud data are different; the second condition is that the sensors used to collect the first point cloud data and the second point cloud data are different.

[0262] The point cloud processing device obtains a noise comparison result between the first point cloud data and the second point cloud data based on the noise evaluation results of the first point cloud data and the second point cloud data.

[0263] In this embodiment, the noise evaluation results of two point cloud data can be compared, and the noise comparison result of the first point cloud data and the second point cloud data can be obtained based on the noise evaluation results of the first point cloud data and the second point cloud data.

[0264] The noise comparison results have the following two application scenarios:

[0265] The first application scenario is comparing the noise results of two point cloud datasets collected by different software versions. Various noise quantification metrics can be compared between different software versions (the first and second point cloud datasets can be collected by the same sensor using different software versions) to evaluate the performance of different software versions.

[0266] The second application scenario involves comparing the noise results of two point cloud datasets collected by different sensors. This allows for the comparison of various noise quantification metrics between different hardware (where the first and second point cloud datasets were collected by different sensors), thus evaluating the performance of different sensors.

[0267] Taking lidar as an example, the software for collecting the first point cloud data can directly calculate or derive information about the reflection point (distance, angle, reflection intensity, etc.) based on the relevant information of the emitted and reflected rays from the lidar, and output the point cloud data. When different software versions are used, the output point cloud data will also differ under the same conditions (e.g., the same sensor, the same road conditions, and the same environment).

[0268] In this embodiment, the first point cloud data and the second point cloud data can be obtained by using the controlled variable method, and the noise comparison result of the first point cloud data and the second point cloud data can be obtained based on the noise evaluation result of the first point cloud data and the second point cloud data.

[0269] Optionally, the first condition is: the software versions used to collect the first point cloud data and the software versions used to collect the second point cloud data are different, while other conditions are the same (e.g., the sensors used to collect the first point cloud data and the second point cloud data are the same, the road conditions and environment used to collect the first point cloud data and the second point cloud data are the same, the number of frames used to collect the first point cloud data and the second point cloud data are the same, the time periods used to collect the first point cloud data and the second point cloud data are the same, etc.).

[0270] Optionally, the second condition is: the sensors used to collect the first point cloud data and the second point cloud data are different, while other conditions are the same (e.g., the software versions used to collect the first point cloud data and the second point cloud data are the same, the road conditions and environment used to collect the first point cloud data and the second point cloud data are the same, the number of frames used to collect the first point cloud data and the second point cloud data are the same, the time periods used to collect the first point cloud data and the second point cloud data are the same, etc.).

[0271] In one possible implementation, after obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the above point cloud processing method may further perform the following steps, including but not limited to:

[0272] The point cloud processing device determines the first type of frame point cloud in the first point cloud data. The first type of frame point cloud is the frame point cloud in the first point cloud data that contains the first perceived target.

[0273] The point cloud processing device outputs one or more of the following: the frame index value of the first type of frame point cloud, the position of the first perceived target in the frame point cloud corresponding to the frame index value, the noise type of the first perceived target in the frame point cloud corresponding to the frame index value, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, a first image synchronously acquired with the frame point cloud corresponding to the frame index value, and a second image synchronously acquired with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

[0274] Optionally, the point cloud processing device may include a display device, which can display information such as the frame index value of the first type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value.

[0275] Optionally, the point cloud processing device may not include a display device. The point cloud processing device can output information such as the frame index value of the first type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value to the display device, and display the information such as the frame index value of the first type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value through the display device.

[0276] In this embodiment, the first type of frame point cloud is the frame point cloud in which the presence of a first sensing target is detected. The first type of frame point cloud can also be referred to as a sensing-level problem frame. The first type of frame point cloud can be identified, and its relevant information can be output. Visual analysis of the first type of frame point cloud can be performed, thereby allowing for rapid manual assessment of the severity of the sensing-level problem. A sensing-level problem may include any one or more of the following: the number of frames of the first type of frame point cloud in the first point cloud data, the proportion of the first type of frame point cloud in the first point cloud data, the location of the first sensing target in the first type of frame point cloud, the noise type of the first sensing target in the first type of frame point cloud, the number of noise points in the first type of frame point cloud, the noise type of the noise points in the first type of frame point cloud, and the location of the noise points in the first type of frame point cloud.

[0277] The first and second images can assist in the analysis of the reasons for the appearance of point clouds in the first type of frame.

[0278] In one possible implementation, after obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the above point cloud processing method may further perform the following steps, including but not limited to:

[0279] The point cloud processing device determines the second type of frame point cloud in the first point cloud data. The second type of frame point cloud is the frame point cloud in the first point cloud data that contains noise.

[0280] The point cloud processing device outputs one or more of the following: the frame index value of the second type of frame point cloud, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, the noise type of the noise point in the frame point cloud corresponding to the frame index value, a first image synchronously acquired with the frame point cloud corresponding to the frame index value, and a second image synchronously acquired with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

[0281] Optionally, the point cloud processing device may include a display device, which can display information such as the frame index value of the second type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value.

[0282] Optionally, the point cloud processing device may not include a display device. The point cloud processing device can output information such as the frame index value of the second type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value to the display device, and display the information such as the frame index value of the second type of frame point cloud and the position of the first perceived target in the frame point cloud corresponding to the frame index value through the display device.

[0283] In this embodiment, the second type of frame point cloud is a frame point cloud in which noise is detected; the second type of frame point cloud can also be referred to as a point cloud-level problem frame. The second type of frame point cloud can be identified, and its relevant information can be output. Visual analysis of the second type of frame point cloud is possible, allowing for rapid manual assessment of the severity of point cloud-level problems. Point cloud-level problems may include any one or more of the following: the number of frames of the second type of frame point cloud in the first point cloud data, the proportion of the second type of frame point cloud in the first point cloud data, the number of noise points in the second type of frame point cloud, the noise type of the noise points in the second type of frame point cloud, and the location of the noise points in the second type of frame point cloud.

[0284] The first and second images can assist in the analysis of the reasons for the appearance of the second type of frame point cloud.

[0285] Please refer to Figure 6, which is a schematic flowchart of a point cloud processing method provided in an embodiment of this application. This point cloud processing method can be executed by a point cloud processing device, as shown in Figure 6. The point cloud processing device may include a perception recognition model, a noise recognition model, a matching determination module, and a statistical analysis module.

[0286] Vehicle posture, extrinsic parameters, and continuous frame point clouds are used as inputs to the perception recognition model and noise recognition model. The perception recognition model outputs the perception result for the current frame, and the noise recognition model outputs the noise recognition result for the current frame. The matching and judgment module matches the perception result and the noise recognition result for the current frame to identify the first perceived target in that frame. The statistical analysis module performs statistical analysis on the continuous multi-frame point clouds to obtain the noise assessment result of the continuous multi-frame point clouds. The continuous frame point clouds can be understood as the aforementioned first frame point cloud. The perception result for this frame is information at the perceived target level, which can be understood as the position, size, and other information of one or more perceived targets in the aforementioned first frame point cloud. The noise recognition result for this frame is a point-level noise recognition result, which can be understood as the noise information in the aforementioned first frame point cloud. The perception recognition model may include functional modules such as self-motion prediction, target tracking prediction, ground segmentation, point cloud clustering, target detection, target tracking, lane line fitting, and roadside fitting. The noise assessment result of the continuous multi-frame point clouds can be understood as the noise assessment result of the aforementioned first point cloud data.

[0287] The point cloud processing method of this application embodiment can be performed in open road testing of vehicles. This point cloud processing method includes a data acquisition stage, a data processing stage, and a data analysis stage. Please refer to Figure 7, which is a schematic diagram of the positional relationship of sensors mounted on a vehicle according to an embodiment of this application. As shown in Figure 7, the vehicle can be equipped with sensors such as a vehicle attitude sensor, LiDAR, and a camera. The sensor positions in Figure 7 are one possible example; the actual installation positions of the sensors can vary. In the data acquisition stage, the original vehicle attitude is acquired using the vehicle attitude sensor on an open road, and point cloud data is acquired using the LiDAR. The extrinsic parameters of the LiDAR in the vehicle coordinate system when acquiring the point cloud data are recorded. These extrinsic parameters can be six-degree-of-freedom extrinsic parameters (x, y, z, yaw, pitch, roll).

[0288] Please refer to Figure 8, which is a schematic diagram of the positional relationship between a perceived target and a point cloud provided in an embodiment of this application. As shown in Figure 8, the three-dimensional rectangle represents the three-dimensional bounding box of the perceived target in the current frame perception result output by the perception recognition model, and the gray dots represent points in the point cloud. The information of the point cloud includes the x-axis coordinate, y-axis coordinate, z-axis coordinate, vertical point cloud channel number, horizontal point cloud slot number, intensity, reflectivity, and noise ID of each point in the point cloud. The information of the perceived target includes the position information, size information, height information, speed information, and ID of the perceived target. In the data processing stage, the perception recognition model obtains information such as the vehicle's motion pose based on the vehicle posture collected by the vehicle posture sensor and the perception recognition algorithm. The perception recognition algorithm may include the iterative closest point (ICP) point cloud matching algorithm, the simultaneous localization and mapping (SLAM) algorithm, etc. The current frame perception result output by the perception recognition model may include: the position information, size information, height information, speed information, and ID of one or more perceived targets in the current frame point cloud. The noise recognition model, based on a noise recognition algorithm, identifies noise information within an input frame point cloud. This noise information includes the location of the noise points and the noise category (noiseID). The noise recognition algorithm can include supervised learning-based extreme gradient boosting (XG-BOOST), neural network algorithms, etc. The noise recognition model outputs the noise recognition result for the current frame, which may include the location information and noise category of one or more noise points in the current frame point cloud.

[0289] Please refer to Figure 9, which is a schematic diagram illustrating the positional relationship between a perceived target, a real point cloud, and a noise point cloud according to an embodiment of this application. As shown in Figure 9, the 3D bounding box of the perceived target on the left contains real point clouds but no noise point clouds, making the perceived target on the left a real target. The 3D bounding box of the perceived target on the right contains both real and noise point clouds, and the number of noise point clouds is 6, which is greater than a first threshold (for example, the first threshold can be set to 3), making the perceived target on the right a false alarm target. In Figure 9, there are 7 noise point clouds on the right side of the 3D bounding box of the perceived target on the right that do not form a perceived target; these are noise point clouds that do not form a perceived target. Real point clouds are the real points in the point cloud, and noise point clouds are the noise points in the point cloud. During the data analysis phase, the matching and judgment module can use the 3D Intersection over Union (3D IoU) algorithm to determine whether a perceived target is a real target or a false alarm target based on the number of noise points within the 3D bounding box of each perceived target in 3D space. This filters out false alarm targets caused by noise. False alarm targets are targets that do not exist in the physical world and are falsely detected due to noise point clouds. False alarm targets can be understood as the first perceived target mentioned above. Noise information within perceived targets has a more severe impact on subsequent intelligent driving algorithms than noise information that does not form a perceived target. Noise information within perceived targets can also characterize the performance of the LiDAR. The noise category of a false alarm target is determined by the mode of noise categories within the false alarm target. Based on the noise points of different noise categories, statistical analysis is performed in various dimensions, and noise quantification indicators of the continuous frame point cloud are output. The performance of the LiDAR is described through various noise quantification indicators rather than fuzzy conclusions determined manually, thus facilitating a more reliable evaluation and comparison of LiDAR performance.

[0290] The matching and determination process during the data analysis phase is specifically executed through steps S1 to S6.

[0291] S1. For each frame of point cloud, input the perceptual target information T1, T2, ..., Tn and the noise information P1, P2, ..., Pn. Based on 3DIOU, calculate whether each noise point is within the 3D bounding box of a certain perceptual target. Noise points within the same target bounding box form a set, thereby obtaining the point cloud index value sets S1, S2, ..., Sn of each perceptual target with noise; where S1 is the noise index value set corresponding to perceptual target T1, S2 is the noise index value set corresponding to perceptual target T2, and Sn is the noise index value set corresponding to perceptual target Tn.

[0292] S2. For each point cloud set Si (i = 1 to n) of a sensing target, calculate the number of noise points within Si. If the number of noise points exceeds a first threshold, the sensing target corresponding to Si is considered a false alarm target, and the ID of the sensing target is recorded. Calculate the mode s_max of the noise type in the false alarm target and the corresponding noise type. For example, if there are 3 noise points in the false alarm target: P1, P2, and P3, where P1 and P2 are both spike crosstalk and P3 is cluster noise, then the mode s_max = 2, and the corresponding noise type is spike crosstalk.

[0293] S3. Among multiple frames of point clouds, count and merge the false alarm targets according to their IDs, with the same ID counted only once; at the same time, calculate the number of instances (cases) corresponding to each noise type based on the frames in which various noise types appear. A case for each noise type is defined as a range in which the noise type appears in consecutive frames. For example, if the road spike crosstalk noise points appear in frames 1, 2, 3, 4, 50, 51, and 52, there are 2 cases, namely frames 1 to 4 and frames 50 to 52.

[0294] S4. For various types of noise, count the number of noise points within each false alarm target, the distance from the center of the three-dimensional bounding box of each false alarm target to the lidar, the height of the center of the three-dimensional bounding box, and the three-dimensional dimensions of the three-dimensional bounding box, and finally obtain the average number of noise points, distance, height, and dimensions of the false alarm targets.

[0295] The statistics in step S4 are divided into two modes: frame counting mode and target counting mode. The former calculates the average value of various indicators such as the number of noise points, distance, height, and size using the frames with noise false alarms as the base (denominator). The latter calculates the average value using the number of targets as the base (denominator). The former outputs the average indicators per frame, while the latter outputs the average indicators per target, focusing on the performance of the point cloud side and the perception side, respectively. The specific difference is that when multiple false alarm targets appear in a frame, the former counts them only once, while the latter counts them multiple times.

[0296] S5. Based on the mileage accumulated during vehicle movement, output the number of false alarm targets formed per unit mileage;

[0297] S6. Output the 3D bounding box of the false alarm target in the problem frame and the corresponding point cloud index value. This supports rapid visual analysis of problems categorized by noise level, allowing for quick qualitative secondary manual confirmation of problem severity. The problem frame can be either the first type or the second type of point cloud mentioned above.

[0298] This application provides a method for matching and statistically analyzing perceived targets and noise points, which enables automated evaluation and improves execution efficiency. Quantitative indicators such as the number of noise points forming false alarm targets, distance, height, size, and the number of false alarm targets per unit mileage can be used to improve the objectivity of point cloud processing methods, resulting in stable and traceable evaluation results.

[0299] In this embodiment, the point cloud processing device can output qualitative and quantitative reports. Please refer to Figure 10, which is a schematic diagram of a visualization interface provided in this embodiment. As shown in Figure 10, the visualization interface can display a qualitative report. The left side of Figure 10 shows an image of a point cloud frame, the upper right corner shows an image containing the road surface captured by the camera corresponding to that point cloud frame, and the lower right corner shows an image containing LiDAR captured by the camera corresponding to that point cloud frame. The acquisition time of the point cloud frame on the left, the capture time of the image in the upper right corner, and the capture time of the image in the lower right corner can all be the same point in time. The user can select the severity level as either perception level or point cloud level. The point cloud frame corresponding to perception level can be understood as the first type of point cloud frame mentioned above, and the point cloud frame corresponding to point cloud level can be understood as the second type of point cloud frame mentioned above. The user can also select the noise type and frame index value of the problem frame to be displayed. As shown in Figure 10, if the user selects perception level severity, multipath noise type for the selected problem frame, and frame index value 7030, then the frame index value of the point cloud frame displayed on the left side of Figure 10 is 7030. The frame point cloud shown on the left side of Figure 10 contains point clouds corresponding to the real target and point clouds corresponding to false alarm targets. The point cloud corresponding to the real target is the point cloud corresponding to the vehicle, and the point cloud corresponding to the false alarm target is the noise point cloud contained within the rectangle on the left side of Figure 10. The noise type of this false alarm target is multipath. The noise point cloud contained within the rectangle on the left side of Figure 10 can be highlighted (for example, it can be marked with a color different from the real point cloud). The image containing the LiDAR in the lower right corner of Figure 10 clearly shows the outer surface of the LiDAR. Users can observe whether there is dust contamination on the outer surface of the LiDAR and whether the LiDAR's viewport is obstructed by foreign objects through the image in the lower right corner, thereby assisting in manual analysis of the cause of the problematic frame. The qualitative report shows the morphological information such as the position and size of the false alarm target in the problematic frame, which can enhance the visualization interaction and can be used for manual analysis and subsequent software and hardware iterations.

[0300] Please refer to Table 2, which is a table for a quantitative report provided in an embodiment of this application.

[0301] Table 2

[0302] As shown in Table 2, the number of cases represents the number of cases with false alarm targets corresponding to each noise type, which can be understood as the number of instances of the first sensing target corresponding to each noise type. The average number of noise points represents the average number of noise points in the false alarm targets corresponding to each noise type, which can be understood as the average number of noise points in the first sensing target corresponding to each noise type. The average position represents the average position of the false alarm targets corresponding to each noise type, which can be understood as the average position of the first sensing target corresponding to each noise type. The average size represents the dimensions (length, width, and height) of the false alarm targets corresponding to each noise type, which can be understood as the average size of the first sensing target corresponding to each noise type.

[0303] The quantitative report, through automated evaluation of extensive drive test data, outputs noise assessment results for point cloud data. These results include various quantitative metrics to evaluate the performance of the point cloud. Performance differences between different hardware / software versions can be assessed by comparing these quantitative metrics.

[0304] Please refer to Table 3, which is a comparison table of the number of instances of false alarm targets corresponding to various noise types for the two software versions provided in the embodiments of this application.

[0305] Table 3

[0306] As can be seen from Table 3, the overall frequency of problems in version 2 is lower than that in version 1.

[0307] Please refer to Figure 11, which is a schematic diagram of another visualization interface provided in an embodiment of this application. As shown in Figure 11, the user can select to display a problem frame of a certain type of noise. In terms of problem dimension, the user can view all problem frames under a certain noise type and the location of the corresponding problem noise, the location and size of the false alarm target caused by it, etc.; the user can also view all noise types and corresponding noise performance of each frame of point cloud frame by frame.

[0308] The embodiments of this application can be applied to the point cloud performance quality evaluation of LiDAR. In the visualization interface, both noisy point clouds and real point clouds belong to the point cloud data output by LiDAR. Noisy point clouds and real point clouds can be distinguished by marking and coloring.

[0309] Meanwhile, changes in point cloud performance are also affected by the hardware and software of the LiDAR. Whether the LiDAR test and evaluation is triggered by hardware changes or software changes, it is applicable to the point cloud processing method involved in the embodiments of this application, so as to prove the effectiveness and usability of software improvements and / or hardware improvements.

[0310] The methods of the embodiments of this application have been described in detail above. The following provides an apparatus for implementing any one of the methods in the embodiments of this application. For example, an apparatus is provided that includes a unit (or means) for implementing the steps performed by the device in any of the above methods.

[0311] Please refer to Figure 12, which is a schematic diagram of the structure of a point cloud processing device provided in an embodiment of this application.

[0312] As shown in Figure 12, the point cloud processing device 120 may include a communication unit 1201 and a processing unit 1202. The communication unit 1201 and the processing unit 1202 may be software, hardware, or a combination of software and hardware.

[0313] The communication unit 1201 can implement sending and / or receiving functions, and can also be described as a transceiver unit. The communication unit 1201 can also be a unit integrating an acquisition unit and a sending unit, wherein the acquisition unit is used to implement the receiving function, and the sending unit is used to implement the sending function. Optionally, the communication unit 1201 can be used to receive information sent by other devices, and can also be used to send information to other devices.

[0314] In one possible design, the point cloud processing device 120 may correspond to the point cloud processing device in the method embodiment shown in FIG4 above. For example, the point cloud processing device 120 may be an electronic device or a chip within an electronic device. The point cloud processing device 120 may include units for performing the operations performed by the point cloud processing device in the method embodiment shown in FIG4 above, and each unit in the point cloud processing device 120 is for implementing the operations performed by the point cloud processing device in the method embodiment shown in FIG4 above. The descriptions of each unit are as follows:

[0315] Processing unit 1202 is used to determine noise information in the first frame point cloud and one or more sensing targets in the first frame point cloud;

[0316] The processing unit 1202 is further configured to determine, based on the noise information and the one or more sensing targets, whether there is a first sensing target among the one or more sensing targets, wherein the first sensing target is determined based on the noise information corresponding to the first sensing target.

[0317] In one possible implementation, the device further includes a communication unit 1201;

[0318] The processing unit 1202 is specifically used to acquire the first frame point cloud through the communication unit 1201.

[0319] Regarding the communication unit 1201 and processing unit 1202 described in this design, the steps they perform can be referred to the implementation method corresponding to the point cloud processing device in the method embodiment shown in Figure 4 above.

[0320] Regarding the technical effects of the implementation methods performed by the communication unit 1201 and processing unit 1202 described in this design, please refer to the description of the technical effects corresponding to the method embodiment shown in FIG4 above.

[0321] The point cloud processing apparatus 120 described in Figure 12 can improve the objectivity of point cloud quality assessment.

[0322] If the point cloud processing device 120 described above can be an electronic device, please refer to the structural schematic diagram of the electronic device shown in Figure 13.

[0323] It should be understood that the electronic device 130 shown in FIG13 is only an example. The electronic device in the embodiments of this application may also include other components, or include components with functions similar to the various components in FIG13, or may not be intended to include all the components in FIG13.

[0324] Electronic device 130 includes a transceiver interface 1301 and at least one processor 1302.

[0325] The electronic device 130 can correspond to a point cloud processing device. The transceiver interface 1301 is used to transmit and receive signals, and at least one processor 1302 executes program instructions, causing the electronic device 130 to implement the corresponding process of the method executed by the corresponding device in the above method embodiment.

[0326] In one possible design, the electronic device 130 may correspond to the point cloud processing device in the method embodiment shown in FIG4 above. For example, the electronic device 130 may be a point cloud processing device or a chip within the point cloud processing device. The electronic device 130 may include components for performing the operations performed by the point cloud processing device in the method embodiment above, and each component in the electronic device 130 is specifically designed to implement the operations performed by the point cloud processing device in the method embodiment above. Specifically, it may be as follows:

[0327] Processor 1302 is used to determine noise information in a first frame point cloud and one or more sensing targets in the first frame point cloud;

[0328] The processor 1302 is further configured to determine, based on the noise information and the one or more sensing targets, whether there is a first sensing target among the one or more sensing targets, wherein the first sensing target is determined based on the noise information corresponding to the first sensing target.

[0329] In one possible implementation, the device further includes a transceiver interface 1301;

[0330] The processor 1302 is specifically used to acquire the first frame point cloud through the transceiver interface 1301.

[0331] Regarding the transceiver interface 1301 and at least one processor 1302 described in this design, the steps performed can be referred to the implementation corresponding to the point cloud processing device in the method embodiment shown in Figure 4 above.

[0332] For the technical effects of the implementation methods performed by the transceiver interface 1301 and at least one processor 1302 described in this design, please refer to the description of the technical effects corresponding to the method embodiment shown in FIG4 above.

[0333] The objectivity of point cloud quality assessment can be improved in the electronic device 130 described in Figure 13.

[0334] If the point cloud processing device 120 described above can be a chip or a chip system, please refer to the schematic diagram of the chip structure shown in Figure 14.

[0335] As shown in Figure 14, chip 140 includes processor 1401 and interface 1402. The number of processors 1401 can be one or more, and the number of interfaces 1402 can be multiple. It should be noted that the functions of processor 1401 and interface 1402 can be implemented through hardware design, software design, or a combination of both; no restrictions are placed here.

[0336] Optionally, chip 140 may also include memory 1403 for storing necessary program instructions and data.

[0337] In this application, processor 1401 can be used to call the implementation program of the point cloud processing method provided in one or more embodiments of this application in a point cloud processing device from memory 1403, and execute the instructions included in the program. Interface 1402 can be used to output the execution result of processor 1401. In this application, interface 1402 can be specifically used to output various messages or information of processor 1401.

[0338] The point cloud processing method provided by one or more embodiments of this application can be referred to the various embodiments shown in Figure 4 above, and will not be repeated here.

[0339] The processor in this application embodiment can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0340] The memory in this application embodiment is used to provide storage space, in which data such as operating system and computer programs can be stored. The memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0341] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a computer program. When the computer program is run on one or more processors, it can implement the method shown in FIG4.

[0342] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer program product, which includes a computer program. When the computer program runs on a processor, it can implement the method shown in FIG4.

[0343] This application embodiment also provides a terminal, which includes at least one point cloud processing device 120, or electronic device 130, or chip 140.

[0344] Optionally, the terminal can be a means of transportation, such as a car, truck, aircraft, drone, slow transport vehicle, spacecraft, or ship, or any other possible means of transportation used in any possible scenario. This application embodiment does not limit this.

[0345] Optionally, the terminal is used to implement the method shown in Figure 4 above.

[0346] Optionally, the point cloud processing method shown in Figure 4 above can be carried in the vehicle operating system (VOS) of the terminal in the form of an executable file.

[0347] This application also provides a processing apparatus, including a processor and an interface; the processor is used to execute the method in any of the above method embodiments.

[0348] It should be understood that the above-described processing device can be a chip. The units in the various device embodiments and the electronic devices in the method embodiments correspond completely, with corresponding modules or units executing corresponding steps. For example, the communication unit (transceiver) executes the receiving or sending steps in the method embodiments, while other steps besides sending and receiving can be executed by the processing unit (processor). The specific functions of each unit can be found in the corresponding method embodiments. There can be one or more processors.

[0349] It is understood that in the embodiments of this application, the electronic device may perform some or all of the steps in the embodiments of this application. These steps or operations are merely examples, and the embodiments of this application may also perform other operations or variations thereof. Furthermore, the steps may be performed in different orders as presented in the embodiments of this application, and it is not necessarily necessary to perform all the operations in the embodiments of this application.

[0350] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0351] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0352] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0353] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the contributing part, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0354] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A point cloud processing method, characterized in that, The point cloud processing method includes: Determine the noise information in the first frame point cloud and one or more perceived targets in the first frame point cloud; Based on the noise information and the one or more sensing targets, it is determined whether there is a first sensing target among the one or more sensing targets, and the first sensing target is determined based on the noise information corresponding to the first sensing target.

2. The method of claim 1, wherein, The noise information includes the location information of at least one noise point; The first sensing target is the sensing target whose number of corresponding noise points is greater than or equal to the first threshold among the one or more sensing targets.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Based on one or more of the noise information corresponding to the first sensing target and the location information of the first sensing target, a noise quantization index of the first sensing target is obtained; the noise quantization index of the first sensing target includes any one or more of the following: the noise type of the first sensing target, the number of noise points corresponding to the first sensing target, the lateral distance between the first sensing target and the vehicle, the longitudinal distance between the first sensing target and the vehicle, the size of the first sensing target, and the height of the first sensing target.

4. The method according to claim 2 or 3, characterized in that, The noise information also includes the noise type of the at least one noise point; The noise type of the first sensing target includes the noise type with the most noise points corresponding to the first sensing target; And / or, The lateral distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the horizontal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or, The longitudinal distance between the first perceived target and the vehicle includes: the distance between the projection point of the center of the first perceived target onto the longitudinal axis of the vehicle's relevant coordinate system and the origin of the coordinate system; and / or, The height of the first perceived target includes: the distance between the projection point of the center of the first perceived target onto the vertical axis of the vehicle-related coordinate system and the origin of the coordinate system; and / or, The size of the first perceived target includes any one or more of the length, width, and height of the first perceived target in the vehicle-related coordinate system.

5. The method of claim 4, wherein, The noise types include any of the following: image noise, cluster noise, multipath noise, adhesion, road sign crosstalk, vehicle crosstalk, road stud crosstalk, wheel chock crosstalk noise, ghosting, and distance blur.

6. The method according to any one of claims 3 to 5, characterized in that, Before determining the noise information in the first frame point cloud and one or more perceived targets in the first frame point cloud, the method further includes: Acquire first point cloud data, wherein the first point cloud data includes at least the point cloud of the first frame; After obtaining the noise quantization index of the first sensing target based on one or more of the noise information corresponding to the first sensing target and the location information of the first sensing target, the method further includes: Based on the noise quantization index of the first perceived target in the first point cloud data, the noise assessment result of the first point cloud data is obtained. The noise assessment results of the first point cloud data include any one or more of the following: the number of instances corresponding to each noise type, the number of instances of the first sensing target corresponding to each noise type, the number of identity IDs of the first sensing target corresponding to each noise type, the average number of noise points of the first sensing target corresponding to each noise type, the average position of the first sensing target corresponding to each noise type, the average size of the first sensing target corresponding to each noise type, and the number of first sensing targets formed per unit mileage. Among these, consecutive frame point clouds in the first point cloud data that have noise points of the same noise type are considered an instance corresponding to that noise type, and consecutive frame point clouds in the first point cloud data that have the same noise type and the same ID are considered an instance of the first sensing target corresponding to that noise type.

7. The method of claim 6, wherein, The average number of noise points of the first sensing target corresponding to each noise type satisfies the following condition: Ca = B1 / A1, or Ca = B1 / A2; Wherein, Ca is the average number of noise points of the first sensing target corresponding to each noise type in the first point cloud data, B1 is the sum of the number of noise points corresponding to the first sensing target corresponding to each noise type in the first point cloud data, A1 is the number of IDs of the first sensing target corresponding to each noise type in the first point cloud data, and A2 is the number of frames in the first point cloud data that contain the first sensing target corresponding to each noise type.

8. The method of claim 6, wherein, The average position of the first sensing target corresponding to each noise type includes any one or more of the following: the average lateral distance, average longitudinal distance, and average height of the first sensing target and the vehicle corresponding to each noise type; The average lateral distance between the first sensing target and the vehicle corresponding to each noise type satisfies the following condition: Da = D1 / A1; and / or, The average longitudinal distance between the first sensing target and the vehicle corresponding to each noise type satisfies the following condition: Db = D2 / A1; and / or, The average height of the first sensing target corresponding to each noise type satisfies the following condition: Dc = D3 / A1; Wherein, Da is the average lateral distance between the first sensing target and the vehicle corresponding to each noise type, D1 is the sum of the lateral distances between the first sensing target and the vehicle corresponding to each noise type in the first point cloud data, A1 is the number of IDs of the first sensing target corresponding to each noise type in the first point cloud data, Db is the average longitudinal distance between the first sensing target and the vehicle corresponding to each noise type, D2 is the sum of the longitudinal distances between the first sensing target and the vehicle corresponding to each noise type in the first point cloud data, Dc is the average height of the first sensing target corresponding to each noise type, and D3 is the sum of the heights of the first sensing targets corresponding to each noise type in the first point cloud data.

9. The method of claim 6, wherein, The average size of the first sensing target corresponding to each noise type includes any one or more of the following: the average length, average width, and average height of the first sensing target corresponding to each noise type in the vehicle-related coordinate system; The average length of the first sensing target corresponding to each noise type in the vehicle-related coordinate system satisfies the following condition: Ea = E1 / A1; and / or, The average width of the first sensing target corresponding to each noise type in the vehicle-related coordinate system satisfies the following condition: Eb = E2 / A1; and / or, The average height of the first sensing target corresponding to each noise type in the vehicle-related coordinate system satisfies the following condition: Ec = E3 / A1; Wherein, Ea is the average length of the first sensing target corresponding to each noise type in the vehicle-related coordinate system, E1 is the sum of the lengths of the first sensing targets corresponding to each noise type in the first point cloud data in the vehicle-related coordinate system, A1 is the number of IDs of the first sensing targets corresponding to each noise type in the first point cloud data, Eb is the average width of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system, E2 is the sum of the widths of the first sensing targets corresponding to each noise type in the first point cloud data in the vehicle-related coordinate system, Ec is the average height of the first sensing targets corresponding to each noise type in the vehicle-related coordinate system, and E3 is the sum of the heights of the first sensing targets corresponding to each noise type in the first point cloud data in the vehicle-related coordinate system.

10. The method of claim 6, wherein, The number of first sensing targets formed per unit mileage satisfies the following condition: Fa = F1 / F2; Where Fa is the number of first sensing targets formed per unit mileage, F1 is the number of IDs of the first sensing targets formed in the first point cloud data, and F2 is the mileage corresponding to the first point cloud data.

11. The method according to any one of claims 6 to 10, characterized in that, The method further includes: Identify a second sensing target in the first point cloud data that triggers an alarm. The second sensing target that triggers the alarm is a second sensing target used to trigger a security system alarm. The second sensing target has the same attributes as the first sensing target. The safety system includes one or more of the following: Emergency Braking (AEB), Lane Keeping Assist (LKA), and Adaptive Cruise Control (ACC).

12. The method according to any one of claims 6 to 11, characterized in that, The method further includes: Acquire second point cloud data and determine the noise assessment result of the second point cloud data; the first point cloud data and the second point cloud data are data collected under a first condition or data collected under a second condition; the first condition is: the software versions used to collect the first point cloud data and the software versions used to collect the second point cloud data are different; the second condition is: the sensors used to collect the first point cloud data and the software versions used to collect the second point cloud data are different; Based on the noise assessment results of the first point cloud data and the second point cloud data, a noise comparison result between the first point cloud data and the second point cloud data is obtained.

13. The method according to any one of claims 6 to 12, characterized in that, After obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the method further includes: Identify a first type of frame point cloud in the first point cloud data, wherein the first type of frame point cloud is a frame point cloud in the first point cloud data in which a first sensing target exists; Output any one or more of the following: the frame index value of the first type of frame point cloud, the position of the first perceived target in the frame point cloud corresponding to the frame index value, the noise type of the first perceived target in the frame point cloud corresponding to the frame index value, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, a first image synchronously acquired with the frame point cloud corresponding to the frame index value, and a second image synchronously acquired with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

14. The method according to any one of claims 6 to 12, characterized in that, After obtaining the noise assessment result of the first point cloud data based on the noise quantization index of the first perceived target in the first point cloud data, the method further includes: Determine the second type of frame point cloud in the first point cloud data, where the second type of frame point cloud is a frame point cloud in the first point cloud data that contains noise. Output any one or more of the following: the frame index value of the second type of frame point cloud, the noise point in the frame point cloud corresponding to the frame index value, the non-noise point in the frame point cloud corresponding to the frame index value, the noise type of the noise point in the frame point cloud corresponding to the frame index value, a first image synchronously acquired with the frame point cloud corresponding to the frame index value, and a second image synchronously acquired with the frame point cloud corresponding to the frame index value, wherein the first image is an image containing the road surface and the second image is an image containing the sensor.

15. A point cloud processing device, characterized in that, Includes units for performing the method as described in any one of claims 1 to 14.

16. A point cloud processing device, characterized in that, Includes a processor for performing the method as described in any one of claims 1 to 14.

17. A chip, characterized by The chip includes logic circuitry and an interface, wherein the logic circuitry and the interface are coupled. The interface is used for inputting and / or outputting information, and the logic circuit is used for performing the method as described in any one of claims 1 to 14.

18. A terminal, characterized by Includes the point cloud processing apparatus as described in claim 15, or the point cloud processing apparatus as described in claim 16, or the chip as described in claim 17.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed, performs the method as described in any one of claims 1 to 14.

20. A computer program product, characterised in that, The computer program product includes a computer program, which, when executed, performs the method as described in any one of claims 1 to 14.

Images