Point cloud test method and device

Abstract

A point cloud test method and device are applied to the technical field of detection and can test the point cloud of a body target output by a to-be-detected detection device, and specifically, the detection device can be a millimeter wave radar. During testing, the point cloud testing device matches the true value of the body target with the point cloud of the body target to obtain the matching condition between the point cloud of the body target and the true value of the body target. As the body target is closer to the detection target of the to-be-detected detection device in the real installation environment, the quality of the point cloud output by the detection device can be more accurately tested, and the detection capability of the detection device can be evaluated. In addition, matching is automatically performed according to the true value of the body target and the point cloud of the body target, so that the evaluation error can be remarkably reduced, and the test precision and the test efficiency are improved.

Description

Point cloud testing method and device Technical Field

[0001] The present application relates to the field of detection technology, and in particular to a point cloud testing method and device. Background Art

[0002] With the development of information technology, detection technology has made rapid progress. Various detection devices have brought great convenience to people's lives and travel. For example, advanced driving assistance systems (ADAS) play a very important role in smart cars. They use detection devices installed on the vehicle to detect the surrounding environment while the vehicle is driving, collect data, identify static and moving objects, and perform systematic calculations and analysis based on maps. This allows the driver to be aware of potential dangers in advance, effectively improving the comfort and safety of driving. The detection device can be thought of as the device's "eyes" that perceive the environment, capable of detecting the surrounding environment and outputting a point cloud. The quality of the point cloud represents the detection capability of the detection device. Therefore, testing the point cloud output by the detection device (hereinafter referred to as point cloud testing) has always been a key test item for detection devices within the industry.

[0003] The technology of using point targets to perform point cloud testing on detection devices is already very mature. Among them, point targets are targets that exist in the form of "points". Exemplary point cloud testing methods for point targets are as follows: the device under test (DUT) is placed in a darkroom (surrounded by absorbing materials) and point targets are set in the darkroom to test the point cloud output by the DUT when detecting the point targets in the darkroom.

[0004] However, point targets differ significantly from detection targets in a real-world environment (hereinafter referred to as real-world targets). For example, point targets typically only have characteristics such as position, distance, and orientation, but real-world targets also have characteristics such as posture or size, and scattering characteristics also vary. Currently, some vendors use only the human eye to estimate the quality of the point cloud output by detection devices when detecting real-world targets. In short, current point cloud testing methods make it difficult to accurately evaluate the quality of point clouds output by detection devices.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a point cloud testing method and device, which can test the point cloud of a body target and can more accurately test the quality of the point cloud output by a detection device.

[0007] In a first aspect, an embodiment of the present application provides a point cloud testing method, comprising:

[0008] Obtain true value data and point cloud. The true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT detecting the volume target.

[0009] The point cloud and the true value data are matched to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0010] Among them, the body target can be regarded as an object with at least two of the length, height and width, which is used as a target for testing the DUT. The body target includes but is not limited to one or more of a sphere, a cuboid, a plate, a dihedral, a trihedral, a cylinder, or a circular top hat. When the detection device detects the body target, the detection result of a body target includes multiple sampling points. Optionally, when detecting the body target, the detection device can detect different surfaces of the body target at different viewing angles. In some scenarios, during the process of the detection device detecting the body target, the body target has characteristics such as posture and size.

[0011] The present embodiment matches the true value of the volume target with the point cloud of the volume target, obtaining a match between the point cloud of the volume target and the true value of the volume target. Because the volume target is closer to the actual target, the present embodiment can more accurately test the quality of the point cloud output by the detection device, which is beneficial for evaluating the detection capability of the detection device.

[0012] In addition, the embodiment of the present application can significantly reduce the evaluation error and improve the test accuracy and test efficiency by automatically obtaining the matching result based on the true value of the volume target and the point cloud of the volume target.

[0013] Optionally, the number of volume targets may be one or more. In order to facilitate the description of the solution of the present application, the number of volume targets is described as at least one below.

[0014] Optionally, the above method can be implemented by a point cloud testing device. The following description takes the point cloud testing device as an example of an execution subject of the method, and the present application is also applicable to other forms of execution subjects.

[0015] In another possible implementation of the first aspect, matching the point cloud with the ground truth data to obtain a matching result set includes:

[0016] Establish a three-dimensional matching box based on the ground truth data;

[0017] The point cloud is matched with the 3D matching box to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0018] In the above embodiment, the point cloud testing device matches the point cloud with the three-dimensional matching box, and can accurately calculate the positional relationship between the point cloud and the volume target, obtain a matching result, and improve the test accuracy.

[0019] In a possible implementation of the first aspect, matching the point cloud with the ground truth data to obtain a matching result set includes:

[0020] Project the true value data to obtain two-dimensional true value data;

[0021] Project the point cloud to obtain a two-dimensional point cloud;

[0022] The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0023] In the above implementation, both the ground truth data and the point cloud are processed into two-dimensional data for matching. First, matching in two dimensions can reduce the computational effort involved in matching, further improving testing efficiency. Second, in some scenarios, the evaluation of a detection device primarily focuses on its range measurement, speed measurement, and angular resolution capabilities. These capabilities are more relevant to the longitudinal and lateral data in the detection results. Therefore, performing a two-dimensional projection of the data can test the quality of the point cloud output by the detection device without significantly compromising accuracy.

[0024] In a possible implementation of the first aspect, projecting the true value data to obtain two-dimensional true value data includes:

[0025] Project the true value data onto the horizontal plane to obtain two-dimensional true value data;

[0026] Project the point cloud to obtain a two-dimensional point cloud, including:

[0027] Project the point cloud onto the horizontal plane to obtain a two-dimensional point cloud.

[0028] In the above embodiment, the point cloud and the true value data can be projected onto a horizontal plane during projection. Projection onto a horizontal plane can largely preserve the horizontal and vertical data of the volume target, which is beneficial to improving test efficiency and saving computational effort.

[0029] The horizontal plane refers to a relatively horizontal plane, such as the XY plane.

[0030] Take the projection of a point cloud as an example. For example, the point cloud contains multiple sampling points, each of which corresponds to a three-dimensional coordinate (taking the Cartesian coordinate system as an example). For any of the sampling points, the vertical data can be discarded during projection (or the Z-axis value is set to 0, or the vertical data is ignored), thereby obtaining a two-dimensional sampling point.

[0031] In another possible implementation of the first aspect, the true value data and the point cloud are time-aligned. Further, when the true value data and the point cloud are projected into two-dimensional data, the two-dimensional true value data and the two-dimensional point cloud are time-aligned.

[0032] For example, the ground truth data contains A frames from the first moment to the second moment, where A is an integer and A>0; the point cloud contains B frames from the third moment to the fourth moment, where B is an integer and B>0. If the two are time-aligned, for any frame in the B frames, the ground truth frame with the closest timestamp can be found.

[0033] In the above implementation, the true value data and the point cloud are aligned in time, so that the point cloud at a certain moment can find the true value frame closest in time, which can improve the accuracy of matching and thus improve the accuracy of point cloud testing.

[0034] In another possible implementation of the first aspect, the coordinates of the ground truth data and the point cloud are aligned. For example, the ground truth data and point cloud are obtained by a truth system installed on a vehicle and detection of the device under test (DUT), respectively. The origins of the ground truth data and point cloud can be converted to the center of the vehicle's rear axle. This implementation can improve matching accuracy, thereby enhancing the accuracy of point cloud testing.

[0035] In another possible implementation of the first aspect, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

[0036] Match the 2D point cloud with the 2D ground truth data to obtain a matching result set, including:

[0037] Determine at least one ground truth frame in a first ground truth frame, wherein one ground truth frame corresponds to one volume target, and the first ground truth frame belongs to multiple ground truth frames;

[0038] A matching result subset is obtained according to a range of at least one ground truth frame and positions of a plurality of sampling points in the first point cloud frame.

[0039] The first point cloud frame belongs to the plurality of point cloud frames, and the timestamp of the first point cloud frame and the first ground truth frame are the same. The matching result subset belongs to the matching result set.

[0040] In the above implementation, a matching method is introduced using the matching of the first point cloud frame as an example. During matching, the true value data is projected to obtain two-dimensional true value data, and a true value frame (or two-dimensional truth value frame) is established. The two-dimensional true value data contains data at multiple time points. For a frame of two-dimensional true value data at a certain time point, matching is performed based on the position of the truth value frame and the position of the sampling points in the two-dimensional point cloud at the same time point. In this way, the true value and point cloud at the same time point can be matched, thereby improving the test accuracy.

[0041] Understandably, the following situations may occur during matching: for a certain sampling point, it may fall into one or more true value boxes, or it may not fall into any true value box; and for a certain true value box, its range may contain one or more sampling points, or it may not contain any sampling points.

[0042] In another possible implementation of the first aspect, the at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target.

[0043] Exemplarily, when the first point cloud frame contains the first sampling point and the first sampling point falls into the first true value frame, the first sampling point belongs to the matching sampling point. Further, the first sampling point matches the true value of the first object.

[0044] Exemplarily, when the first point cloud frame includes the second sampling point and the second sampling point does not fall into any true value frame of the at least one true value frame, the second sampling point is an unmatched sampling point;

[0045] Exemplarily, when any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

[0046] In the above implementation, the classification of matching results is described using the first ground truth box, the first sampling point, and the second sampling point as examples. A matched sampling point is a sampling point that successfully matches the ground truth of the volume target. An unmatched sampling point is a point cloud that fails to successfully match the ground truth of the volume target. An unmatched ground truth value is a ground truth value that fails to successfully match any sampling point. It is not difficult to see that the number of matched point clouds is generally positively correlated with point cloud quality, while the number of unmatched point clouds and the number of unmatched ground truth values ​​are negatively correlated with point cloud quality. Therefore, the classification of matching results provides a preliminary reflection of the accuracy of the point cloud, facilitating subsequent classification testing of different categories of matching results, thereby improving the richness and accuracy of point cloud testing.

[0047] In another possible implementation of the first aspect, the number of truth boxes is greater than or equal to 2. Matching the two-dimensional point cloud with the two-dimensional truth data to obtain a matching result set further includes:

[0048] When the first point cloud frame includes the third sampling point and the third sampling point falls within at least two ground truth frames, the ground truth value of the volume object matching the third sampling point is determined according to a position between the third sampling point and the ground truth values ​​of the volume objects corresponding to the at least two ground truth frames.

[0049] The above implementation illustrates how to determine the true value of the volumetric target that a sampling point matches when it falls within multiple ground truth boxes. This clarifies the matching relationship between the point cloud and the ground truth, improving the accuracy of point cloud testing.

[0050] In another possible implementation of the first aspect, determining the true value of the volume target matching the third sampling point based on a position between the third sampling point and at least two ground truth boxes includes:

[0051] Establish point pairs between the true values ​​of the volume targets corresponding to at least two truth frames and the third sampling point, construct a distance matrix based on the point pairs, obtain the distance between the true value and the third sampling point, and take the true value with the closest distance as the true value matching the third sampling point.

[0052] By constructing a distance matrix, we can more accurately determine the distance relationship between the sampling points and the true value, determine the true value of the volume target matching the sampling point, and improve the accuracy of point cloud testing.

[0053] In another possible implementation of the first aspect, obtaining true value data and a point cloud includes:

[0054] Preprocess the initial ground truth and initial point cloud to obtain ground truth data and a point cloud. Preprocessing can include one or more of the following: time alignment (or timestamp alignment), coordinate conversion, and format conversion. Preprocessing can improve the correspondence between ground truth data and the point cloud, reduce matching complexity, and improve point cloud testing efficiency.

[0055] In another possible implementation of the first aspect, test items such as accuracy, false alarms, and missed detections of the point cloud are evaluated through a set of matching results.

[0056] In another possible implementation of the first aspect, the method further includes:

[0057] According to the matching sampling points in the matching result set, the accuracy evaluation data about the DUT is obtained, wherein the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy, and height accuracy.

[0058] In another possible implementation of the first aspect, obtaining accuracy assessment data about the DUT according to the matching sampling points in the matching result set includes:

[0059] The number of matching sampling points on the fourth body target is obtained according to the number of sampling points that match the true value of the fourth body target among the matching sampling points.

[0060] In another possible implementation of the first aspect, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0.

[0061] The distance measurement accuracy of the DUT when detecting a second target can be evaluated through N sampling points and M corner points.

[0062] In another possible implementation of the first aspect, the N sampling points include a closest sampling point, and the M corner points include a closest transverse corner point. "Closest" refers to the distance between the point and a reference point or reference device. For example, the reference point may be the point where the DUT is located. In this case, the closest sampling point is the sampling point closest to the DUT among the N sampling points, and the closest transverse corner point is the corner point closest to the DUT among the M corner points.

[0063] Furthermore, the ranging accuracy includes a lateral ranging accuracy with respect to the second object. The lateral ranging accuracy with respect to the second object is related to the lateral distance between the nearest sampling point and the DUT, and the lateral distance between the nearest lateral corner point and the DUT.

[0064] It is understandable that the above description uses the DUT as a reference point. In the specific implementation process, the DUT can also be replaced by a vehicle, a vehicle rear axle center, or a true value system.

[0065] In another possible implementation of the first aspect, the lateral ranging accuracy σ of the second target is x Satisfies the following formula:

[0066] σ x =|X pi -X cj |

[0067] Among them, X pi is the lateral distance between the nearest sampling point and the DUT, X cj is the lateral distance between the nearest lateral corner and the DUT.

[0068] In another possible implementation of the first aspect, the N sampling points include a nearest sampling point, and the M corner points include a radially nearest corner point. "Nearest" refers to the distance between the point and a preset point. The preset vertex may be, for example, a DUT. In this case, the nearest sampling point is the sampling point closest to the DUT among the N sampling points, and the radially nearest corner point is the corner point closest to the DUT in radial distance among the M corner points.

[0069] Furthermore, the ranging accuracy includes a longitudinal ranging accuracy with respect to the second object. The longitudinal ranging accuracy with respect to the second object is related to the radial distance between the nearest sampling point and the DUT and the radial distance between the radial nearest corner point and the DUT.

[0070] In another possible implementation of the first aspect, the longitudinal distance measurement accuracy σ of the second target d Satisfies the following formula:

[0071] σ d =|D pi -Dck |

[0072] Among them, D pi is the radial distance between the nearest sampling point and the DUT, D ck is the radial distance between the closest radial corner and the DUT.

[0073] In another possible implementation of the first aspect, the velocity measurement accuracy includes velocity measurement accuracy with respect to a third-body target;

[0074] The matching sampling points include K sampling points that match the true value of the third-body target, where K is an integer and K>0;

[0075] The K sampling points include the strongest sampling point. The velocity measurement accuracy of the third-body target is related to the radial velocity of the strongest sampling point and the true radial velocity of the third-body target.

[0076] The above embodiment describes a method for determining the speed measurement accuracy. v It can be indicated by the absolute value of the radial velocity error between the strongest point in the matching point and the reference true value. For example, the velocity accuracy σ v Satisfies the following formula:

[0077] σ v =|V pi -V t |

[0078] Among them, V pi is the radial velocity of the strongest sampling point, V t is the true radial velocity of the third body target.

[0079] As a possible implementation, the strongest sampling point is the sampling point with the strongest radar cross section (RCS) among the K sampling points, and the number of sampling points that match the true value is K. The RCS of the K sampling points are expressed as R p1 ,R p2 ,R p3 ,…,R pK The RCS of the strongest sampling point can be expressed as R pi , which can satisfy the following formula:

[0080] R pi =max(R p1 ,R p2 ,R p3 ,…,R pK )

[0081] Optionally, the true radial velocity of the third-body target may be replaced by the radial velocity of the third-body target.

[0082] In another possible implementation of the first aspect, unmatched sampling points can be used to determine false alarms. A false alarm occurs when, under certain circumstances, a target is not present but the detection device determines that a target is present and outputs a point cloud. A false alarm can correspond to a point cloud that does not match the true value. When determining a false alarm, unmatched sampling points in a point cloud frame can be tracked, and a multi-frame correlation method can be used to determine whether a false alarm exists in the point cloud. This allows tracking of unmatched sampling points in subsequent point cloud frames, reducing false alarm errors caused by point cloud flicker, improving the accuracy of false alarm determination, and enhancing the accuracy of point cloud testing.

[0083] In another possible implementation of the first aspect, the two-dimensional point cloud includes multiple continuous point cloud frames, the matching set includes unmatched sampling points, and the method further includes:

[0084] False alarm targets in the multiple continuous point cloud frames are determined according to sampling points in the multiple continuous point cloud frames among the unmatched sampling points.

[0085] In another possible implementation of the first aspect, the plurality of consecutive point cloud frames include the second point cloud frame and Q point cloud frames following the second point cloud frame, where Q is an integer and Q>0;

[0086] Determine the false alarm targets in the point cloud based on the sampling points in multiple consecutive point cloud frames among the unmatched sampling points, including:

[0087] Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

[0088] assigning an initial life value to a first point cloud cluster in the at least one point cloud cluster;

[0089] Determine the point cloud clusters in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames;

[0090] False alarm targets in a plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

[0091] Here, Q can be a fixed number or a non-fixed number.

[0092] In the above embodiment, the testing device clusters the unmatched point cloud to obtain multiple point cloud clusters, and each point cloud cluster is assigned an initial life value. For a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with multiple subsequent point cloud frames. If there is a matching point cloud cluster in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; and the matching of multiple point cloud frames is repeated in this way. If the life value of the point cloud cluster reaches the first threshold, the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster reaches the second threshold or is lower than the third threshold, the point cloud cluster does not form a false alarm target and can be optionally discarded. Since a single sampling point is prone to flickering, the matching complexity is high and the reliability of the result is low, the above embodiment clusters the unmatched sampling points and realizes the tracking of the unmatched point cloud in the form of point cloud clusters, which not only reduces the complexity of matching, but also greatly improves the reliability and availability of false alarm judgment, and improves the accuracy of point cloud testing.

[0093] It should be understood that reaching the first threshold may be higher than or equal to the first threshold, which is subject to specific design. The same applies to the second threshold and the third threshold.

[0094] In some scenarios, when matching point cloud clusters, the cluster bounding box is determined based on the cluster size within the point cloud frame. This bounding box should encompass the points within the cluster. For the current point cloud frame, if the cluster bounding box in the next point cloud frame overlaps with the current frame's, the overlap matrix between the two cluster bounding boxes is calculated to establish an association between the two frames. If the cluster bounding box in the next point cloud frame successfully matches the current frame's cluster bounding box, the cluster's health is increased; otherwise, the cluster's health is decreased.

[0095] By establishing associations between frames through matching frames, the computational complexity can be further reduced and the efficiency of point cloud testing can be improved.

[0096] In yet another possible implementation of the first aspect, the unmatched point cloud may also be used to determine a false alarm rate.

[0097] Exemplarily, the point cloud testing device determines the false alarm rate based on the number of point cloud frames involved in the warning target calculation and the number of false alarm point cloud frames. A false alarm point cloud frame is a point cloud frame containing a false alarm target, or a false alarm point cloud frame is a point cloud frame containing at least one point cloud cluster whose health value reaches a first threshold.

[0098] For example, the false alarm rate ρ satisfies the following formula:

[0099]

[0100] Among them, n is the number of point cloud frames involved in the calculation of false alarm targets, n false is the number of frames with false alarm targets.

[0101] In another possible implementation of the first aspect, the unmatched true value can be used to determine missed detections. A missed detection refers to an event in which a target is present but the radar determines that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

[0102] Optionally, during missed detection determination, it is possible to determine whether the volumetric object is occluded. If the volumetric object is occluded, a missed detection of that volumetric object is not considered a valid missed detection. This can reduce missed detection errors caused by volumetric object occlusion, improve the accuracy of missed detection determination, and enhance the accuracy of point cloud testing.

[0103] In another possible implementation of the first aspect, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

[0104] The method also includes:

[0105] Determine the suspected missed detection target in the third point cloud frame according to the unmatched true value corresponding to the third point cloud frame;

[0106] determining an obscured volume target based on a field of view relationship between at least one volume target and the radar;

[0107] The occluded volume targets in the suspected missed detection targets in the third point cloud frame are filtered out to determine the missed detection targets contained in the third point cloud frame.

[0108] In the above embodiment, the testing device determines suspected missed detection targets based on the unmatched true value, and removes the obscured body targets from the suspected missed detection targets based on the occlusion relationship, thereby reducing missed detection judgment errors caused by body target occlusion.

[0109] In another possible implementation of the first aspect, determining the obscured volume target based on a field of view relationship between at least one volume target and a radar includes:

[0110] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0111] For example, if there are V occluded corner points among the multiple corner points of the fifth body target, then the fifth body target is occluded, V is an integer and V>0, where the occluded corner points are corner points where the lines connecting the corner points intersect with the edges of other body targets.

[0112] For another example, if the fifth body target is an occluding target, it satisfies the following two conditions: ① V corner points of the fifth body target intersect with the DUT, V is an integer and V>0; ② The number of valid edges in the fifth body target is greater than or equal to the fourth threshold. The fourth threshold can be predefined or pre-set. For example, the fourth threshold can be 4, or the fourth threshold is 1. The valid edge can be determined as follows: for any corner point or any edge corner point (edge ​​corner point refers to a corner point located on an edge) in the fifth body target, if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, then the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner points on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, then the first edge is a valid edge.

[0113] In another possible implementation of the first aspect, during missed detection determination, areas corresponding to unmatched ground truth values ​​can be tracked within point cloud frames, and multi-frame correlation can be used to determine whether a volumetric target has been missed. This allows tracking of unmatched ground truth values ​​in subsequent point cloud frames, reducing missed detection errors caused by point cloud flicker, improving the accuracy of false alarm determination, and enhancing the accuracy of point cloud testing.

[0114] For example, when there is a missed detection of a certain object in three consecutive point cloud frames, the object is determined to be a missed detection object.

[0115] In yet another possible implementation of the first aspect, the unmatched true value may also be used to determine a missed detection rate.

[0116] Exemplarily, the point cloud testing device determines the missed detection rate based on the number of point cloud frames involved in the early warning target calculation and the number of missed detection point cloud frames, wherein the missed detection point cloud frames are point cloud frames in which missed detection targets exist (or are determined to have missed detection targets).

[0117] For example, the missed detection rate γ satisfies the following formula:

[0118]

[0119] Among them, n is the number of point cloud frames involved in the calculation of missed targets, and n_lose is the number of missed point cloud frames.

[0120] In a second aspect, an embodiment of the present application provides a point cloud testing device, which includes a data acquisition module and a data matching module, wherein:

[0121] The data acquisition module is used to obtain true value data and point cloud. The true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT on the volume target. The detection result includes sampling points.

[0122] The data matching module is used to match the point cloud and the true value data to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0123] Optionally, the number of volume targets may be one or more. In order to facilitate the description of the solution of the present application, the number of volume targets is described as at least one below.

[0124] In yet another possible implementation of the second aspect, the data matching module is configured to:

[0125] Establish a three-dimensional matching box based on the ground truth data;

[0126] The point cloud is matched with the 3D matching box to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0127] In a possible implementation of the second aspect, the data matching module is configured to:

[0128] Project the true value data to obtain two-dimensional true value data;

[0129] Project the point cloud to obtain a two-dimensional point cloud;

[0130] The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0131] In a possible implementation of the second aspect, the data matching module is configured to:

[0132] Project the true value data onto the horizontal plane to obtain two-dimensional true value data;

[0133] Project the point cloud to obtain a two-dimensional point cloud, including:

[0134] Project the point cloud onto the horizontal plane to obtain a two-dimensional point cloud.

[0135] In another possible implementation of the second aspect, the time of the true value data and the point cloud is aligned. Further, when the true value data and the point cloud are projected into two-dimensional data, the time of the two-dimensional true value data and the two-dimensional point cloud is aligned.

[0136] In yet another possible implementation of the second aspect, the coordinates of the true value data and the point cloud are aligned.

[0137] In another possible implementation of the second aspect, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

[0138] The data matching module is also used to:

[0139] Determine at least one ground truth frame in a first ground truth frame, wherein one ground truth frame corresponds to one volume target, and the first ground truth frame belongs to multiple ground truth frames;

[0140] A matching result subset is obtained based on a range of at least one true value frame and positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.

[0141] In another possible implementation of the second aspect, the at least one truth box includes a first truth box corresponding to a first volume object, and the first volume object belongs to the at least one volume object;

[0142] When the first point cloud frame includes the first sampling point and the first sampling point falls within the first ground truth frame, the first sampling point is a matching sampling point, and the first sampling point matches the ground truth of the first object.

[0143] In a case where the first point cloud frame includes the second sampling point and the second sampling point does not fall into any of the truth value frames in the at least one truth value frame, the second sampling point is an unmatched sampling point;

[0144] When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

[0145] In another possible implementation of the second aspect, the number of at least one truth box is greater than or equal to 2.

[0146] The data matching module is also used to:

[0147] When the first point cloud frame includes the third sampling point and the third sampling point falls within at least two ground truth frames, the ground truth value of the volume object matching the third sampling point is determined according to a position between the third sampling point and the ground truth values ​​of the volume objects corresponding to the at least two ground truth frames.

[0148] In yet another possible implementation of the second aspect, the data matching module is further configured to:

[0149] Establish point pairs between the true values ​​of the volume targets corresponding to at least two truth frames and the third sampling point, construct a distance matrix based on the point pairs, obtain the distance between the true value and the third sampling point, and take the true value with the closest distance as the true value matching the third sampling point.

[0150] In yet another possible implementation of the second aspect, the data acquisition module is further configured to:

[0151] Preprocess the initial ground truth and initial point cloud to obtain ground truth data and a point cloud. This preprocessing can include one or more of the following: time alignment, coordinate conversion, and format conversion. Preprocessing can improve the correspondence between the ground truth data and the point cloud, reduce matching complexity, and increase point cloud testing efficiency.

[0152] In another possible implementation of the second aspect, the point cloud testing device further includes a data calculation module, which is used to evaluate the accuracy, false alarms, and missed detections of the point cloud through a set of matching results.

[0153] In another possible implementation of the second aspect, the point cloud testing apparatus further includes a data calculation module configured to obtain accuracy assessment data regarding the DUT based on the matching sampling points in the matching result set. The accuracy assessment data includes one or more of the following: the number of matching sampling points, ranging accuracy, velocity accuracy, and height accuracy.

[0154] In yet another possible implementation of the second aspect, the data calculation module is further configured to:

[0155] The number of matching sampling points on the fourth body target is obtained according to the number of sampling points that match the true value of the fourth body target among the matching sampling points.

[0156] In another possible implementation of the second aspect, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0.

[0157] The data calculation module is further used to evaluate the ranging accuracy of the DUT when detecting the second target based on the N sampling points and the M corner points.

[0158] In another possible implementation of the second aspect, the N sampling points include a nearest sampling point, and the M corner points include a lateral nearest corner point. Furthermore, the ranging accuracy includes a lateral ranging accuracy with respect to the second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT.

[0159] In another possible implementation of the second aspect, the lateral ranging accuracy σ of the second target is x Satisfies the following formula:

[0160] σ x =|X pi -X cj |

[0161] Among them, X pi is the lateral distance between the nearest sampling point and the DUT, Xcj is the lateral distance between the nearest lateral corner and the DUT.

[0162] In another possible implementation of the second aspect, the N sampling points include a nearest sampling point, and the M corner points include a radially nearest corner point. Furthermore, the ranging accuracy includes a longitudinal ranging accuracy with respect to the second object, and the longitudinal ranging accuracy with respect to the second object is related to the radial distance between the nearest sampling point and the DUT, and the radial distance between the radially nearest corner point and the DUT.

[0163] In another possible implementation of the second aspect, the longitudinal ranging accuracy σ of the fourth target is d Satisfies the following formula:

[0164] σ d =|D pi -D ck |

[0165] Among them, D pi is the radial distance between the nearest sampling point and the DUT, D ck is the radial distance between the closest radial corner and the DUT.

[0166] In another possible implementation of the second aspect, the velocity measurement accuracy includes velocity measurement accuracy with respect to a third-body target;

[0167] The matching sampling points include K sampling points that match the true value of the third-body target, where K is an integer and K>0;

[0168] The K sampling points include the strongest sampling point. The velocity measurement accuracy of the third-body target is related to the radial velocity of the strongest sampling point and the true radial velocity of the third-body target.

[0169] The above embodiment describes a method for determining the speed measurement accuracy. v It can be indicated by the absolute value of the radial velocity error between the strongest point in the matching point and the reference true value. For example, the velocity accuracy σ v Satisfies the following formula:

[0170] σ v =|V pi -V t |

[0171] Among them, V pi is the radial velocity of the strongest sampling point, V t is the true radial velocity of the third body target.

[0172] As a possible implementation, the strongest sampling point is the sampling point with the strongest radar cross section (RCS) among the K sampling points, and the number of sampling points that match the true value is K. The RCS of the K sampling points are expressed as R p1 ,R p2 ,R p3 ,…,R pK The RCS of the strongest sampling point can be expressed as R pi , which can satisfy the following formula:

[0173] R pi =max(R p1 ,R p2 ,R p3 ,…,R pK )

[0174] Optionally, the true radial velocity of the third-body target may be replaced by the radial velocity of the third-body target.

[0175] In another possible implementation of the second aspect, unmatched sampling points may be used for false alarm judgment.

[0176] In another possible implementation of the second aspect, the point cloud testing device further includes a data calculation module, which is further configured to:

[0177] False alarm targets in the multiple continuous point cloud frames are determined according to sampling points in the multiple continuous point cloud frames among the unmatched sampling points.

[0178] In another possible implementation of the second aspect, the plurality of consecutive point cloud frames include the second point cloud frame and Q point cloud frames following the second point cloud frame, where Q is an integer and Q>0;

[0179] The data calculation module is also used to:

[0180] Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

[0181] assigning an initial life value to a first point cloud cluster in the at least one point cloud cluster;

[0182] Determine the point cloud clusters in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames;

[0183] False alarm targets in a plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

[0184] In the above embodiment, the testing device clusters the unmatched point cloud to obtain multiple point cloud clusters, and each point cloud cluster is assigned an initial life value. For a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with multiple subsequent point cloud frames. If there is a matching point cloud cluster in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; the matching of multiple point cloud frames is repeated in this way. If the life value of the point cloud cluster reaches the first threshold, the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster reaches the second threshold or is lower than the third threshold, the point cloud cluster does not form a false alarm target, and the point cloud cluster can be optionally discarded.

[0185] In some scenarios, when matching point cloud clusters, the cluster bounding box is determined based on the cluster size within the point cloud frame. This bounding box should encompass the points within the cluster. For the current point cloud frame, if the cluster bounding box in the next point cloud frame overlaps with the current frame's, the overlap matrix between the two cluster bounding boxes is calculated to establish an association between the two frames. If the cluster bounding box in the next point cloud frame successfully matches the current frame's cluster bounding box, the cluster's health is increased; otherwise, the cluster's health is decreased.

[0186] In yet another possible implementation of the second aspect, the unmatched point cloud may also be used to determine a false alarm rate.

[0187] Exemplarily, the point cloud testing device determines the false alarm rate based on the number of point cloud frames involved in the warning target calculation and the number of false alarm point cloud frames. A false alarm point cloud frame is a point cloud frame containing a false alarm target, or a false alarm point cloud frame is a point cloud frame containing at least one point cloud cluster whose health value reaches a first threshold.

[0188] For example, the false alarm rate ρ satisfies the following formula:

[0189]

[0190] Among them, n is the number of point cloud frames involved in the calculation of false alarm targets, n false is the number of frames with false alarm targets.

[0191] In another possible implementation of the second aspect, the unmatched true value can be used to determine missed detections. A missed detection refers to an event in which a target is present but the radar determines that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

[0192] In another possible implementation of the second aspect, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

[0193] The data calculation module is also used to:

[0194] Determine the suspected missed detection target in the third point cloud frame according to the unmatched true value corresponding to the third point cloud frame;

[0195] determining an obscured volume target based on a field of view relationship between at least one volume target and the radar;

[0196] The occluded volume targets in the suspected missed detection targets in the third point cloud frame are filtered out to determine the missed detection targets contained in the third point cloud frame.

[0197] In another possible implementation of the second aspect, the data calculation module is further configured to:

[0198] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0199] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0200] For example, if there are V occluded corner points among the multiple corner points of the fifth body target, then the fifth body target is occluded, V is an integer and V>0, where the occluded corner points are corner points where the lines connecting the corner points intersect with the edges of other body targets.

[0201] For another example, if the fifth body target is an occluding target, it satisfies the following two conditions: ① The lines connecting the V corner points of the fifth body target and the DUT intersect with the edges of other body targets, V is an integer and V>0; ② The number of valid edges in the fifth body target is greater than or equal to the fourth threshold. The fourth threshold can be predefined or pre-set. For example, the fourth threshold can be 4, or the fourth threshold is 1. The valid edge can be determined as follows: for any corner point or any edge corner point (edge ​​corner point refers to a corner point located on an edge) in the fifth body target, if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, then the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner points on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, then the first edge is a valid edge.

[0202] In another possible implementation of the second aspect, when judging missed detection, the area corresponding to the unmatched true value can be tracked in the point cloud frame, and a multi-frame association method can be used to determine whether the volume target is missed.

[0203] For example, when there is a missed detection of a certain object in three consecutive point cloud frames, the object is determined to be a missed detection object.

[0204] In yet another possible implementation of the second aspect, the unmatched true value may also be used to determine a missed detection rate.

[0205] Exemplarily, the point cloud testing device determines the missed detection rate based on the number of point cloud frames involved in the early warning target calculation and the number of missed detection point cloud frames, wherein the missed detection point cloud frames are point cloud frames in which missed detection targets exist (or are determined to have missed detection targets).

[0206] For example, the missed detection rate γ satisfies the following formula:

[0207]

[0208] Among them, n is the number of point cloud frames involved in the calculation of missed targets, and n_lose is the number of missed point cloud frames.

[0209] In a third aspect, an embodiment of the present application provides a chip comprising a processor. When the processor invokes a computer program or instruction, the method described in any one of the first aspects is executed. That is, the processor is configured to implement the method described in any one of the first aspects.

[0210] Optionally, the chip further includes a communication interface, where the communication interface is used to receive and / or send data, and / or the communication interface is used to provide input and / or output for the processor.

[0211] Optionally, the chip may further include a memory, which may be used to store computer programs or instructions. Furthermore, the memory may be located outside the processor, or may be integrated with the memory.

[0212] In a fourth aspect, an embodiment of the present application provides a computing device comprising a processor; when the processor calls a computer program or instruction in a memory, the method described in any one of the first aspects is executed.

[0213] Optionally, the computing device further includes a communication interface, where the communication interface is used to receive and / or send data, and / or the communication interface is used to provide input and / or output for the processor.

[0214] It should be noted that the above embodiments are described using a processor (or general-purpose processor) that executes a method by calling a computer instruction. In specific implementations, the processor may also be a dedicated processor, in which case the computer instructions are pre-loaded into the processor. Alternatively, the processor may include both a dedicated processor and a general-purpose processor.

[0215] Optionally, the computing device may further include a memory, which may be used to store computer programs or instructions. Furthermore, the memory may be located outside the processor, or may be integrated with the memory.

[0216] In the fifth aspect, an embodiment of the present application provides a point cloud testing system, which includes a data preprocessing module, a data matching module and a data calculation module. The point cloud test is used to implement any method described in the first aspect.

[0217] Furthermore, the point cloud testing system also includes a data statistics module, which is used to count the matching results.

[0218] Furthermore, the point cloud testing system also includes a data storage module, which is used to store the initial true value data and the initial point cloud. Furthermore, it is also used to store the true value data, point cloud, matching result set, test item results, etc.

[0219] Furthermore, the point cloud testing system also includes a test vehicle, on which the truth system and the DUT are mounted. The truth system is used to collect initial truth data, and the DUT is used to collect the initial point cloud. Alternatively, the test vehicle can be replaced with a mobile terminal, such as a drone, robot, or other transportation tool, or an intelligent terminal.

[0220] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions or computer programs. When the instructions or computer programs are executed, the method described in any one of the first aspects above is implemented.

[0221] In a seventh aspect, the present application provides a computer program product, the computer program product including computer instructions or a computer program,

[0222] When the instruction or computer program is executed, the method described in any one of the first aspects above is implemented.

[0223] Optionally, the computer program product may be a software installation package or an image package. When the aforementioned method is required, the computer program product may be downloaded and executed on a computing device.

[0224] The beneficial effects of the technical solutions provided in the second to seventh aspects of this application can refer to the beneficial effects of the technical solution in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0225] The following is a brief introduction to the drawings used in describing the embodiments.

[0226] FIG1 is a schematic diagram of a point cloud testing technology based on a point target;

[0227] FIG2 is a schematic diagram of a scene for collecting the true value of a volume target and a point cloud of the volume target provided by an embodiment of the present application;

[0228] FIG3 is a schematic diagram of the architecture of a point cloud testing system provided in an embodiment of the present application;

[0229] FIG4 is a schematic diagram of a flow chart of a point cloud testing method provided in an embodiment of the present application;

[0230] FIG5 is a schematic diagram of ground truth data and point cloud provided by an embodiment of the present application;

[0231] FIG6 is a schematic diagram of two-dimensional true value data provided by an embodiment of the present application;

[0232] FIG7 is a schematic diagram of a two-dimensional point cloud provided in an embodiment of the present application;

[0233] FIG8 is a schematic diagram of a point cloud frame and a true value frame provided in an embodiment of the present application;

[0234] FIG9 is a schematic diagram of a truth value box provided in an embodiment of the present application;

[0235] FIG10 is a schematic diagram of a matching result provided in an embodiment of the present application;

[0236] FIG11 is a flow chart of another point cloud testing method provided in an embodiment of the present application;

[0237] FIG12 is a schematic diagram of a three-dimensional truth box provided in an embodiment of the present application;

[0238] FIG13 is a schematic diagram of a matching result provided in an embodiment of the present application;

[0239] FIG14 is a schematic diagram of a possible number of matching sampling points provided by an embodiment of the present application;

[0240] FIG15 is a schematic diagram of the distance between a sampling point and a DUT provided in an embodiment of the present application;

[0241] FIG16 is a schematic diagram of the distance between another sampling point and the DUT provided in an embodiment of the present application;

[0242] FIG17 is a schematic diagram of a possible unmatched point cloud provided by an embodiment of the present application;

[0243] FIG18 is a flow chart of another false alarm determination method provided in an embodiment of the present application;

[0244] FIG19 is a schematic diagram of the position of a body target provided in an embodiment of the present application;

[0245] FIG20 is a schematic structural diagram of a point cloud testing device provided in an embodiment of the present application;

[0246] Figure 21 is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0247] The embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0248] For ease of understanding, the following examples provide some explanations of concepts related to the embodiments of the present application for reference.

[0249] 1. Detection Device: Detection devices capable of outputting point clouds include, but are not limited to, radar or lidar. Radars may include millimeter-wave radars, centimeter-wave radars, and the like. In some scenarios, devices that integrate both radar and cameras (fused detection devices) can also output point clouds, and such fused detection devices also fall within the scope of the detection devices of this application.

[0250] 2. Field of view: The area between the transmitter of the detection device and the target, and / or between the receiver of the detection device and the target, where there must be uninterrupted line of sight (LOS) for signal (e.g., radio waves, laser) transmission. This line of sight can be understood as the field of view.

[0251] 3. Device under test (DUT): The detection device being tested.

[0252] 4. True Value: The true value refers to the actual value of the measured object under certain time and space (or position or state) conditions. The true value is the true value of a variable itself and is usually an ideal concept. In the embodiments of the present application, the true value can be a reference true value.

[0253] 5. Volume Target: A volume target can be considered an object with at least two of the following: length, height, and width. It serves as the target for testing the DUT. When the DUT detects a volume target, multiple sampling points are typically obtained.

[0254] Exemplarily, the body target includes but is not limited to one or more of a sphere, a cuboid (including a cube), a flat plate, a cylinder, or a round top hat. It is not difficult to see that when the detection device detects the body target, it can detect different surfaces of the body target at different viewing angles. In some scenarios, during the process of the detection device detecting the body target, the body target has characteristics such as posture and size, which makes it closer to objects in production and living environments (including living organisms).

[0255] The above description of technical terms may be optionally used in the following embodiments.

[0256] The quality of the point cloud represents the detection capability of the detection device. Currently, the testing of the point cloud output by the detection device (hereinafter referred to as point cloud testing) has always been a key test item for the detection device in the industry.

[0257] Figure 1 shows a schematic diagram of point cloud testing technology based on point targets. The radar under test (exemplary DUT) is placed on a rotatable turntable within a microwave anechoic chamber (the floor, walls, and ceiling are all covered with absorbing materials). The chamber also houses a radar target simulator, which can simulate point targets at various distances and speeds. The radar under test detects the point target (the double-headed arrows in Figure 1 represent the radar signal emission and its echo) to evaluate its range and accuracy.

[0258] As shown in Figure 1, point cloud testing technology based on point targets has matured. However, point targets differ significantly from real-world targets. Real-world targets can be represented by volumetric targets, and detection devices output multiple sampling points when detecting volumetric targets. Currently, automated testing solutions for volumetric point clouds are largely unavailable. The industry urgently needs to test the point clouds generated by the device under test (DUT) for volumetric targets.

[0259] In light of this, embodiments of the present application provide a point cloud testing method and apparatus. This embodiment matches the true value of a volumetric target with its point cloud, determining the match between the point cloud of the volumetric target output by the device under test (DUT) and the true value of the volumetric target. Because volumetric targets are closer to actual targets, embodiments of the present application can more accurately test the quality of the point cloud output by a detection device, facilitating the evaluation of the device's detection capabilities.

[0260] The following is an example of a method for obtaining the true value of a volume target and a point cloud of the volume target.

[0261] Please refer to Figure 2, which is a schematic diagram of a scene for collecting the true value of a volume target and the point cloud of a volume target provided in an embodiment of the present application, and the device to be tested is loaded on a vehicle. The vehicle is placed in an environmental test field, and further, the vehicle can travel in the environmental test field. One or more volume targets are also set in the environmental test field, such as volume target T1, volume target T2, volume target T3 and volume target T4 shown in Figure 2. The device to be tested can collect an initial point cloud (or original point cloud), which includes a point cloud obtained by detecting the aforementioned volume target.

[0262] In one embodiment, the vehicle further includes a truth system capable of collecting initial truth data, the truth accuracy of which meets a preset accuracy requirement. For example, the truth system includes a laser radar or a camera capable of obtaining depth data.

[0263] Of course, the aforementioned vehicle is an exemplary device for carrying a detection device, which can be replaced by other mounting platforms, or moving devices, such as logistics robots, drones and other means of transportation.

[0264] The following is a schematic diagram of the architecture of the point cloud testing system of this application. It should be noted that the system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided by this application. As the system architecture evolves and new business scenarios emerge, the technical solutions provided by this application will also be applicable to similar technical problems.

[0265] Please refer to Figure 3, which is a schematic diagram of the architecture of a point cloud testing system provided in an embodiment of the present application. Point cloud testing system 30 includes a point cloud testing device 301. Point cloud testing device 301 has computing capabilities and can match the true value of a volume target with the point cloud of the volume target to obtain a match between the point cloud of the volume target output by the DUT and the true value of the volume target.

[0266] As shown in Figure 3, the point cloud testing device 301 includes a data matching module, which can perform the matching operations described above. Optionally, the testing device also includes one or more of a data calculation module, a data statistics module, and a visualization module. The data calculation module and the data statistics module are used to evaluate the detection capability of the device under test based on the matching results. The visualization module is used to output a test report that can indicate the detection capability of the device under test.

[0267] Optionally, the point cloud testing system 30 further includes a data preprocessing module. The data preprocessing module is capable of preprocessing the data collected by the test vehicle. The data collected by the test vehicle includes an initial point cloud and optionally also includes ground truth data (as shown in FIG2 ).

[0268] In conjunction with Figure 3 and Figure 2 , the test vehicle is equipped with a device under test (DUT) capable of detecting a volumetric target set within the test field. Optionally, the test vehicle is equipped with a data storage module in which data collected by the test vehicle can be stored. The data preprocessing module can retrieve the data collected by the test vehicle from the data storage module, for example, via communication or copying.

[0269] In some scenarios, the data pre-processing module may be included in the point cloud testing device 301. Alternatively, the data pre-processing module may be located outside the point cloud testing device 301.

[0270] In a possible implementation, the data preprocessing module may be located in a data center (DC), and the point cloud testing device 301 may obtain the preprocessed point cloud or the true value data from the DC.

[0271] In addition, the names of the devices and modules in the embodiments of the present application are only examples. During the specific implementation process, the names of the devices, modules, etc. can be replaced arbitrarily.

[0272] The following describes the method of the embodiment of the present application. Please refer to Figure 4, which is a flow chart of a point cloud testing method provided by the embodiment of the present application. Optionally, the method can be implemented based on the system shown in Figure 3.

[0273] The point cloud testing method shown in FIG4 may include one or more steps from step S401 to step S404. It should be understood that for the sake of convenience, the description here is based on the order of S401 to S404, and is not intended to limit the execution to the above order. The embodiment of the present application does not limit the order of execution, execution time, number of executions, etc. of the above one or more steps. S401 to step S404 are as follows:

[0274] Step S401: The point cloud testing device obtains true value data and point cloud.

[0275] Among them, the point cloud testing device is a device with computing capabilities, such as a server, a personal computer (PC), or an intelligent terminal. When the point cloud testing device is implemented by a server, the number of servers used to implement its functions can be one or more (such as a server cluster). In some possible solutions, the point cloud testing device can be implemented by a software functional unit. For example, the point cloud testing device can be implemented by a virtual machine, a container, a cloud, etc. Among them, a virtual machine is a computer system with complete hardware system functions simulated by software and running in an isolated environment. A container is an isolated environment obtained by packaging applications and application dependency packages. The cloud is a software platform that uses application virtualization technology to enable one or more software and applications to be developed and run in an independent virtualized environment.

[0276] Ground truth data is the true value (or reference true value) of a volumetric object. For example, ground truth data can be obtained by detecting the volumetric object using a ground truth system. Furthermore, ground truth data can be annotated by the user or corrected by an artificial intelligence program to reduce the error between the ground truth data and the actual state of the volumetric object.

[0277] A point cloud is a collection of points (i.e., sampling points), which contains one or more sampling points. A sampling point in the collection usually represents a set of data, which can indicate characteristics such as coordinates, distance, intensity, speed, reflectivity, or color.

[0278] In the scenario of performing point cloud testing on a DUT, a point cloud is the detection result obtained by the DUT detecting a volume target. For example, the DUT can transmit a detection signal and receive an echo of the detection signal, which can be processed to obtain a point cloud.

[0279] The aforementioned true value data and point cloud are both data about volumetric targets. Optionally, the number of volumetric targets can be one or more. In some embodiments, the number of volumetric targets is described as at least one. It should be understood that during the detection device's field of view, at certain moments, due to factors such as the angle or movement path of the detection device, its field of view may not cover or not fully cover the volumetric target. However, the above special circumstances do not affect the detection device's detection of the volumetric target when its field of view covers the volumetric target.

[0280] In one possible implementation, the coordinates of the ground truth data and the point cloud are aligned. For example, the ground truth data and the point cloud are obtained by a ground truth system and DUT detection, respectively, mounted on a vehicle, and the origins of the ground truth data and the point cloud can be converted to the center of the vehicle's rear axle.

[0281] Please refer to Figure 5, which is a schematic diagram of a ground truth data and point cloud provided by an embodiment of the present application. The ground truth data is shown in part (a) of Figure 5, and the point cloud is shown in part (b) of Figure 5. The coordinate axes of the two are aligned, that is, they have the same origin. Furthermore, the directions of their coordinate axes are also aligned. Coordinate axis alignment can reduce the complexity of calculations, improve the accuracy of matching, and thus improve the accuracy of point cloud testing.

[0282] As shown in Figure 5, the ground truth data is the reference true value of the volume target and can also contain multiple points. To facilitate distinction, Figure 5 represents the points in the ground truth as solid black dots and the points in the point cloud obtained by the DUT as hollow dots. Of course, this is only to facilitate the distinction between the ground truth and the point cloud under test and does not represent any difference in presentation or data content. In addition, to facilitate the identification of the outline of the volume target, the outline of the volume target is represented by a dotted line in Figure 5. In actual implementation, the outline of the volume target may not necessarily exist in the ground truth and / or point cloud.

[0283] In one possible implementation, the ground truth data and the point cloud are time-aligned. For example, the ground truth data consists of A frames from the first moment to the second moment, where A is an integer and A>0; the point cloud consists of B frames from the third moment to the fourth moment, where B is an integer and B>0. With the two time-aligned, for any frame in the B frames, the ground truth frame with the closest timestamp can be found.

[0284] Optionally, the frame rates of the ground truth data and the point cloud can be the same or different. The frame rate generally describes the number of frames per unit time, with each frame representing the data obtained by the detection device after completing a single detection of the field of view. For example, the frame rate of the point cloud can be 120 frames per second, and similarly, the frame rate of the ground truth data can also be 120 frames per second. For another example, the frame rate of the point cloud can be no less than 100 frames per second.

[0285] In one possible embodiment, the ground truth data and the point cloud are preprocessed. Exemplarily, the data preprocessing includes time alignment, coordinate conversion, or format conversion between the point cloud and the ground truth data. The preprocessed data is provided to a point cloud testing device for point cloud testing.

[0286] Optionally, the preprocessing can be performed by a point cloud testing device. For example, the point cloud testing device preprocesses the initial true value data and the initial point cloud to obtain the aforementioned true value data and point cloud.

[0287] Alternatively, preprocessing can be performed by other modules or devices. In the system shown in FIG3 , the preprocessing module provides the initial true value data and the initial point cloud to the point cloud testing device after preprocessing, and the point cloud testing device can obtain the true value data and the point cloud accordingly.

[0288] Step S402: The point cloud testing device projects the true value data to obtain two-dimensional true value data.

[0289] Since the true value data represents the true value of the volume target, and the true value of the volume target is three-dimensional, projection refers to projecting the true value data onto a plane.

[0290] As an example of projection, as shown in part (a) of Figure 5, for one of the points F1 in the true value data, its position in the Cartesian coordinate system can be expressed as F1 (x1, y1, z1). Please refer to Figure 6, which is a schematic diagram of two-dimensional true value data provided in an embodiment of the present application. The two-dimensional true value data shown in Figure 6 is obtained by projecting the true value data shown in part (a) of Figure 5. Point F1 is projected onto the XY plane to obtain point F1' (x1, y1). Optionally, during the projection process of F1, the data on its Z-axis dimension is discarded or set to a preset value (such as 0).

[0291] It is understandable that the points after projection correspond one-to-one with the points before projection. That is, the projection process does not generate new points. For a point F1' in the two-dimensional true value data, the corresponding point F1 can be found in the true value data.

[0292] In one possible embodiment, the point cloud testing device projects the true value data onto a horizontal plane. A horizontal plane refers to a relatively horizontal plane. For example, a three-dimensional Cartesian coordinate system is established based on the origin, and three mutually perpendicular axes are drawn through the origin, namely: the x-axis (horizontal axis), the y-axis (vertical axis), and the z-axis (vertical axis). The three-dimensional Cartesian coordinate system includes three planes, namely the XY plane, the YZ plane, and the XZ plane. The horizontal plane can be one of the planes, for example, the XY plane.

[0293] Optionally, the origin, X-axis, Y-axis, and Z-axis can be defined by the user or manufacturer. As one possible example, the ground truth and point cloud are obtained by, for example, a ground truth system installed on a vehicle and a DUT probe, respectively. The origin can be the center of the vehicle's rear axle, the Y-axis can be the vehicle's forward direction, and the X-axis can be the vehicle's lateral direction. Of course, these parameters can be defined in other ways during implementation.

[0294] In addition, the Cartesian coordinate system is used to list the dimensions for ease of understanding. In specific implementations, the coordinate system may also be a spherical coordinate system, a polar coordinate system, etc. This application does not strictly limit the coordinate system and projection plane used for projection.

[0295] In one possible implementation, the projected two-dimensional ground truth data may include multiple frames. For ease of description, the embodiments of this application refer to the frames included in the two-dimensional ground truth data as ground truth frames. Furthermore, since the two-dimensional ground truth data is projected from the ground truth data, it also includes multiple frames at multiple time instants, which are referred to as original ground truth frames for ease of distinction.

[0296] Step S403: The point cloud testing device projects the point cloud to obtain a two-dimensional point cloud.

[0297] As an example of projection, as shown in part (b) of Figure 5, there is a point L1 (x2, y2, z2) in the point cloud. Please refer to Figure 7, which is a schematic diagram of a two-dimensional point cloud provided in an embodiment of the present application. The two-dimensional point cloud shown in Figure 7 is obtained by projecting the point cloud shown in part (b) of Figure 5. L1 is projected onto the XY plane to obtain point L1' (x2, y2). Optionally, during the projection process of L1, the data on its Z-axis dimension is discarded or set to a preset value (such as 0).

[0298] In one possible implementation, the point cloud testing device projects the point cloud onto a horizontal plane to obtain a two-dimensional point cloud. The horizontal plane is, for example, an XY plane. For related descriptions, please refer to step S402 and will not be repeated here.

[0299] Optionally, the projected 2D point cloud can contain multiple frames. For ease of description, the embodiments of this application refer to the frames contained in the 2D point cloud as point cloud frames. Furthermore, since the 2D point cloud is obtained by projecting the point cloud, it also contains multiple frames at multiple time instants, which are referred to as original point cloud frames for ease of distinction.

[0300] In one possible implementation, the two-dimensional point cloud and the two-dimensional truth are time-aligned. Figure 8 is a schematic diagram of a possible point cloud frame and a truth frame provided in an embodiment of the present application. The two-dimensional truth data includes truth frames such as truth frame #0, truth frame #1, truth frame #2, and truth frame #3 (the number is only an example), and the two-dimensional point cloud includes point cloud frame #0, point cloud frame #1, point cloud frame #2, and point cloud frame #3 (the number is only an example). Exemplarily, the truth frame aligned with point cloud frame #0 in timestamp is truth frame #0, and similarly, the truth frame aligned with point cloud frame #1 in timestamp is truth frame #1, and so on for other cases. Of course, the above numbering is only an example and is not intended to limit the embodiments of the present application.

[0301] Optionally, if the frame rate of the 2D ground truth data is different from the frame rate of the 2D point cloud, the point cloud frames and the ground truth frames may not correspond one-to-one. In this case, the subsequent matching process can find the ground truth frame with the closest timestamp to the point cloud frame for matching.

[0302] Step S404: The point cloud testing device matches the two-dimensional point cloud with the two-dimensional true value data to obtain a matching result set.

[0303] Matching is the process of verifying whether a sample point corresponds to (or is associated with) the true value of a volume object. For a sample point in a 2D point cloud, it is matched with the 2D true value data to determine whether the sample point corresponds to (or is associated with) the true value of a volume object.

[0304] The matching result set can include one or more of the following matching results: matched sampling points, unmatched sampling points, and unmatched ground truth values. A matched sampling point is a sampling point that successfully matches the ground truth value of the volume target, an unmatched sampling point is a point cloud that does not successfully match the ground truth value of the volume target, and an unmatched ground truth value is a ground truth value that does not successfully match any sampling point.

[0305] As a possible implementation, the number of matched point clouds is generally positively correlated with point cloud quality, while the number of unmatched point clouds and the number of unmatched ground truth values ​​are negatively correlated with point cloud quality. This indicates that the matching results provide a preliminary reflection of the accuracy of the point cloud, facilitating subsequent classification testing of different matching results, thereby improving the richness and accuracy of point cloud testing.

[0306] In one possible implementation, the two-dimensional point cloud and the two-dimensional ground truth are matched sequentially by frame. Alternatively, the matching can be performed sequentially by ground truth frame, or by point cloud frame.

[0307] As an example of sequential matching according to point cloud frames, taking the point cloud frames shown in Figure 8 as an example, the point cloud testing device matches point cloud frame #0 with the true value frame (i.e., true value frame #0) at the same moment, and matches point cloud frame #1 with the true value frame (i.e., true value frame #1) at the same moment, and so on for the remaining point cloud frames. It is understandable that if the frame rate of the two-dimensional point cloud is lower than the frame rate of the two-dimensional true value data, there may be a situation where some true value frames are not matched; if the frame rate of the two-dimensional point cloud is lower than the frame rate of the two-dimensional true value data, there may be a situation where multiple point cloud frames are matched to the same true value frame. The method of matching according to point cloud frames is mainly based on point cloud frames, which can avoid the occurrence of missed point cloud frames and facilitates the testing of the number of points of the point cloud, detection accuracy, etc.

[0308] As an example of sequential matching according to the true value frames, taking the frame shown in Figure 8 as an example, the point cloud testing device matches the true value frame #0 with the point cloud frame at the same time (i.e., point cloud frame #0), and matches the true value frame #1 with the point cloud frame at the same time (i.e., point cloud frame #1), and the same applies to the remaining true value frames.

[0309] The following are two possible ways to match 2D point clouds with 2D ground truth data:

[0310] Implementation method 1: When matching, the two-dimensional true value of the volume target (i.e., the true value of the volume target after projection) is used for matching. Taking point cloud frame #0 as an example, if the sampling point in point cloud frame #0 (for ease of description, called sampling point P1) coincides with the true value (or a point in the true value) of the volume target or the distance is less than the preset matching distance threshold, then the sampling point P1 is a matched sampling point. If the sampling point in point cloud frame #0 (for ease of description, called sampling point P2) does not coincide with the true value (or a point in the true value) of any volume target, or the distance to the true value (or a point in the true value) of any volume target is greater than the preset matching distance threshold, then the sampling point P2 is an unmatched sampling point. If any sampling point in point cloud frame #0 fails to successfully match a volume target (for ease of distinction, called Target1), then Target1 is an unmatched true value (or an unmatched true value in point cloud frame #0).

[0311] In implementation method 2, the point cloud testing device creates a truth box based on the 2D ground truth data and obtains matching results based on the truth box and the sampling points. For example, the matching results are obtained based on the position of the sampling points within the bounds of the truth box. Another example is the matching results obtained based on the distance between the sampling points and the truth box. Optionally, the size of the truth box can be designed based on requirements, for example, related to the true size of the volume target.

[0312] As a possible example, taking the matching of the true value frame #0 as an example, the point cloud testing device determines the true value frame in the true value frame #0. According to the range of the true value frame and the position of the sampling point in the point cloud frame #0, a matching result subset is obtained. Among them, the point cloud frame #0 and the true value frame #0 are timestamp-aligned (or located at the same time), or the true value frame #0 is the closest true value frame to the point cloud frame #0, or the point cloud frame #0 is the closest point cloud frame to the true value frame #0. Optionally, the number of true value frames is usually the same as the number of volume targets, and one true value frame corresponds to one volume target.

[0313] Figure 9 is a schematic diagram of a ground truth frame provided in an embodiment of the present application. This ground truth frame is exemplarily a ground truth frame in ground truth frame #0. There are multiple frames, which are conveniently distinguished as ground truth frame C1, ground truth frame C2, ground truth frame C3, and ground truth frame C4. As can be seen from Figure 1, ground truth frame C1 corresponds to volume target T1, ground truth frame C2 corresponds to volume target T2, ground truth frame C3 corresponds to volume target T3, and ground truth frame C4 corresponds to volume target T4.

[0314] As a possible example, during matching, the point cloud testing device searches for sampling points within the corresponding point cloud frame that are within the true value frame. If the sampling point falls within the true value frame, the match is successful. It is understandable that the following matching results may appear during matching: a sampling point may fall within one or more true value frames, or may not fall within any true value frame; and a true value frame may contain one or more sampling points, or may not contain any sampling points.

[0315] Please refer to FIG10, which is a schematic diagram of a matching result provided by an embodiment of the present application, which shows the matching results of point cloud frame #0, including the following three categories:

[0316] Category 1, matching sampling points. For the first sampling point in point cloud frame #0, if the first sampling point falls into the first ground truth box (the first ground truth box corresponds to the first volumetric target), then the first sampling point is a matching sampling point, and the first sampling point matches the ground truth of the first volumetric target (or matches the first volumetric target). As shown in Figure 10, the first sampling point, for example, point P1, falls into ground truth box C1, i.e., it matches the ground truth of volumetric target T1 (or matches volumetric target T1); the first sampling point can also be point P3, which falls into ground truth box C2, i.e., it matches the ground truth of volumetric target T2 (or matches volumetric target T2); the first sampling point can also be point P4, which falls into ground truth box C3, i.e., it matches the ground truth of volumetric target T3 (or matches volumetric target T3).

[0317] Category 2: Unmatched sampling point. For the second sampling point in point cloud frame #0, if the second sampling point does not fall into any of the ground truth boxes, the second sampling point is considered an unmatched sampling point. As shown in Figure 10, the second sampling point, such as point P2, does not fall into any of the four ground truth boxes and is therefore considered an unmatched sampling point.

[0318] Category 3: Unmatched Ground Truth. If any sampling point in point cloud frame #3 does not fall within the first ground truth box, the ground truth corresponding to the first ground truth box is considered unmatched. As shown in Figure 10, none of the sampling points in point cloud frame #0 fall within ground truth box C4, so the ground truth of volume object T4 is considered unmatched.

[0319] The results shown in Figure 10 above are only examples. In some scenarios, more or fewer categories of results may be obtained during matching, which are not listed here one by one.

[0320] Since there may be multiple truth value frames, a sampling point may match multiple truth value frames during matching. In one possible implementation, when a sampling point (referred to as the third sampling point for ease of distinction) falls within multiple truth value frames, the true value of the volumetric target (or volumetric target) matched by the third sampling point can be determined based on the positional relationship between the third sampling point and the true value of the volumetric target. The positional relationship may include distance, inclusion, or overlap, etc.

[0321] It should be noted that the true values ​​of the sampling points and volume targets used in determining the positional relationship may be projected (i.e., belonging to a two-dimensional point cloud and two-dimensional true value data, respectively), or may be unprojected (i.e., belonging to a point cloud and true value data).

[0322] In one possible implementation, taking the first point cloud frame as an example, if the first point cloud frame includes a third sampling point and the third sampling point falls within at least two ground truth frames, the point cloud testing device determines the ground truth value of the volumetric object that matches the third sampling point based on the position between the third sampling point and the ground truth values ​​of the volumetric objects corresponding to the at least two ground truth frames. The first point cloud frame and the third sampling point are used to distinguish and represent a specific quantity and are not used to define order, importance, or other criteria.

[0323] In one possible implementation, the testing device can determine the true value of the volumetric target matched by the sampling point by establishing a point pair (or corresponding point pair) between the true value of the volumetric target corresponding to at least two truth value frames and a third sampling point. A distance matrix is ​​constructed based on the point pairs to determine the distance between the true value and the third sampling point, and the true value closest to the true value is used as the true value matched to the third sampling point. By constructing a distance matrix, the distance relationship between the sampling point and the true value can be more accurately determined, the true value of the volumetric target matched to the sampling point can be determined, and the accuracy of point cloud testing can be improved.

[0324] In one possible implementation, the point cloud testing device can also evaluate the accuracy, false alarms, missed detections, etc. of the point cloud obtained by the DUT by matching the result set. The accuracy can include one or more of the number of point clouds, ranging accuracy, speed accuracy, or height accuracy.

[0325] In the embodiment shown in Figure 5, the point cloud testing device matches the true value of the volume target with the point cloud of the volume target to obtain a set of matching results between the point cloud of the volume target output by the DUT and the true value of the volume target. Because the volume target is closer to the actual target, the embodiment of the present application can more accurately test the quality of the point cloud output by the detection device, which is conducive to evaluating the detection capability of the detection device. Moreover, by automatically comparing the true value of the volume target and the point cloud of the volume target to obtain the matching result, the evaluation error can be significantly reduced, and the test accuracy and test efficiency can be improved.

[0326] Furthermore, in the embodiment shown in Figure 5 , both the ground truth data and the point cloud are processed as two-dimensional data, and matching is performed based on this two-dimensional data. This reduces computational effort during matching and improves testing efficiency. Furthermore, the evaluation of a detection device primarily focuses on its range measurement, velocity measurement, and angular resolution capabilities, which are more relevant to the longitudinal and lateral data in the detection results. Therefore, performing a two-dimensional projection of the data allows for testing the quality of the point cloud output by the detection device without significantly compromising accuracy.

[0327] Moreover, considering that the height accuracy of the detection device in some scenarios is low, projecting the data onto the XY plane for matching can reduce the matching error caused by the low height accuracy and improve the accuracy of test items such as ranging and speed measurement.

[0328] The above describes the point cloud testing method for post-projection matching. In some possible implementations, when matching the point cloud and ground truth data, the matching can be performed directly in three dimensions without projection.

[0329] Please refer to Figure 11, which is a flow chart of another point cloud testing method provided in an embodiment of the present application. Optionally, the method can be implemented based on the system shown in Figure 3.

[0330] The point cloud testing method shown in FIG11 may include one or more steps from step S1101 to step S1103. It should be understood that for the sake of convenience, the description here is based on the order of S1101 to S1103, and is not intended to limit the execution to the above order. The embodiment of the present application does not limit the order of execution, execution time, number of executions, etc. of the above one or more steps. S1101 to step S1103 are as follows:

[0331] Step S1101: The point cloud testing device obtains true value data and point cloud. See step S401 for details.

[0332] Optionally, the ground truth data may include multiple ground truth frames (or original ground truth frames), and the point cloud may include multiple point cloud frames (or original point cloud frames). For related descriptions, please refer to the aforementioned descriptions of ground truth frames and point cloud frames, but the point cloud frames and ground truth frames in this embodiment are not projected.

[0333] The time alignment and / or coordinate alignment of the ground truth data and the point cloud are performed. For example, ground truth frame #0 corresponds to point cloud frame #0. For details, please refer to the relevant description of FIG8.

[0334] Step S1102: The point cloud testing device establishes a three-dimensional matching frame based on the true value data.

[0335] Optionally, the number of 3D truth boxes is usually the same as the number of volume targets, and one 3D truth box corresponds to one volume target.

[0336] Please refer to Figure 12, which is a schematic diagram of a three-dimensional truth box provided in an embodiment of the present application. The three-dimensional truth box is exemplarily a three-dimensional truth box in truth frame #0. There are multiple boxes, which are easily distinguished in Figure 12 as truth box D1, truth box D2, truth box D3 and truth box D4, corresponding to volume targets T1-T4 respectively.

[0337] Step S1103: The point cloud testing device matches the point cloud with the three-dimensional matching box to obtain a matching result set.

[0338] The matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0339] As a possible example, taking the matching of ground truth frame #0 as an example, the point cloud testing device determines the 3D ground truth bounding box in ground truth frame #0. Based on the range of the 3D ground truth bounding box and the positions of the sampling points in point cloud frame #0, a matching result subset is obtained. Point cloud frame #0 and ground truth frame #0 are timestamp-aligned (or located at the same time), or ground truth frame #0 is the closest ground truth frame to point cloud frame #0, or point cloud frame #0 is the closest point cloud frame to ground truth frame #0.

[0340] During matching, the point cloud testing device searches for sampling points within the ground truth box in the corresponding point cloud frame. If the sampling point falls within the ground truth box, the match is successful. Understandably, the following matching results may appear during matching: a sampling point may be contained in one or more ground truth boxes, or in any ground truth box; and a ground truth box may contain one or more sampling points, or no sampling points.

[0341] Please refer to FIG13, which is a schematic diagram of a matching result provided by an embodiment of the present application, which shows the matching results of point cloud frame #0, including the following three categories:

[0342] Category 1: Matching sampling points. For the first sampling point in point cloud frame #0, if the first sampling point is contained in the first ground truth box (the first ground truth box corresponds to the first volumetric object), then the first sampling point is a matching sampling point and the first sampling point matches the ground truth of the first volumetric object (or matches the first volumetric object). As shown in Figure 13, the first sampling point, for example, point P5, is contained in the ground truth box D1, which means it matches the ground truth of volumetric object T1 (or matches volumetric object T1).

[0343] Category 2: Unmatched sampling point. For the second sampling point in point cloud frame #0, if the second sampling point does not fall into any of the ground truth boxes, the second sampling point is considered an unmatched sampling point. As shown in Figure 13, the second sampling point, for example, point P6, does not fall into any of the four ground truth boxes and is therefore considered an unmatched sampling point.

[0344] Category 3: Unmatched Ground Truth. If no sampling point in point cloud frame #3 falls within the first ground truth box, the ground truth corresponding to the first ground truth box is considered unmatched. As shown in Figure 13, ground truth box D4 does not contain any sampling point in point cloud frame #0, so the ground truth corresponding to volume object T4 is considered unmatched.

[0345] The results shown in Figure 10 above are only examples. In some scenarios, more or fewer categories of results may be obtained during matching, which are not listed here one by one.

[0346] In a possible implementation, the point cloud testing device may also evaluate the accuracy, false alarms, missed detections, etc. of the point cloud obtained by the DUT through a set of matching results.

[0347] For related descriptions, reference may also be made to the description in step S404 , but the point cloud frame and the true value frame in this embodiment are not projected.

[0348] In the embodiment shown in Figure 11, the point cloud testing device matches the true value of the volume target with the point cloud of the volume target to obtain a set of matching results between the point cloud of the volume target output by the DUT and the true value of the volume target. Because the volume target is closer to the actual target, the embodiment of the present application can more accurately test the quality of the point cloud output by the detection device, which is conducive to evaluating the detection capability of the detection device. Moreover, by automatically comparing the true value of the volume target and the point cloud of the volume target to obtain the matching result, the evaluation error can be significantly reduced, and the test accuracy and test efficiency can be improved.

[0349] In addition, in the embodiment of the present application, the point cloud is matched with the three-dimensional matching box, which can accurately calculate the positional relationship between the point cloud and the volume target and obtain the matching result, and the point cloud test has high accuracy.

[0350] In the above embodiment, the point cloud testing device can obtain a matching result set. The following describes a possible design for evaluating the point cloud of the DUT based on some or all of the matching results in the matching result set.

[0351] As a possible design, matching sampling points can be used to perform accuracy testing on the DUT output point cloud. Specifically, the point cloud testing device obtains accuracy evaluation data about the DUT based on the matching sampling points in the matching result set. The accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, velocity accuracy, and height accuracy. The following describes several test items:

[0352] Test Item 1: Number of matching sampling points. The number of matching sampling points is the number of sampling points that match the true value of the volume target. The number of point clouds of the DUT can reflect the algorithm processing capability.

[0353] As a possible implementation, the number of matching sampling points can be calculated per target. For example, taking a fourth target as an example, the point cloud testing apparatus calculates the number of matching sampling points for the fourth target based on the number of matching sampling points that match the true value of the fourth target.

[0354] Optionally, there are multiple possible calculation methods for the number of matching sampling points for a single target. For example, calculation method 1 is to accumulate the number of sampling points in multiple point cloud frames that match the true value of the fourth-body target. For another example, calculation method 2 is to list the number of sampling points in each point cloud frame that match the true value of the fourth-body target. For another example, calculation method 3 is to calculate the average number of sampling points in multiple point cloud frames that match the true value of the fourth-body target.

[0355] Please refer to Figure 14, which is a schematic diagram of a possible number of matching sampling points provided by an embodiment of the present application. As shown in Figure 14, the truth box C1 (the truth box corresponding to the volume target T1) can match the sampling points in point cloud frame #0, point cloud frame #1, point cloud frame #2 and point cloud frame #3. In point cloud frame #0, the number of sampling points in the truth box C1 is 38; in point cloud frame #1, the number of sampling points in the truth box C1 is 39; in point cloud frame #2, the number of sampling points in the truth box C1 is 20; in point cloud frame #3, the number of sampling points in the truth box C1 is 15. When calculation method 1 is adopted, the number of matching sampling points corresponding to the volume target T1 is 112.

[0356] The above description uses the number of matching sampling points for a single volume object as an example. In specific implementations, if there are multiple volume objects, the number of matching sampling points can also be multiple. Table 1 shows the number of matching sampling points corresponding to a volume object provided in an embodiment of the present application, including the serial number, volume object ID, and the number of matching sampling points. For example, the number of matching sampling points corresponding to volume object T1 is 112, and the number of matching sampling points corresponding to volume object T2 is 287. For other cases, see Table 1. Of course, the format, attributes, and numbers in Table 1 are merely examples.

[0357] Table 1 Number of matching sampling points corresponding to volume targets

[0358] In some scenarios, the number of matching sampling points can also be calculated based on multiple targets. For example, the number of matching sampling points for each volume target can be accumulated to obtain the number of matching sampling points.

[0359] It should be understood that the above description of the number of matching sampling points in a tabular format is provided for ease of understanding only and is not intended to limit the storage, output, or transmission format of the number of matching sampling points. In specific implementations, the number of matching sampling points may also be indicated using other data formats, such as linked lists, heaps, stacks, database tables, objects, etc., which are not listed here. Similarly, the other tables in this application are merely illustrative data formats and are not intended to strictly limit the present invention.

[0360] Test Item 2: Ranging Accuracy. Ranging accuracy can include one or more of lateral, radial, and longitudinal ranging accuracy. Ranging accuracy reflects the accuracy of the DUT's position detection of physical targets. Higher accuracy facilitates back-end processing. For example, higher accuracy improves the accuracy of functions such as collision warning and traversable area identification.

[0361] As a possible implementation, ranging accuracy can be calculated on a per-target basis. For example, taking a second target as an example, the point cloud testing apparatus determines the ranging accuracy for the second target based on the sampling points that match the second target's true value and the second target's true value.

[0362] In one possible implementation, the matching sampling points include N sampling points that match the true value of the second body target, where N is an integer and N>0. The true value of the second body target includes M corner points, where M is an integer and M>0. A corner point is a point whose attributes are particularly prominent in a certain aspect. Exemplarily, a corner point can be a point located at the intersection of two sides of the body target (or a point that is close to the intersection), and / or a midpoint of an edge of the body target (or a point that is close to the midpoint). In some scenarios, the conditions for the corner points can be defined by the user or the manufacturer (for example, setting specific conditions for corner point detection). For example, the corner points can be points obtained by the Harris corner detection algorithm.

[0363] The point cloud test device can evaluate the ranging accuracy of the DUT when detecting a second body target through N sampling points and M corner points.

[0364] The following description uses the lateral ranging accuracy of a single target as an example. Please refer to Figure 15, which is a schematic diagram of the distance between a sampling point and a DUT provided in an embodiment of the present application. The N sampling points include the nearest sampling point, and the M corner points include the lateral nearest corner point. The lateral ranging accuracy of the second target is related to the lateral distance between the nearest sampling point and the DUT (i.e., Xpi shown in Figure 15) and the lateral distance between the lateral nearest corner point and the DUT (i.e., Xcj shown in Figure 15).

[0365] For example, the lateral ranging accuracy σ of the second target x Satisfies the following formula:

[0366] σ x =|X pi -X cj |

[0367] Among them, X pi is the lateral distance between the nearest sampling point and the DUT, X cj is the lateral distance between the nearest lateral corner and the DUT.

[0368] It should be understood that "nearest" refers to the distance between the point and a preset point. The above example uses the preset point being the DUT as an example, and it can be replaced by other points or other devices during the specific implementation process.

[0369] As shown in Figure 15, the closest sampling point is the sampling point that is closest to the DUT (shortest radial distance) among the N sampling points. For example, the distances between the N sampling points and the DUT can be expressed as D p1 ,D p2 ,D p3 ,…,D pN , where the distance D between the nearest sampling point and the DUT is pi Satisfies the following formula:

[0370] D pi =min(D p1 ,D p2 ,D p3 ,…,D pn )

[0371] Similarly, the lateral closest corner point is the corner point with the shortest lateral distance to the DUT among the M corner points. For example, the lateral distances between the M corner points and the DUT can be expressed as X c1 ,X c2 ,X c3 ,…,X cm , where the lateral distance between the nearest lateral corner and the DUT is X cj Satisfies the following formula:

[0372] X cj =min(X c1 ,X c2 ,X c3 ,…,X cm )

[0373] The following is an example of the mirror ranging accuracy of a single target. Please refer to Figure 16, which is a schematic diagram of the distance between another sampling point and the DUT provided in an embodiment of the present application. Among them, the N sampling points include the nearest sampling point, and the M corner points include the radial nearest corner point. The radial ranging accuracy of the second body target is related to the radial distance between the nearest sampling point and the DUT (i.e., Dpi shown in Figure 16) and the lateral distance between the lateral nearest corner point and the DUT (i.e., Dck shown in Figure 16).

[0374] For example, the longitudinal ranging accuracy of the second target σ d Satisfies the following formula:

[0375] σ d =|D pi -D ck |

[0376] Among them, D pi is the radial distance between the nearest sampling point and the DUT, D ck The radial distance between the closest corner point and the DUT. It should be understood that "closest" refers to the distance between the point and a predetermined point. The above example uses the DUT as the predetermined point for illustration. Other points or other devices can be used in the specific implementation.

[0377] Optionally, the calculation method of the nearest sampling point can refer to the above.

[0378] Optionally, the lateral closest corner point is the corner point with the shortest radial distance to the DUT among the M corner points. For example, the radial distances between the M corner points and the DUT can be expressed as Dc1 ,D c2 ,D,…,D cm , where the radial distance D between the closest radial corner and the DUT is ck Satisfies the following formula:

[0379] D ck =min(D c1 ,D c2 ,D,…,D cm )

[0380] The above description is based on the ranging accuracy of a single target. In the specific implementation process, if there are multiple targets, the ranging accuracy can also be multiple. Table 2 shows the ranging accuracy corresponding to a target provided by the embodiment of the present application, including the serial number, the ID of the target, the horizontal ranging accuracy, or the vertical ranging accuracy. For example, the horizontal ranging accuracy of the DUT with respect to the target T1 is σ x1 , the longitudinal distance measurement accuracy of DUT with respect to the target T1 is σ d1 The lateral ranging accuracy of DUT with respect to the volume target T2 is σ x2 , the longitudinal ranging accuracy of DUT with respect to the volume target T2 is σ d2 For other situations, please refer to Table 2, which will not be described here one by one. Of course, the format, attributes, numbers, etc. in Table 2 are only examples.

[0381] Table 2 Ranging accuracy of volume targets

[0382] In some scenarios, ranging accuracy can also be calculated based on multiple targets. For example, the ranging accuracy corresponding to each target can be averaged, or weighted averaged, to obtain the average ranging accuracy.

[0383] Test item 3, speed measurement accuracy. Speed ​​measurement accuracy reflects the accuracy of DUT speed detection, and speed measurement accuracy may affect the decision-making accuracy of intelligent driving functions. Intelligent driving functions include but are not limited to autonomous emergency braking (AEB), lane keeping assist (LKA), adaptive cruise control (ACC), parking assistance (PA), or lane change assist (LCA). Taking ACC as an example, ACC can determine the driving state of the vehicle (such as acceleration or deceleration) based on the speed of the vehicle in front. If the speed measurement of the vehicle in front is inaccurate, it may affect driving safety.

[0384] As a possible implementation, ranging accuracy can be calculated on a per-target basis. For example, taking the second target as an example, the point cloud testing apparatus determines the ranging accuracy for the second target based on the sampling points that match the third target's true value and the third target's true value.

[0385] In one possible implementation, the matching sampling points include K sampling points that match the true value of the third-body target, where K is an integer and K>0. The K sampling points include the strongest sampling point. Optionally, the strongest sampling point can be the sampling point with the strongest echo energy. The velocity measurement accuracy of the third-body target is related to the radial velocity of the strongest sampling point and the radial velocity of the true value of the third-body target. Since a body target may correspond to multiple points, this implementation uses the strongest sampling point to evaluate the velocity measurement error, which can improve the test accuracy of the velocity measurement accuracy.

[0386] For example, the speed accuracy σ v It can be indicated by the absolute value of the radial velocity error between the strongest point in the matching point and the reference true value. For example, the velocity accuracy σ v Satisfies the following formula:

[0387] σ v =|V pi -V t |

[0388] Among them, V pi is the radial velocity of the strongest sampling point, V t is the true radial velocity of the third body target.

[0389] As a possible implementation, the strongest sampling point is the sampling point with the strongest radar cross section (RCS) among the K sampling points, and the number of sampling points that match the true value is K. The RCS of the K sampling points are expressed as R p1 ,R p2 ,R p3 ,…,R pK The RCS of the strongest sampling point can be expressed as R pi , which can satisfy the following formula:

[0390] R pi =max(R p1 ,R p2 ,R p3 ,…,R pK )

[0391] Optionally, the true radial velocity of the aforementioned third-body target may be replaced by the radial velocity of the third-body target.

[0392] Test Item 4: Height Accuracy. Height accuracy reflects the accuracy of the height measurement of the point cloud.

[0393] In one possible implementation, the height accuracy may be indicated by the height distribution being a data distribution range and / or a normal value distribution range located in the middle 50% of the height values ​​of the matching sampling points.

[0394] The middle 50% of the data is obtained by arranging the height values ​​corresponding to the matching sampling points from smallest to largest and dividing them into four equal parts, obtaining three quartiles (values ​​at the three split points). The data between the first and third quartiles constitute the middle 50% of the data. By analyzing the distribution of the middle 50% of the data, the distribution range of the middle 50% of the data is obtained.

[0395] Furthermore, two normal values ​​are obtained by expanding the first quartile and the third quartile by 1.5 times. The height value between these two normal values ​​is the normal value data, and the distribution range of the normal value data is the normal value distribution range.

[0396] In some scenarios, determining the data distribution range can be achieved through boxplot statistics. Boxplot statistics are suitable for data that does not strictly follow a normal distribution, and the quartiles are highly resistant to outliers, allowing for relatively objective identification of outliers.

[0397] As another possible design, unmatched sampling points in the matching result set can be used to determine false alarms. A false alarm is an event in which the detection device detects a target and outputs sampling points even though the target is not present. A false alarm can correspond to a point cloud that does not match the true value.

[0398] In one possible implementation, when determining false alarms, unmatched sampling points in a point cloud frame can be tracked (or traced), and a multi-frame correlation method can be used to determine whether a false alarm exists in the point cloud. For example, a two-dimensional point cloud includes multiple consecutive point cloud frames, and a matching set includes unmatched sampling points. The point cloud testing device determines false alarm targets in the multiple consecutive point cloud frames based on the sampling points of the unmatched sampling points that are located in the multiple consecutive point cloud frames.

[0399] The following is a possible method for tracking unmatched point clouds.

[0400] If a certain point cloud cluster appears multiple times in multiple consecutive point cloud frames, a false alarm may be generated. Specifically, the point cloud testing device clusters the sampling points in the second point cloud frame among the unmatched sampling points to obtain point cloud clusters, where the number of point cloud clusters can be one or more. The point cloud testing device assigns an initial life value to the first point cloud cluster, and determines the point cloud clusters in the Q point cloud frames based on the sampling points in the Q (Q is an integer and Q>0) point cloud frames after the second point cloud frame. According to the position of the first point cloud cluster and the position of the point cloud clusters in the Q point cloud frames, the false alarm targets in multiple consecutive point cloud frames are determined. Among them, Q can be a fixed number or a non-fixed number.

[0401] In some possible implementations, for a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with one or more subsequent point cloud frames. If there is a point cloud cluster that matches it in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; repeat the matching of multiple point cloud frames. If the life value of the point cloud cluster meets the false alarm condition (for example, when it reaches the first threshold), the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster meets the false alarm cancellation condition (for example, it reaches the second threshold or is lower than the third threshold), the point cloud cluster does not form a false alarm target, and the point cloud cluster can be optionally discarded. It should be understood that reaching the first threshold can be higher than or higher than or equal to the first threshold, subject to the specific design. The same applies to the multiple thresholds and conditions below.

[0402] Please refer to Figure 17, which is a schematic diagram of a possible unmatched point cloud provided by an embodiment of the present application. The unmatched sampling points in point cloud frame #0 (which can be regarded as the second point cloud frame) can be clustered to obtain point cloud cluster G1 and point cloud cluster G2, where the health value of point cloud cluster G1 and the health value of point cloud cluster G2 are both 60 (initial health value). When the health value of the point cloud cluster is greater than or equal to 100, the false alarm condition is met. When the health value of the point cloud cluster is less than 60, the false alarm elimination condition is met.

[0403] For point cloud frame #1, which is the next point cloud frame after point cloud frame #0, the unmatched sampling points in the point cloud frame can be clustered to obtain point cloud cluster G3 and point cloud cluster G4. The point cloud testing device matches the point cloud clusters in point cloud frame #0 according to point cloud clusters G3 and G4. Exemplarily, if point cloud cluster G4 matches point cloud cluster G1, the life value of point cloud cluster G4 (i.e., G1) increases by 20, and the current life value is 80; and if point cloud cluster G2 has no matching point cloud cluster in point cloud frame #1, the life value of point cloud cluster G2 decreases by 20, and the life value of point cloud cluster G2 is 40. When the third threshold is 60, since point cloud cluster G2 is already below 60, it does not form a false alarm target and is discarded. Point cloud cluster G3 is a newly discovered point cloud cluster in point cloud frame #0 and is assigned a life value of 60 (initial life value).

[0404] For point cloud frame #2, which is the next point cloud frame after point cloud frame #1, the unmatched sampling points in the point cloud frame can be clustered to obtain point cloud cluster G5. The point cloud testing device matches the point cloud cluster G5 with the point cloud cluster in point cloud frame #1. For example, if point cloud cluster G5 matches point cloud cluster G4 (i.e., G1), the life value of point cloud cluster G5 (i.e., G1) increases by 20, and the current life value is 100. When the first threshold is 100, since the life value of point cloud cluster G5 (i.e., G1) reaches the first threshold, point cloud cluster G5 forms a false alarm target. Further, the point cloud testing device matches the point cloud cluster 5 with the point cloud cluster in point cloud frame #1. If point cloud cluster G3 has no matching point cloud cluster in point cloud frame #2, the life value of point cloud cluster G2 is reduced by 20, and the life value of point cloud cluster G3 is 40, which is already lower than 60, and does not form a false alarm target and is discarded.

[0405] Regarding point cloud frame #3, which is the point cloud frame following point cloud frame #2, the unmatched sampling points in this point cloud frame can be clustered to obtain point cloud cluster G6. The point cloud testing device matches point cloud cluster G6 with the point cloud clusters in point cloud frame #2. For example, if point cloud cluster G5 matches point cloud cluster G5 (i.e., G1), the health value of point cloud cluster G6 (i.e., G1) increases by 20. The current health value is 120, reaching the first threshold, and point cloud cluster G6 becomes a false alarm target.

[0406] As a possible implementation method, when matching point cloud clusters, a point cloud cluster frame is determined based on the size of the point cloud cluster in the point cloud frame. This point cloud cluster frame can enclose the point cloud within the cluster. For the current point cloud frame, if the point cloud cluster frame in the next point cloud frame overlaps with the point cloud cluster frame in the current frame, the overlap matrix between the point cloud cluster frame and the point cloud cluster frame is calculated, and an inter-frame association is established. If the point cloud cluster frame in the next point cloud frame successfully matches the point cloud cluster frame in the current frame, the health value of this point cloud cluster increases; otherwise, the health value of the point cloud cluster decreases. Establishing inter-frame associations by matching frames can further reduce computational complexity and improve point cloud testing efficiency.

[0407] It should be noted that the above embodiments use backward matching as an example, that is, matching the point cloud clusters in the current point cloud frame with the point cloud clusters in the subsequent point cloud frame. In some possible implementations, the point cloud clusters in the current point cloud frame can also be matched forward, that is, matching the point cloud clusters in the current point cloud frame with the point cloud clusters in the previous point cloud frame.

[0408] The above is an introduction to false alarm determination in a graphical manner. For ease of understanding, a flow chart of false alarm determination is provided below. Please refer to Figure 18, which is a flow chart of another false alarm determination method provided in an embodiment of the present application. This false alarm determination method can be performed by a point cloud testing device. Specifically, it can include steps S1801 to S1804, as follows:

[0409] Step S1801: point cloud clustering.

[0410] The point cloud testing device clusters the unmatched point clouds in the current point cloud frame to obtain point cloud clusters.

[0411] Step S1802: Allocate ID and initial health value.

[0412] The point cloud testing device assigns an ID (optional) and initial health value to each point cloud cluster. The ID is, for example, G1 or G2 as shown in Figure 17. Optionally, when point cloud clusters in different point clouds can be matched, they can share the same ID to facilitate false alarm detection.

[0413] The initial health value is, for example, 60.

[0414] Step S1803: Matching with the point cloud cluster in the next point cloud frame.

[0415] As shown in FIG17 , when the next frame of point cloud frame #0 is point cloud frame #1, the point cloud clusters in point cloud frame #0 are matched with the point cloud clusters in point cloud frame #1.

[0416] Step S1804: If the match is successful, the health value is +20; if the match is unsuccessful, the health value is -20.

[0417] As shown in FIG17 , point cloud cluster G4 is successfully matched with point cloud cluster G1, and the health value of point cloud cluster G4 (ie, G1) increases by 20. Point cloud cluster G2 is not successfully matched in point cloud frame #1, and its health value decreases by 20.

[0418] If the health value of the point cloud cluster in the next point cloud frame is ≤ 60, the point cloud cluster is discarded. For example, if the health value of point cloud cluster G2 is 40, the point cloud cluster G2 is discarded if it meets this condition.

[0419] If the life value of the point cloud cluster in the next point cloud frame is 100, a false alarm is generated. This cycle is repeated to determine the false alarm target in the point cloud frame.

[0420] In the above implementation, since a single sampling point is prone to flickering, the matching complexity is high and the result reliability is low, the above implementation clusters the unmatched sampling points and tracks the unmatched point cloud in the form of point cloud clusters. This not only reduces the matching complexity, but also greatly improves the reliability and availability of false alarm judgment, and improves the accuracy of point cloud testing.

[0421] In a possible implementation, the unmatched point cloud may also be used to determine a false alarm rate.

[0422] Exemplarily, the point cloud testing device determines the false alarm rate based on the number of point cloud frames involved in the warning target calculation and the number of false alarm point cloud frames. A false alarm point cloud frame is a point cloud frame containing a false alarm target, or a false alarm point cloud frame is a point cloud frame containing at least one point cloud cluster whose health value reaches a first threshold.

[0423] For example, the false alarm rate ρ satisfies the following formula:

[0424]

[0425] Among them, n is the number of point cloud frames involved in the calculation of false alarm targets, n false is the number of frames with false alarm targets.

[0426] For example, in Figure 17, the number of point cloud frames involved in the false alarm target calculation is 4, and the point cloud frames containing false alarm targets are point cloud frames #3 and #4, so the false alarm rate is 50%. In this case, although point cloud frames #0 and #1 also contain point cloud clusters that form false alarm targets, the false alarm targets had not yet been confirmed at that time and are therefore not included in the false alarm rate calculation.

[0427] Of course, in some scenarios, once the false alarm target is determined, the point cloud frame where the point cloud cluster in the undetermined stage is located is also regarded as the point cloud frame with false alarm. At this time, point cloud frame #0 and point cloud frame #1 can also be regarded as false alarm point cloud frames, that is, the false alarm rate is 100%.

[0428] As another possible design, unmatched ground truth values ​​can be used to determine missed detections. A missed detection occurs when a target is present but the radar interprets it as absent and does not output a point cloud. A missed detection in a point cloud can correspond to the ground truth value of an unmatched point cloud.

[0429] Optionally, when determining missed detection, it may be determined whether the volume object is blocked. If the volume object is blocked, the missed detection of the volume object is not considered a valid missed detection.

[0430] In one possible embodiment, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame. When judging missed detection, the point cloud testing device determines the suspected missed detection target in the third point cloud frame based on the unmatched true value corresponding to the third point cloud frame, and determines the obscured volume target based on the field of view relationship between at least one volume target and the radar. The point cloud testing device filters the obscured volume targets in the suspected missed detection targets in the third point cloud frame, and determines the missed detection targets contained in the third point cloud frame. In this way, the obscured volume targets in the suspected missed detection targets are removed according to the occlusion relationship, thereby reducing the missed detection judgment error caused by the occlusion of the volume target.

[0431] In one possible implementation, determining the obscured volume target based on a field of view relationship between at least one volume target and a radar includes:

[0432] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0433] Please refer to Figure 19, which is a schematic diagram of the position of a body target provided in an embodiment of the present application. The body targets shown in Figure 19 include body target T5, body target T6, body target T7 and body target T8, where the number and ID are only examples. If in a certain point cloud frame, the true values ​​of body target T5, body target T7 and body target T7 are all unmatched true values, the occlusion relationship between the body targets needs to be determined. Among them, the occlusion relationship can be determined by the corner points in the true value of the body target.

[0434] As an example of a method for confirming an occlusion relationship, if there are V occluded corner points among the multiple corner points of the fifth object, then the fifth object is occluded, where V is an integer and V>0. An occluded corner point is a corner point where the line connecting the corner points intersects the edges of other objects. As shown in Figure 19, taking V = 4 as an example, the eight corner points of object T5 are represented by A1 to A8. Among them, the lines connecting A3, A5, A7, and A8 with the DUT all intersect with the edges of object T8. Therefore, these four corner points are all occluded corner points, and therefore object T5 is an occluded object. In contrast, the eight corner points of object T6 are represented by B1 to B8, and only B6 is an occluded corner point, so object T6 is not an occluded object.

[0435] As another example of a method for confirming an occlusion relationship, if the fifth object is an occlusion object, it satisfies the following two conditions: ① V corner points of the fifth object intersect with the lines connecting the DUT and the edges of other objects, where V is an integer and V>0; ② The number of valid edges (see below for explanation) in the fifth object is greater than or equal to a fourth threshold. The fourth threshold can be predefined or pre-set. For example, the fourth threshold can be 4, or the fourth threshold can be 1.

[0436] For condition ①, V can be equal to the total number of corner points, for example, V = 8. For condition ②, the valid edge can be determined as follows: for any corner point or any edge corner point in the fifth body target (edge ​​corner points refer to corner points located on the edge, such as A2, A4, A5, and A7 shown in Figure 19), if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, then the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner point on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, then the first edge is a valid edge. For example, as shown in Figure 19, the line connecting corner point A2 and the DUT intersects on the edge where A5 is located, so the edge where corner point A2 is located is an invalid edge of body target T5. Similarly, the edge where corner point A4 is located is an invalid edge. The line connecting corner point A5 and DUT does not intersect with other edges in volume target T5, so the edge where corner point A5 is located is a valid edge of volume target T5. Similarly, the edge where corner point A7 is located is a valid edge.

[0437] Taking V = 8 and the fourth threshold value of 1 as an example, as shown in Figure 19, the lines connecting the eight corner points of volume object T5 all intersect with the true value of volume object T8, thus satisfying condition ①. The edges of volume object T5 containing A5 and A7 are valid edges, thus satisfying condition ②. Therefore, volume object T5 is an occluded object.

[0438] In another possible implementation, when determining missed detections, the area corresponding to the unmatched ground truth value can be tracked in the point cloud frame, and multi-frame correlation can be used to determine whether the volumetric target was missed. This allows the unmatched ground truth value to be tracked in subsequent point cloud frames, reducing missed detection errors caused by point cloud flicker, improving the accuracy of false alarm determination, and enhancing the accuracy of point cloud testing.

[0439] Exemplarily, when there is a missed detection of a sixth object in three consecutive point cloud frames, the sixth object is determined to be a missed detection object.

[0440] In yet another possible implementation, the unmatched true value may also be used to determine the missed detection rate.

[0441] Exemplarily, the point cloud testing device determines the missed detection rate based on the number of point cloud frames involved in the early warning target calculation and the number of missed detection point cloud frames, wherein the missed detection point cloud frames are point cloud frames in which missed detection targets exist (or are determined to have missed detection targets).

[0442] For example, the missed detection rate γ satisfies the following formula:

[0443] Where n is the number of point cloud frames involved in the missed target calculation, and n_lose is the number of missed point cloud frames. For more information, refer to the previous example for calculating the false alarm rate.

[0444] The above describes in detail the method of the embodiment of the present application. The following provides an apparatus of the embodiment of the present application.

[0445] It should be understood that the division of units in the device provided in the embodiments of the present application is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the units in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units by designing the logical relationship of the components in the circuit. For another example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units. All units of the above devices can be implemented in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0446] The present application is reasonable, and each unit in the device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0447] In addition, the various units in the above devices can be fully or partially integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the device. The type of the at least one processor can be different, for example, including a CPU and FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0448] Several possible arrangements are listed below.

[0449] Please refer to Figure 20, which is a structural diagram of a point cloud testing device provided in an embodiment of the present application. Optionally, the point cloud testing device 200 can be an independent device, such as a server. Alternatively, the point cloud testing device 200 can also be a device in an independent device (such as a node), such as a chip or an integrated circuit. The point cloud testing device 200 is used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18. For example, the point cloud testing device 200 can replace the point cloud testing device 301 in the system shown in Figure 3.

[0450] The point cloud testing device 200 shown in FIG20 includes a data acquisition module 2001 and a data matching module 2002, wherein:

[0451] The data acquisition module 2001 is used to obtain true value data and point cloud. The true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT on the volume target. The detection result includes sampling points.

[0452] The data matching module 2002 is used to match the point cloud with the true value data to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0453] Optionally, the number of volume targets may be one or more. In order to facilitate the description of the solution of the present application, the number of volume targets is described as at least one below.

[0454] In yet another possible implementation, the data matching module 2002 is configured to:

[0455] Establish a three-dimensional matching box based on the ground truth data;

[0456] The point cloud is matched with the 3D matching box to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0457] In one possible implementation, the data matching module 2002 is configured to:

[0458] Project the true value data to obtain two-dimensional true value data;

[0459] Project the point cloud to obtain a two-dimensional point cloud;

[0460] The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

[0461] In one possible implementation, the data matching module 2002 is configured to:

[0462] Project the true value data onto the horizontal plane to obtain two-dimensional true value data;

[0463] Project the point cloud to obtain a two-dimensional point cloud, including:

[0464] Project the point cloud onto the horizontal plane to obtain a two-dimensional point cloud.

[0465] In another possible implementation, the time of the ground truth data and the point cloud is aligned. Furthermore, when the ground truth data and the point cloud are projected into two-dimensional data, the time of the two-dimensional ground truth data and the two-dimensional point cloud is aligned.

[0466] In yet another possible implementation, the coordinates of the ground truth data and the point cloud are aligned.

[0467] In yet another possible implementation, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

[0468] The data matching module 2002 is further configured to:

[0469] Determine at least one ground truth frame in a first ground truth frame, wherein one ground truth frame corresponds to one volume target, and the first ground truth frame belongs to multiple ground truth frames;

[0470] A matching result subset is obtained based on a range of at least one true value frame and positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.

[0471] In another possible implementation, the at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target;

[0472] When the first point cloud frame includes the first sampling point and the first sampling point falls within the first ground truth frame, the first sampling point is a matching sampling point, and the first sampling point matches the ground truth of the first object.

[0473] In a case where the first point cloud frame includes the second sampling point and the second sampling point does not fall into any of the truth value frames in the at least one truth value frame, the second sampling point is an unmatched sampling point;

[0474] When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

[0475] In another possible implementation, the number of at least one truth box is greater than or equal to 2.

[0476] The data matching module 2002 is further configured to:

[0477] When the first point cloud frame includes the third sampling point and the third sampling point falls within at least two ground truth frames, the ground truth value of the volume object matching the third sampling point is determined according to a position between the third sampling point and the ground truth values ​​of the volume objects corresponding to the at least two ground truth frames.

[0478] In yet another possible implementation, the data matching module 2002 is further configured to:

[0479] Establish point pairs between the true values ​​of the volume targets corresponding to at least two truth frames and the third sampling point, construct a distance matrix based on the point pairs, obtain the distance between the true value and the third sampling point, and take the true value with the closest distance as the true value matching the third sampling point.

[0480] In another possible implementation, the data acquisition module 2001 is further configured to:

[0481] Preprocess the initial ground truth and initial point cloud to obtain ground truth data and a point cloud. This preprocessing can include one or more of the following: time alignment, coordinate conversion, and format conversion. Preprocessing can improve the correspondence between the ground truth data and the point cloud, reduce matching complexity, and increase point cloud testing efficiency.

[0482] In another possible implementation, the point cloud testing device 200 further includes a data calculation module 2003 , which is configured to evaluate the accuracy, false alarms, and missed detections of the point cloud through a set of matching results.

[0483] In another possible embodiment, the point cloud testing device further includes a data calculation module configured to obtain accuracy assessment data regarding the DUT based on the matching sampling points in the matching result set. The accuracy assessment data includes one or more of the following: the number of matching sampling points, ranging accuracy, velocity accuracy, and height accuracy.

[0484] In yet another possible implementation, the data calculation module 2003 is further configured to:

[0485] The number of matching sampling points on the fourth body target is obtained according to the number of sampling points that match the true value of the fourth body target among the matching sampling points.

[0486] In another possible implementation, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0.

[0487] The data calculation module is further used to evaluate the ranging accuracy of the DUT when detecting the second target based on the N sampling points and the M corner points.

[0488] In another possible implementation, the N sampling points include the nearest sampling point, and the M corner points include the lateral nearest corner point. Furthermore, the ranging accuracy includes lateral ranging accuracy with respect to the second object, and the lateral ranging accuracy with respect to the second object is related to the lateral distance between the nearest sampling point and the DUT, and the radial distance between the radial nearest corner point and the DUT.

[0489] In another possible implementation, the lateral ranging accuracy σ of the second target is x Satisfies the following formula:

[0490] σ x =|X pi -X cj |

[0491] Among them, X pi is the lateral distance between the nearest sampling point and the DUT, X cj is the lateral distance between the nearest lateral corner and the DUT.

[0492] In another possible implementation, the N sampling points include the nearest sampling point, and the M corner points include the radially nearest corner point. Furthermore, the ranging accuracy includes the longitudinal ranging accuracy with respect to the second object. The longitudinal ranging accuracy with respect to the second object is related to the radial distance between the nearest sampling point and the DUT, and the radial distance between the radially nearest corner point and the DUT.

[0493] In another possible implementation, the longitudinal distance measurement accuracy σ of the fourth target is d Satisfies the following formula:

[0494] σ d =|D pi -D ck |

[0495] Among them, D pi is the radial distance between the nearest sampling point and the DUT, D ck is the radial distance between the closest radial corner and the DUT.

[0496] In yet another possible implementation, the velocity measurement accuracy includes velocity measurement accuracy with respect to a third-body target;

[0497] The matching sampling points include K sampling points that match the true value of the third-body target, where K is an integer and K>0;

[0498] The K sampling points include the strongest sampling point. The velocity measurement accuracy of the third-body target is related to the radial velocity of the strongest sampling point and the true radial velocity of the third-body target.

[0499] The above embodiment describes a method for determining the speed measurement accuracy. v It can be indicated by the absolute value of the radial velocity error between the strongest point in the matching point and the reference true value. For example, the velocity accuracy σ v Satisfies the following formula:

[0500] σ v =|V pi -V t |

[0501] Among them, V pi is the radial velocity of the strongest sampling point, V t is the true radial velocity of the third body target.

[0502] As a possible implementation, the strongest sampling point is the sampling point with the strongest radar cross section (RCS) among the K sampling points, and the number of sampling points that match the true value is K. The RCS of the K sampling points are expressed as R p1 ,R p2 ,R p3 ,…,R pK The RCS of the strongest sampling point can be expressed as R pi , which can satisfy the following formula:

[0503] R pi =max(R p1 ,R p2 ,R p3 ,…,R pK )

[0504] Optionally, the true radial velocity of the third-body target may be replaced by the radial velocity of the third-body target.

[0505] In yet another possible implementation, unmatched sampling points may be used for false alarm determination.

[0506] In another possible embodiment, the point cloud testing device further includes a data calculation module, which is further configured to:

[0507] False alarm targets in the multiple continuous point cloud frames are determined according to sampling points in the multiple continuous point cloud frames among the unmatched sampling points.

[0508] In another possible implementation, the plurality of consecutive point cloud frames include the second point cloud frame and Q point cloud frames following the second point cloud frame, where Q is an integer and Q>0;

[0509] The data calculation module 2003 is further used to:

[0510] Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

[0511] assigning an initial life value to a first point cloud cluster in the at least one point cloud cluster;

[0512] Determine the point cloud clusters in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames;

[0513] False alarm targets in a plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

[0514] In the above embodiment, the testing device clusters the unmatched point cloud to obtain multiple point cloud clusters, and each point cloud cluster is assigned an initial life value. For a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with multiple subsequent point cloud frames. If there is a matching point cloud cluster in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; the matching of multiple point cloud frames is repeated in this way. If the life value of the point cloud cluster reaches the first threshold, the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster reaches the second threshold or is lower than the third threshold, the point cloud cluster does not form a false alarm target, and the point cloud cluster can be optionally discarded.

[0515] In some scenarios, when matching point cloud clusters, the cluster bounding box is determined based on the cluster size within the point cloud frame. This bounding box should encompass the points within the cluster. For the current point cloud frame, if the cluster bounding box in the next point cloud frame overlaps with the current frame's, the overlap matrix between the two cluster bounding boxes is calculated to establish an association between the two frames. If the cluster bounding box in the next point cloud frame successfully matches the current frame's cluster bounding box, the cluster's health is increased; otherwise, the cluster's health is decreased.

[0516] In yet another possible implementation, the unmatched point cloud may also be used to determine a false alarm rate.

[0517] Exemplarily, the point cloud testing device determines the false alarm rate based on the number of point cloud frames involved in the warning target calculation and the number of false alarm point cloud frames. A false alarm point cloud frame is a point cloud frame containing a false alarm target, or a false alarm point cloud frame is a point cloud frame containing at least one point cloud cluster whose health value reaches a first threshold.

[0518] For example, the false alarm rate ρ satisfies the following formula:

[0519] Among them, n is the number of point cloud frames involved in the calculation of false alarm targets, n false is the number of frames with false alarm targets.

[0520] In another possible implementation, the unmatched true value can be used to determine missed detections. A missed detection refers to an event in which a target is present but the radar determines that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

[0521] In yet another possible implementation, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

[0522] The data calculation module 2003 is further used to:

[0523] Determine the suspected missed detection target in the third point cloud frame according to the unmatched true value corresponding to the third point cloud frame;

[0524] determining an obscured volume target based on a field of view relationship between at least one volume target and the radar;

[0525] The occluded volume targets in the suspected missed detection targets in the third point cloud frame are filtered out to determine the missed detection targets contained in the third point cloud frame.

[0526] In another possible implementation, the data calculation module 2003 is further configured to:

[0527] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0528] Whether the fifth volume object in the at least one volume object is blocked is determined according to an intersection of a line between a fifth volume object and the DUT and edges of other volume objects.

[0529] For example, if there are V occluded corner points among the multiple corner points of the fifth body target, then the fifth body target is occluded, V is an integer and V>0, where the occluded corner points are corner points where the lines connecting the corner points intersect with the edges of other body targets.

[0530] For another example, if the fifth body target is an occluding target, it satisfies the following two conditions: ① The lines connecting the V corner points of the fifth body target and the DUT intersect with the edges of other body targets, V is an integer and V>0; ② The number of valid edges in the fifth body target is greater than or equal to the fourth threshold. The fourth threshold can be predefined or pre-set. For example, the fourth threshold can be 4, or the fourth threshold is 1. The valid edge can be determined as follows: for any corner point or any edge corner point (edge ​​corner point refers to a corner point located on an edge) in the fifth body target, if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, then the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner points on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, then the first edge is a valid edge.

[0531] In another possible implementation, when making missed detection judgments, the point cloud testing device 210 may track areas corresponding to unmatched true values ​​in the point cloud frame, and use a multi-frame association method to determine whether the volume target is missed.

[0532] For example, when there is a missed detection of a certain object in three consecutive point cloud frames, the object is determined to be a missed detection object.

[0533] In yet another possible implementation, the unmatched true value may also be used to determine the missed detection rate.

[0534] Exemplarily, the point cloud testing device determines the missed detection rate based on the number of point cloud frames involved in the early warning target calculation and the number of missed detection point cloud frames, wherein the missed detection point cloud frames are point cloud frames in which missed detection targets exist (or are determined to have missed detection targets).

[0535] For example, the missed detection rate γ satisfies the following formula:

[0536] Among them, n is the number of point cloud frames involved in the calculation of missed targets, and n_lose is the number of missed point cloud frames.

[0537] Please refer to Figure 21, which is a structural diagram of a computing device provided in an embodiment of the present application.

[0538] The computing device 210 may be an independent device, such as a node, or a device included in an independent device, such as a chip, software module, or integrated circuit. The computing device 210 may include at least one processor 2101 and a communication interface 2102. Optionally, it may also include at least one memory 2103. Further optionally, it may also include a connection line 2104, wherein the processor 2101, communication interface 2102, and / or memory 2103 are connected via the connection line 2104 and / or communicate with each other via the connection line 2104 to transmit control signals and / or data signals.

[0539] Among them, processor 2101 is a module that performs arithmetic operations and / or logical operations. In one implementation, the processor can be a circuit with instruction reading and running capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP), etc.; in another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by a processor as a dedicated integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU) tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0540] The communication interface 2102 may be used to provide information input or output for at least one processor, or to receive externally transmitted signals and / or send externally transmitted signals.

[0541] For example, the communication interface 2102 may include an interface circuit. For example, the communication interface 2102 may include a wired link interface such as an Ethernet cable, or a wireless link interface (Wi-Fi, Bluetooth, general wireless transmission, vehicle-mounted short-range communication technology, other short-range wireless communication technology, etc.).

[0542] Optionally, the communication interface 2102 may further include a radio frequency transmitter, an antenna, etc. When the communication interface 2102 includes an antenna, the number of antennas may be one or more.

[0543] As a possible design, if the computing device 210 is a standalone device, the communication interface 2102 may include a receiver and a transmitter. The receiver and transmitter may be the same component or different components. When the receiver and transmitter are the same component, the component may be referred to as a transceiver.

[0544] As another possible design, if the computing device 210 is a chip or a circuit, the communication interface 2102 may include an input interface and an output interface. The input interface and the output interface may be the same interface, or may be different interfaces.

[0545] Optionally, the functions of the communication interface 2102 may be implemented by a transceiver circuit or a dedicated transceiver chip.

[0546] Memory 2103 is used to provide storage space for storing data such as the operating system and computer programs. Memory 2103 can be one or a combination of random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0547] The functions and actions of the modules or units in the computing device 210 listed above are merely exemplary.

[0548] Each functional unit in the computing device 210 can be used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18.

[0549] Optionally, the processor 2101 may be a processor specifically used to execute the aforementioned method (for convenience of distinction, referred to as a dedicated processor), or a processor that executes the aforementioned method by calling a computer program (for convenience of distinction, referred to as a dedicated processor). Optionally, the at least one processor may include both a dedicated processor and a general-purpose processor.

[0550] Optionally, in the case where the computing device 210 includes at least one memory 2103 , if the processor 2101 implements the aforementioned point cloud testing method by calling a computer program, the computer program may be stored in the memory 2103 .

[0551] The present application also provides a chip comprising a logic circuit and a communication interface. The communication interface is configured to receive or transmit signals, and the logic circuit is configured to receive or transmit signals via the communication interface. The chip is configured to implement the aforementioned point cloud testing methods, such as those shown in Figures 5, 11, or 18.

[0552] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on at least one processor (or communication device), the aforementioned point cloud testing method is implemented, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18.

[0553] An embodiment of the present application also provides a computer program product, which includes computer instructions, and the computing instructions are used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18.

[0554] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0555] In the embodiments of this application, "at least one" refers to one or more, and "more" refers to two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items.

[0556] For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or plural. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, A and / or B can represent: A alone, A and B together, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0557] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, timing, priority, or importance of multiple objects. For example, the first point cloud frame, the second point cloud frame, and the third point cloud frame are only used to facilitate the description of point cloud frames in different implementations and do not indicate differences in their order, importance, data content, etc. In some scenarios, the first point cloud frame and the second point cloud frame can be the same point cloud frame.

[0558] In the above embodiments, the terms "when..." and "if..." can be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. The above are merely optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concepts and principles of the present application shall be included in the scope of protection of the present application.

[0559] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or by programs instructing related hardware to accomplish the steps. The programs may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

Claims

1. A point cloud testing method, It is characterized in that The method comprises: Acquire true value data and a point cloud, wherein the true value data is the true value of at least one volume target, and the point cloud is a detection result obtained by the device under test (DUT) detecting the at least one volume target; Projecting the true value data to obtain two-dimensional true value data; Projecting the point cloud to obtain a two-dimensional point cloud; The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, wherein the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

2. The method according to claim 1, It is characterized in that The step of projecting the true value data to obtain two-dimensional true value data includes: Projecting the true value data onto a horizontal plane to obtain the two-dimensional true value data; Projecting the point cloud to obtain a two-dimensional point cloud includes: The point cloud is projected onto the horizontal plane to obtain the two-dimensional point cloud.

3. The method according to claim 1 or 2, It is characterized in that The two-dimensional true value data includes a plurality of true value frames, and the two-dimensional point cloud includes a plurality of point cloud frames; The matching of the two-dimensional point cloud with the two-dimensional true value data to obtain a matching result set includes: Determine at least one truth frame in a first truth frame, wherein one truth frame corresponds to one volume target, and the first truth frame belongs to the plurality of truth frames; A matching result subset is obtained based on the range of the at least one true value frame and the positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.

4. The method according to claim 3, It is characterized in that The at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target; In a case where the first point cloud frame includes a first sampling point and the first sampling point falls within the first true value frame, the first sampling point belongs to a matching sampling point, and the first sampling point matches the true value of the first volume target; In a case where the first point cloud frame includes a second sampling point and the second sampling point does not fall into any truth frame of the at least one truth frame, the second sampling point belongs to an unmatched sampling point; When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

5. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: According to the matching sampling points in the matching result set, accuracy evaluation data about the DUT is obtained, and the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy and height accuracy.

6. The method according to claim 5, It is characterized in that The matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0; The N sampling points include the nearest sampling point, and the nearest sampling point is one of the N sampling points that is closest to the DUT. The M corner points include the closest lateral corner point and the closest radial corner point. The closest lateral corner point is the corner point with the closest lateral distance to the DUT among the M corner points, and the closest radial corner point is the corner point with the closest radial distance to the DUT among the M corner points. The ranging accuracy includes a lateral ranging accuracy with respect to a second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT; And / or, the ranging accuracy includes a longitudinal ranging accuracy with respect to a second object, and the longitudinal ranging accuracy with respect to the second object is related to a radial distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT.

7. The method according to claim 5, It is characterized in that The velocity measurement accuracy includes the velocity measurement accuracy about a third-body target; The matching sampling points include K sampling points that match the true value of the third object, where K is an integer and K>0; The K sampling points include the strongest sampling point, and the strongest sampling point is the sampling point with the strongest radar cross section RCS among the K sampling points. The velocity measurement accuracy of the third target is related to the radial velocity of the strongest sampling point and the radial velocity of the third target.

8. The method according to any one of claims 1 to 7, It is characterized in that The two-dimensional point cloud includes a plurality of continuous point cloud frames, the matching set includes unmatched sampling points, and the method further includes: According to sampling points in the multiple continuous point cloud frames that are located in the multiple continuous point cloud frames among the unmatched sampling points, false alarm targets in the multiple continuous point cloud frames are determined.

9. The method according to claim 8, It is characterized in that The multiple continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0; The determining of the false alarm target in the point cloud according to the sampling points in the unmatched sampling points that are located in the plurality of continuous point cloud frames comprises: Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster; assigning an initial life value to a first point cloud cluster among the at least one point cloud cluster; Determine a point cloud cluster in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames; The false alarm targets in the plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

10. The method according to any one of claims 1 to 6, It is characterized in that The two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame; The method further comprises: Determining a suspected missed target in the third point cloud frame according to an unmatched true value corresponding to the third point cloud frame; Determining the obscured volume target according to the field of view relationship between the at least one volume target and the radar; The obscured volume targets in the suspected missed detection targets in the third point cloud frame are filtered to determine the missed detection targets included in the third point cloud frame.

11. A point cloud testing device, It is characterized in that The point cloud testing device comprises a data acquisition module and a data matching module, wherein: The data acquisition module is used to acquire true value data and point cloud, wherein the true value data is the true value of at least one volume target, and the point cloud is the detection result obtained by the device under test DUT detecting the at least one volume target; The data matching module is used to: Projecting the true value data to obtain two-dimensional true value data; Projecting the point cloud to obtain a two-dimensional point cloud; The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, wherein the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

12. The point cloud testing device according to claim 11, It is characterized in that The data matching module is used for: Projecting the true value data onto a horizontal plane to obtain the two-dimensional true value data; The point cloud is projected onto the horizontal plane to obtain the two-dimensional point cloud.

13. The point cloud testing device according to claim 11 or 12, It is characterized in that The two-dimensional true value data includes a plurality of true value frames, and the two-dimensional point cloud includes a plurality of point cloud frames; The data matching module is also used for: Determine at least one truth frame in a first truth frame, wherein one truth frame corresponds to one volume target, and the first truth frame belongs to the plurality of truth frames; A matching result subset is obtained based on the range of the at least one true value frame and the positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.

14. The point cloud testing device according to claim 13, It is characterized in that The at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target; In a case where the first point cloud frame includes a first sampling point and the first sampling point falls within the first true value frame, the first sampling point belongs to a matching sampling point, and the first sampling point matches the true value of the first volume target; In a case where the first point cloud frame includes a second sampling point and the second sampling point does not fall into any truth frame of the at least one truth frame, the second sampling point belongs to an unmatched sampling point; When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

15. The point cloud testing device according to any one of claims 1 to 14, It is characterized in that The point cloud testing device further comprises a data calculation module, which is used for: According to the matching sampling points in the matching result set, accuracy evaluation data about the DUT is obtained, and the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy and height accuracy.

16. The point cloud testing device according to claim 15, It is characterized in that The matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0; The N sampling points include a nearest sampling point, which is a sampling point that is closest to the DUT among the N sampling points; the M corner points include a lateral nearest corner point and a radial nearest corner point, which is a corner point that is closest to the DUT in lateral distance among the M corner points; and the radial nearest corner point is a corner point that is closest to the DUT in radial distance among the M corner points; The ranging accuracy includes a lateral ranging accuracy with respect to a second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT; And / or, the ranging accuracy includes a longitudinal ranging accuracy with respect to a second object, and the longitudinal ranging accuracy with respect to the second object is related to a radial distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT.

17. The point cloud testing device according to claim 16, It is characterized in that The velocity measurement accuracy includes the velocity measurement accuracy about a third-body target; The matching sampling points include K sampling points that match the true value of the third object, where K is an integer and K>0; The K sampling points include the strongest sampling point, and the strongest sampling point is the sampling point with the strongest radar cross section RCS among the K sampling points. The velocity measurement accuracy of the third target is related to the radial velocity of the strongest sampling point and the radial velocity of the third target.

18. The point cloud testing device according to any one of claims 11 to 17, It is characterized in that The two-dimensional point cloud includes a plurality of continuous point cloud frames, and the matching set includes unmatched sampling points; The point cloud testing device further includes a data calculation module, which is used to determine false alarm targets in the multiple continuous point cloud frames based on sampling points in the unmatched sampling points that are located in the multiple continuous point cloud frames.

19. The point cloud testing device according to claim 18, It is characterized in that The multiple continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0; The point cloud testing method further comprises a data calculation module, which is used to: Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster; assigning an initial life value to a first point cloud cluster among the at least one point cloud cluster; Determine a point cloud cluster in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames; The false alarm targets in the plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

20. The point cloud testing device according to any one of claims 11 to 17, It is characterized in that The two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame; The point cloud testing device further comprises a data calculation module, which is used for: Determining a suspected missed target in the third point cloud frame according to an unmatched true value corresponding to the third point cloud frame; Determining the obscured volume target according to the field of view relationship between the at least one volume target and the radar; The obscured volume targets in the suspected missed detection targets in the third point cloud frame are filtered to determine the missed detection targets included in the third point cloud frame.

21. A chip, It is characterized in that The chip includes a processor and a communication interface; The communication interface is used to receive and / or send data, and / or the communication interface is used to provide input and / or output for the processor; The processor is configured to implement the method according to any one of claims 1 to 10.

22. A computing device, It is characterized in that The computing device comprises a memory and a processor, wherein the memory stores computer instructions, and the processor is configured to call the computer instructions to implement the method according to any one of claims 1 to 10.

23. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on at least one processor, the method according to any one of claims 1 to 10 is implemented.

Images