Perception system validation method, apparatus, device, and medium

Abstract

The embodiment of the present disclosure provides a perception system verification method, device, apparatus and medium, and relates to the technical field of data processing. The perception system verification method comprises the following steps: determining a candidate perception data set comprising a plurality of candidate perception points and a candidate reference data set comprising a plurality of candidate reference points based on environment data collected by a movable device; determining one or more target point sets according to the distance of the candidate reference points and the candidate perception points in a target direction; and generating a verification result for a target perception system based on the target point set. Each candidate perception point is one of a plurality of data points used to represent a perception object, each candidate reference point is one of a plurality of data points used to represent a reference object, the reference object and the perception object are objects with linear extension, and the target direction of each candidate reference point is a direction determined based on the extension direction of the reference object at the candidate reference point.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to a method, device, apparatus, and medium for verifying a sensing system. Background Technology

[0002] For mobile devices such as vehicles, motion control can be achieved based on driving automation technology. Mobile devices equipped with driving automation capabilities can deploy perception systems to perceive their environment, such as lane lines, vehicles, and pedestrians, in order to control their motion. The perception results of linearly extending environmental objects (such as lane lines, lane center lines, road boundaries, and curbs) provide crucial geometric constraints and semantic information for functional modules such as device localization, path planning, trajectory tracking, and lateral control. Therefore, the accuracy of linear object perception is of great significance to the motion control of mobile devices.

[0003] In related technologies, to verify the accuracy of lane line perception for linear objects, the accuracy of lane line perception results can be evaluated based on methods such as manual observation or curve fitting. For example, the lane lines perceived by the perception system can be compared with the actual ground truth lane lines to obtain the curve overlap between the two, thus obtaining the verification result of lane line perception. However, perception verification based on this method suffers from insufficient accuracy. Therefore, how to improve the accuracy of perception result verification for linear objects has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the present disclosure proposes a new technical solution for verification of a sensing system.

[0005] According to a first aspect of the present disclosure, a sensing system verification method is provided, the method comprising: Based on environmental data collected by a mobile device, a candidate sensing data set including multiple candidate sensing points and a candidate benchmark data set including multiple candidate benchmark points are determined. Each candidate sensing point is one of multiple data points used to characterize a sensing object, and each candidate benchmark point is one of multiple data points used to characterize a benchmark object. The benchmark object and the sensing object are objects with a linearly extended shape. The candidate sensing data set is a data set obtained after processing the environmental data using the target sensing system of the mobile device. The candidate benchmark data set is used to verify the accuracy of the candidate sensing data set. Based on the distance between the candidate reference point and the candidate sensing point in the target direction, one or more sets of target points are determined; wherein each set of target points includes a target reference point determined from a plurality of candidate reference points and a target sensing point corresponding to the target reference point determined from a plurality of candidate sensing points, and the target direction of each candidate reference point is a direction determined based on the extension direction of the reference object at the candidate reference point; A verification result for the target perception system is generated based on the set of target points.

[0006] Optionally, for the first reference point among the candidate reference points, the target direction of the first reference point is determined based on the following method: Among the plurality of candidate reference points, a second reference point and a third reference point adjacent to the first reference point are determined; Based on a preset tangent direction determination algorithm, the tangent direction of the first reference point is calculated according to the coordinates of the first reference point, the second reference point, and the third reference point; The tangent direction is taken as the extension direction of the reference object at the first reference point; The target direction of the first reference point is determined based on the extended direction.

[0007] Optionally, determining the target direction of the first reference point based on the extension direction includes: Based on the pose information of the mobile device when collecting the environmental data, at least two candidate reference directions are determined; Based on the extension direction, a candidate reference direction is determined from at least two candidate reference directions as the target reference direction of the first reference point; wherein, the target angle between the target reference direction and the extension direction is the minimum value of the angles between at least two candidate reference directions and the extension direction; The direction perpendicular to the target reference direction is taken as the target direction of the first reference point.

[0008] Optionally, determining one or more target point sets based on the distance between the candidate reference point and the candidate sensing point in the target direction includes: For each candidate reference point, if there is at least one first sensing point among the multiple candidate sensing points in the candidate sensing data set that satisfies the preset matching conditions, then the candidate reference point is taken as the target reference point, and the first sensing point with the smallest target distance from the target reference point in the target direction is determined among at least one first sensing point as the target sensing point corresponding to the target reference point, and the target point set is generated based on the target reference point and the target sensing point. The preset matching conditions include: the straight-line distance between the first sensing point and the candidate reference point is less than or equal to a preset distance threshold, and / or the sensing object corresponding to the first sensing point is of the same object type as the reference object corresponding to the candidate reference point.

[0009] Optionally, generating a verification result for the target perception system based on the target point set includes: For each set of target points, the baseline object attribute information of the baseline object corresponding to the target baseline point and the perception object attribute information of the perception object corresponding to the target perception point are determined; if it is determined that the perception object attribute information and the baseline object attribute information do not meet the preset attribute matching verification rules, the set of target points is determined to be an attribute error set. Based on the number of attribute error sets and the number of target point sets, an attribute-aware error index is calculated, and the attribute-aware error index is used as at least a part of the verification result.

[0010] Optionally, generating a verification result for the target perception system based on the target point set includes: Based on the target point set, determine the unmatched benchmark data points in the candidate benchmark data set and the unmatched sensing data points in the candidate sensing data set. The verification result of the target perception system is calculated based on the target point set, the unmatched sensing data points, and the unmatched reference data points.

[0011] Optionally, calculating the verification result of the target perception system based on the target point set, the unmatched sensing data points, and the unmatched reference data points includes: Based on the target point set, unmatched perception data points, and unmatched benchmark data points of each benchmark object, an object-level statistical index of the benchmark object is calculated, and the object-level statistical index is used as at least a part of the index of the verification result.

[0012] Optionally, determining a candidate sensing data set including multiple candidate sensing points and a candidate reference data set including multiple candidate reference points based on environmental data collected by a mobile device includes: Based on the target perception system of the mobile device, the environmental data is processed to generate a raw perception data set including multiple raw perception points. Obtain the original benchmark data set corresponding to the environmental data, wherein the original benchmark data set includes multiple original benchmark points; The original reference data set and the original sensing data set are resampled to generate a candidate reference data set including multiple candidate reference points and a candidate sensing data set including multiple candidate sensing points.

[0013] Optionally, the original baseline data set and the original sensed data set are resampled, including: Based on the pose information of the mobile device when collecting the environmental data, resampling information for resampling is determined. The resampling information includes a first sampling direction, a second sampling direction, a first resampling distance interval corresponding to the first sampling direction, and a second resampling distance interval corresponding to the second sampling direction. The first sampling direction is the longitudinal direction of the mobile device, and the second sampling direction is perpendicular to the first sampling direction. The original reference data set or the original sensing data set are respectively used as the original data set. Based on the resampling information, each original data point in the original data set, and the original object corresponding to each original data point, the original data set is resampled to generate a candidate data set. The original data point is the original reference point or the original sensing point, the original object is the reference object or the sensing object, and the candidate data set is the candidate reference data set or the candidate sensing data set.

[0014] Optionally, based on the resampling information, each original data point in the original dataset, and the original object corresponding to each original data point, the original dataset is resampled to generate a candidate dataset, including: For each original data point in the original dataset, target resampling is performed to obtain sampled data points corresponding to the original data points; The candidate data set is generated based on the multiple original data points and the sampled data points corresponding to each original data point. The target resampling process includes: Determine the extension direction and resampling interval corresponding to the original data point; wherein, the resampling interval is the linear interval between the original data point and other adjacent original data points where the original object is located; Based on the extension direction, the direction with the smallest angle to the extension direction is determined from the first sampling direction and the second sampling direction of the resampling information as the resampling direction corresponding to the original data point; Based on the resampling direction and the resampling distance interval corresponding to the resampling direction, resampling is performed within the resampling interval to obtain the sampled data points.

[0015] According to a second aspect of the present disclosure, a mobile device is provided, the mobile device being equipped with a target perception system, the target perception system being a perception system verified based on the perception system verification method described in the first aspect.

[0016] According to a third aspect of the present disclosure, a sensing system verification apparatus is provided, including a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to invoke the computer instructions from the memory to execute the method described in the first aspect.

[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, the computer program implementing the method described in the first aspect when executed by a processor.

[0018] Based on the perception system verification method provided in this disclosure, a target point set is constructed by point-by-point matching between candidate reference points and candidate perception points. This method can retain the independent error information of each sampling point, avoid the smoothing masking of local deviations by polynomial curve fitting, and by setting the target direction of each candidate reference point to the direction determined by the extension direction of the reference object at that point, and using the distance between the candidate perception point and the candidate reference point in the target direction as the basis for matching and error measurement, the target direction can dynamically change with the local orientation of the linear object. Thus, it can accurately reflect the true geometric deviation of the perception point relative to the reference point in various road network scenarios such as straight roads, curves, and ramps. Compared with the method of measuring by straight distance or fixed direction in related technologies, the method of this embodiment can avoid the geometric deviation distortion problem caused by high curvature scenarios in related technologies, and improve the verification accuracy and reliability of the target perception system's linear object perception results in complex road network scenarios.

[0019] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0021] Figure 1 This is a schematic diagram of an intelligent connected system to which the methods provided in the embodiments of this disclosure can be applied.

[0022] Figure 2 It is based on Figure 1 The illustrated embodiment provides a structural schematic diagram of a mobile device.

[0023] Figure 3This is a flowchart illustrating a sensing system verification method provided in an embodiment of this disclosure.

[0024] Figure 4 This is a schematic diagram of data point sampling in a straight road scene provided in an embodiment of this disclosure.

[0025] Figure 5 This is a schematic diagram of data point sampling in a curve scene provided in an embodiment of this disclosure.

[0026] Figure 6 This is a schematic diagram of another curve scene data point sampling provided in an embodiment of this disclosure.

[0027] Figure 7 This is a flowchart illustrating another sensing system verification method provided in this embodiment.

[0028] Figure 8 This is a schematic diagram of the structure of a sensing system verification device provided in an embodiment of this disclosure. Detailed Implementation

[0029] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0030] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this disclosure or its application or use.

[0031] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0032] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0033] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0034] The elements involved in the embodiments of this disclosure may represent part or all of an element. For example, the elements involved in the embodiments of this disclosure may be at least a part of an element or all of an element.

[0035] The elements involved in the embodiments of this disclosure may be one or more, such as "a," "the," "the above," "described," "the foregoing," etc., used to indicate that the corresponding element is mentioned for the first time or is mentioned again, and do not have the meaning of limiting the quantity. It should also be noted that "multiple elements" in the embodiments of this disclosure may refer to two or more elements, and other quantifiers are similar.

[0036] It should be noted that all actions involving the collection, storage, use, processing, transmission, provision, disclosure, and deletion of data in this disclosure are carried out in accordance with the relevant data protection laws and regulations of the country or region where the data is located, and with the full authorization of the relevant data owner.

[0037] First, the application scenarios of the embodiments of this disclosure will be described.

[0038] Figure 1 This is a schematic diagram of an intelligent connected system 100 to which the methods provided in the embodiments of this disclosure can be applied. Figure 1 As shown, the intelligent connected system 100 may include: a mobile device 101, a server 102, and a user terminal 103.

[0039] In some examples, mobile device 101 may include mobile devices such as vehicles, robots, ships, and aircraft, such as vehicles, ships, and aircraft with driving automation features, and autonomously moving robots (such as cargo robots, exploration robots, or sweeping robots).

[0040] The driving automation function can include advanced driver assistance functions (ADAS) and automated driving functions (AWD). Automated driving, also known as intelligent driving or driverless driving, enables vehicles with driving automation functions to perform some or all of the driving tasks, such as environmental perception, decision-making, planning, and control execution. The levels of driving automation functions can refer to the vehicle intelligence classification standards established by the Society of Automotive Engineers (SAE), for example, divided into six levels from L0 to L5. L0 is emergency assistance, L1 is partial driver assistance, L2 is combined driver assistance, L3 is conditional automated driving, L4 is highly automated driving, and L5 is fully automated driving. The above classification of driving automation function levels is merely an example, and this disclosure does not limit the classification standards and levels of driving automation functions.

[0041] In some examples, server 102 can be a single server or a distributed server cluster consisting of multiple servers, and its deployment method can include local servers or cloud servers. Server 102 can communicate with mobile device 101 and / or user terminal 103 via a communication network, providing various services to mobile device 101 and / or user terminal 103. For example, the server can receive sensing data sent by mobile device 101, provide services such as high-precision maps, data analysis, and decision planning for mobile device 101, or receive query commands or control commands sent by user terminal 102, providing corresponding services to the user.

[0042] In some examples, user terminal 103 can be any form of electronic device providing services to the user, such as a personal computer, laptop, smart tablet, smartphone, smart wearable device, etc. The user can interact with the mobile device or server through the human-computer interaction terminal configured on the mobile device 101, or through user terminal 103. For example, the user can query the status and / or parameters of the mobile device, or control the mobile device to perform set tasks and / or modify configuration parameters, etc. The user terminal runs an application based on the intelligent network system to achieve interaction with the mobile device or server. This application can be a local application, a web application, or a mini-program, etc., and is not limited thereto.

[0043] In some examples, the aforementioned application running on the user's terminal can provide authentication or authorization services to the user. The user who is successfully authenticated and granted the corresponding permissions can query and / or control the mobile device within the scope of the granted permissions.

[0044] The mobile device 101, server 102, and user terminal 103 can communicate via a communication link provided by communication network 104. This communication network 104 can include one or more networks of any type, such as the Internet, Local Area Network (LAN), Wide Area Network (WAN), Virtual Private Network (VPN), Public Switched Telephone Network (PSTN), satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC, or a combination of these networks. The communication networks between the mobile device 101 and server 102, between the user terminal 103 and server 102, and between the user terminal 103 and mobile device 101 can be the same or different.

[0045] It should be noted that, Figure 1The structure of the intelligent connected system 100 shown is merely illustrative. The intelligent connected system in this embodiment is not limited to the above structure and may include more or fewer devices as needed, and the devices may be combined or split. For example, the intelligent connected system may not include user terminals and / or servers; as another example, user terminals and servers may be deployed together.

[0046] Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of a mobile device 101. As shown... Figure 2 As shown, the mobile device 101 may include a sensing component 1011, a computing platform 1012, an execution component 1013, etc. The sensing component 1011, the computing platform 1012, and the execution component 1013 may be connected via a bus or other means.

[0047] In some examples, the sensing component 1011 can be used to collect information about the mobile device itself or its external environment. The sensing component 1011 may include at least one of a visual sensing unit, radar, a positioning and navigation unit, an inertial measurement unit (IMU), an odometer, or other sensing units. The visual sensor unit may include one or more cameras, the radar may include at least one of lidar, millimeter-wave radar, ultrasonic radar, or other radar types, and the positioning and navigation unit may include at least one of a GPS system, a BeiDou system, or other global positioning systems. The sensing component can collect environmental data related to the mobile device. For example, the environmental data may include one or more of the following: image data collected by the visual sensing unit, point cloud data collected by the radar, inertial data collected by the IMU, location data collected by the positioning and navigation unit, and device mileage data collected by the odometer.

[0048] In some examples, the computing platform 1012 may include a computing-capable device for processing the sensing information collected by the sensing component 1011 to obtain control information, and sending corresponding control commands to the execution component 1013 to cause the execution component 1013 to perform corresponding actions, thereby realizing the control of the mobile device 101. For example, the computing platform 1012 can perform one or more of the following actions on the mobile device: information collection and processing, positioning, decision-making, planning, and control, thereby realizing the autonomous control of the mobile device. The computing platform 1012 may include at least one processor and at least one memory, wherein each processor can individually or jointly execute instructions stored in the memory to implement the methods provided in the embodiments of this disclosure. The processor in this disclosure embodiment may include at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), Tensor Processing Unit (TPU), Data Processing Unit (DPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), System on Chip (SOC), Application Specific Integrated Circuit (ASIC), Micro Controller Unit (MCU), or other processors. The memory may be implemented using any type of volatile or non-volatile computer-readable storage medium or a combination thereof. In addition to storing instructions, the memory may also store data, such as map data, image data, sound data, text data, configuration parameters of the mobile device, location, orientation, speed, etc. The data stored in the memory can be accessed and used by the processor.

[0049] In some examples, the computing platform of a mobile device can perform computing tasks independently or communicate with a server to complete computing tasks. For example, the computing platform of a mobile device can cooperate with a server to complete corresponding computing tasks. These computing tasks can include any task performed to achieve autonomous control of the mobile device, such as information collection and processing, positioning, decision-making, planning, or control of the mobile device.

[0050] The computing platform 1012 can be located in the mobile device 101. Some or all of the computing platform 1012 can also be located in the server corresponding to the mobile device. For example, some functions of the computing platform 1012 with high real-time requirements can be located in the mobile device, while other functions with low real-time requirements can be located in the server corresponding to the mobile device.

[0051] In some examples, the execution component 1013 is used to perform corresponding actions based on the control of the computing platform 1012, enabling the mobile device 101 to complete the movement task. The execution component 1013 may include, for example, a power component, a braking component, a transmission component, a steering component, etc.

[0052] It should be noted that, Figure 2 The structure of the mobile device 101 shown is merely illustrative. The mobile device in this embodiment is not limited to the above structure and may include more or fewer components as needed. The device may also be combined or disassembled. For example, the mobile device may not include the aforementioned computing platform. Furthermore, the mobile device may also include communication components, interface components, multimedia components, input components, output components, display components, etc.

[0053] In some embodiments of this disclosure, the mobile device may be configured with a target perception system. This system can process environmental data collected by the perception components to identify various environmental objects such as lane lines, vehicles, and pedestrians in the surrounding environment, thereby enabling motion control of the mobile device. The perception results of linearly extending environmental objects (which can be referred to as linear objects), such as lane lines, lane center lines, road boundary lines, curbs, and road fences, can provide crucial geometric constraints and semantic information for functional modules such as device positioning, path planning, trajectory tracking, and lateral control. Therefore, the accuracy of linear object perception is of great significance to the motion control of the mobile device.

[0054] In related technologies, to verify the accuracy of lane line perception, the accuracy of lane line perception results can be evaluated based on methods such as manual observation or curve fitting. For example, the lane lines perceived by the perception system can be compared with the actual ground truth lane lines to obtain the curve overlap between the two, thus obtaining the verification result of lane line perception. For example, a cubic polynomial fitting method can be used to fit curves to the baseline ground truth data and the actual perceived data respectively, and then the performance of the perception system can be evaluated by comparing the differences between the fitted curves. However, perception verification based on this method suffers from insufficient accuracy and low efficiency. For example, this verification method has several drawbacks. First, the fitting direction is unidirectional. The cubic curve is measured in a single direction within the vehicle's Cartesian coordinate system, while the lane tangent or normal in the world coordinate system changes with curvature. This single-axis error cannot reflect the true geometric deviation. Second, this method is insensitive to curves and complex shapes. In high-curvature, bifurcation, or occlusion scenarios, the polynomial smooths out local deviations, masking key error peaks and local mismatches. Third, this method is temporally unstable. Independent fitting of each frame leads to parameter jitter, causing errors to jump between frames, making it difficult to assess temporal stability and persistent deviations. Fourth, substituting parameter differences for geometric matching after fitting can easily bring some missed or false detections closer together. This lack of strict matching leads to statistical distortion of True Positive (TP), False Positive (FP), and False Negative (FN) cases, affecting the accuracy and reliability of the perception verification results. Therefore, improving the accuracy of perception verification results for linear objects is a pressing issue.

[0055] To address the problems in related technologies, embodiments of this disclosure provide a method such as... Figure 3 The sensing system verification method is shown. This sensing system verification method can be derived from... Figure 1 The illustrated mobile device and / or server can execute this, or it can be executed by any computing device. For example... Figure 3 As shown, the perception system verification method of this embodiment may include the following steps S310 to S330.

[0056] Step S310: Based on the environmental data collected by the mobile device, determine a candidate sensing data set including multiple candidate sensing points and a candidate reference data set including multiple candidate reference points.

[0057] In some examples, the candidate sensing dataset can be a dataset obtained after processing environmental data using a target sensing system based on a mobile device. Each candidate sensing point in the dataset can be used to represent one of multiple data points representing a sensing object. For example, the mobile device can be, as shown below: Figure 1 or Figure 2 The mobile device shown. This mobile device can be accessed via... Figure 2 The sensing components shown collect environmental data. This environmental data may include one or more of the following: image data collected by the visual sensing unit, point cloud data collected by radar, inertial data collected by the IMU, position data collected by the positioning and navigation unit, and device mileage data collected by the odometer. The target perception system of this mobile device can process the environmental data to generate a raw sensing data set containing multiple raw sensing points. This raw sensing data set can be converted into the aforementioned candidate sensing data set, or it can be resampled to obtain the aforementioned candidate sensing data set. This resampling process can be either densification or sparsification. For example, sparse data points in the raw sensing data set can be transformed into dense candidate data points with a certain sampling density; conversely, dense data points in the raw sensing data set can be transformed into sparse candidate data points with a certain sampling density.

[0058] The candidate sensing data set may include multiple candidate sensing points. Each candidate sensing point can be used to represent one of multiple data points representing a sensing object. The sensing object can be an object with a linear extension shape, such as a lane line, lane center line, road boundary line, curb, or road fence (which can be called a linear object). For example, each candidate sensing point can be represented by candidate sensing point information. The candidate sensing point information corresponding to a candidate sensing point may include the location information and / or acquisition time of the candidate sensing point. The location information may include the coordinate information of the candidate sensing point in the device coordinate system or world coordinate system. The acquisition time may be the timestamp information of the environmental data used to generate the candidate sensing point set. The candidate sensing point information may also include the object type information and / or object attribute information of the sensing object corresponding to the candidate sensing point. The object type can be used to represent the classification of the sensing object, such as lane line, lane center line, road boundary line, curb, or road fence. The object attribute information may include one or more of the following attributes: object identifier, object function, line type, color, etc.

[0059] In some examples, this candidate benchmark dataset can be used to verify the accuracy of the candidate sensing dataset. Each candidate benchmark point in the candidate benchmark dataset can be one of multiple data points characterizing a benchmark object, which can be an object with a linearly extended shape. This candidate benchmark dataset can also be called a candidate ground truth dataset, and the candidate benchmark points can also be called candidate ground truth points. The candidate benchmark points in this candidate benchmark dataset can serve as ground truth data for verifying the accuracy of each candidate sensing point in the candidate sensing dataset.

[0060] For example, the candidate benchmark dataset can be generated by a benchmark data generation system based on the aforementioned environmental data. This benchmark data generation system may include a manual annotation system and / or an automatic annotation system to generate an original benchmark dataset comprising multiple original benchmark points based on the environmental data. This original benchmark dataset can be used to obtain the aforementioned candidate benchmark dataset, or the original benchmark dataset can be resampled to obtain the aforementioned candidate benchmark dataset. This resampling process can be either intensive or sparse processing; specific methods can be referred to the foregoing description and will not be repeated here. It should be noted that resampling the original sensing dataset and the original benchmark dataset can ensure that the resulting candidate benchmark dataset and candidate sensing dataset have the same sampling density, thereby improving the efficiency and accuracy of subsequent data point matching to determine the target point set.

[0061] Furthermore, the aforementioned automatic annotation system can be configured with a highly complex and fully trained automatic annotation model. Based on this automatic standard model, the environmental data is processed to obtain a first benchmark dataset. This first benchmark dataset can be used as the aforementioned original benchmark dataset. Alternatively, the first benchmark dataset can be further manually verified and modified by a manual annotation system to generate a second benchmark dataset. This second benchmark dataset can then be used as the aforementioned original benchmark dataset.

[0062] In some examples, each candidate reference point in the aforementioned candidate reference data set can be represented by candidate reference point information. The candidate reference point information corresponding to a candidate reference point may include the location information and / or acquisition time of that candidate reference point. The location information may include the coordinates of the candidate reference point in the device coordinate system or world coordinate system, and the acquisition time may be the timestamp information of the environmental data collected to generate the candidate reference point set. The candidate reference point information may also include the object type information and / or object attribute information of the reference object corresponding to that candidate reference point. The object type can be used to characterize the classification of the reference object, such as lane lines, lane center lines, road boundary lines, curbs, road fences, etc.; the object attribute information may include one or more of the following attributes: object identifier, object function, line type, color, etc.

[0063] It should be noted that multiple candidate sensing points in a candidate sensing dataset can constitute one or more sensing objects, and multiple candidate reference points in a candidate reference dataset can also constitute one or more reference objects. There can be a partial or complete correspondence between sensing objects and reference objects.

[0064] In some examples, the environmental data mentioned above may include multiple frames of data, each frame corresponding to a specific time point. Mobile devices can collect multiple frames of environmental data at different times, and the target perception system can process each frame to generate a corresponding set of candidate perception data. A benchmark data generation system can generate a corresponding set of candidate benchmark data for each frame of environmental data. This allows for the verification of the target perception system in the temporal dimension.

[0065] In some examples, mobile devices can acquire environmental data through multi-sensor fusion. For instance, a mobile device can acquire camera image data, LiDAR point cloud data, IMU inertial data, and GNSS location data in parallel, and then synchronize and spatially align these data to generate fused environmental data. The target perception system can then perform perception processing based on this fused environmental data, improving the accuracy and robustness of the perception results.

[0066] Step S320: Determine one or more target point sets based on the distance between the candidate reference points and the candidate sensing points in the target direction.

[0067] Each set of target points may include a target reference point determined from a plurality of candidate reference points and a target sensing point corresponding to the target reference point determined from a plurality of candidate sensing points.

[0068] In some examples, this target direction can also be called the projection direction or key direction, and the distance between the candidate reference point and the candidate sensing point in this target direction can be the projected distance along this target direction. For example, the connection vector between the candidate reference point and the candidate sensing point can be projected onto the target direction, and the absolute value of the projected component can be taken as the distance. In this way, the interference of offset components in non-target directions on the distance measurement can be effectively eliminated, and only the relative positional difference in the target direction can be retained, thereby accurately measuring the geometric deviation of the sensing point relative to the reference point in the target direction and improving the accuracy of matching between the candidate reference point and the candidate sensing point.

[0069] In this step, point-to-point matching can be performed at least based on the distance between candidate reference points and candidate sensing points in the target direction to obtain one or more sets of target points. Each set of target points can include successfully matched target reference points and target sensing points. For example, a target sensing point can be the candidate sensing point with the smallest distance between multiple candidate sensing points and the target reference point in the target direction.

[0070] In some examples, different candidate reference points can use the same target direction, which can be a pre-defined direction, such as a direction set by the user based on engineering experience, or a longitudinal direction (x-axis direction) and a lateral direction (y-axis direction) set based on the device coordinate system of the mobile device. Different candidate reference points can use the same target direction.

[0071] In other examples, the target direction for each candidate reference point can be set individually. For example, the target direction can be determined based on the extension direction of a reference object at the candidate reference point. The reference object can be an object with a linear extension shape, and the extension direction can characterize the direction in which the reference object extends at the candidate reference point. The angle between the target direction and the extension direction can be greater than or equal to a preset angle threshold, which can be a pre-set threshold, such as 45 degrees or 50 degrees. In one implementation, the target direction can be a normal direction perpendicular to the extension direction. In another implementation, the target direction can be determined based on the extension direction and pre-set candidate reference directions. The target direction can be the candidate reference direction with the smallest angle between it and the normal direction of the extension direction among at least two candidate reference directions. For example, the normal direction perpendicular to the extension direction can be determined first, and the candidate reference direction with the smallest angle between it and the normal direction can be determined from at least two candidate reference directions as the target direction, so that the target direction is the direction in which the angle between at least two candidate reference directions and the extension direction is closest to a right angle.

[0072] In this way, determining the target direction based on the extension direction can adapt to complex scenarios such as curves. The distance along the target direction can filter out redundant error interference along the lane line extension direction, more accurately measuring the geometric deviation between the sensing point and the reference point, thus improving the accuracy of data point matching. Furthermore, determining the target direction based on the extension direction and pre-defined candidate reference directions avoids the problem of continuous changes in the normal direction (projection direction) caused by curvature variations, improving the efficiency and accuracy of data point matching.

[0073] For example, taking a first reference point among the candidate reference points as an example, the method for determining the target direction of the first reference point may include: determining the tangent direction of the first reference point, using the tangent direction as the extension direction of the reference object at the first reference point, and determining the target direction of the first reference point based on the extension direction. Here, the extension direction represents the orientation of the reference object at the first reference point. It should be noted that the first reference point can be any one of multiple candidate reference points.

[0074] In this example, the determination of the tangent direction of the first reference point may include: identifying a second reference point and a third reference point adjacent to the first reference point from among multiple candidate reference points; and calculating the tangent direction of the first reference point based on the coordinates of the first reference point, the second reference point, and the third reference point using a preset tangent direction determination algorithm. The second reference point may be a backward neighbor of the first reference point, and the third reference point may be a forward neighbor of the first reference point. The forward and backward directions can be determined based on the extension direction of the reference object corresponding to the candidate reference point; for example, points located in front of the extension direction of the first reference point are forward neighbors, and points located behind the extension direction of the first reference point are backward neighbors.

[0075] The index order of the candidate reference points can be consistent with the extension direction of the reference object. For example, if the first reference point is the i-th candidate reference point Q... i Then the second reference point can be the (i-1)th candidate reference point Q. i-1 The third reference point can be the (i+1)th candidate reference point Q. i+1 Optionally, the second reference point can be one of the K candidate reference points that are backward adjacent to the first reference point, and the third reference point can be one of the K points that are forward adjacent to the first reference point, where K can be an integer greater than or equal to 1. The preset tangent direction determination algorithm in this example can be, for example, the central difference method, the forward difference method, the backward difference method, the sliding window least squares fitting method, etc., and this example does not limit it.

[0076] In this way, the tangent direction is calculated based on the coordinates of the first reference point and its adjacent reference points, and the tangent direction is used as the extension direction of the reference object at that point. This allows the local orientation of the linear object to be determined independently point by point without relying on global curve fitting. Thus, the extension direction at each reference point can still be accurately obtained under complex forms such as bifurcation, occlusion, and abrupt curvature changes, avoiding the smooth masking of local geometric features by polynomial fitting.

[0077] In this example, there are several ways to determine the target direction based on the extension direction. For example, the normal direction perpendicular to the tangent direction can be used as the target direction; another example is to determine a candidate reference direction from at least two candidate reference directions as the target direction of the first reference point based on the extension direction. The target direction can be the direction in which the angle between the at least two candidate reference directions and the extension direction is closest to a right angle.

[0078] The candidate reference direction can be implemented in several ways. In one example, the candidate reference direction can be any pre-defined direction. For instance, the entire 360-degree radius can be divided into N sectors to generate N candidate reference directions, ensuring that the included angles between adjacent candidate reference directions are equal. N can be an integer greater than or equal to 2. In another example, the candidate reference direction can be determined based on the pose information of the mobile device when acquiring environmental data. For example, the longitudinal direction (x-axis) and lateral direction (y-axis) of the mobile device's coordinate system can be determined based on the pose information, and the target direction can be determined based on these longitudinal and lateral directions. Furthermore, these longitudinal and lateral directions can be used as the target direction. Alternatively, the entire 360-degree radius can be divided into M sectors based on the longitudinal and lateral directions to obtain M candidate reference directions. These M candidate reference directions can include both longitudinal and lateral directions, and M can be an integer greater than or equal to 2.

[0079] For example, determining the target direction of the first reference point based on the extension direction may include: determining at least two candidate reference directions based on the pose information of the mobile device when collecting environmental data; determining one candidate reference direction from the at least two candidate reference directions as the target reference direction of the first reference point based on the extension direction; and using the direction perpendicular to the target reference direction as the target direction of the first reference point. Wherein, the target angle between the target reference direction and the extension direction is the minimum value of the angles between the at least two candidate reference directions and the extension direction.

[0080] For example, the pose information may include the travel direction, velocity vector, attitude angle, etc. of the mobile device, and the device coordinate system of the mobile device can be determined based on the pose information. The at least two candidate reference directions may include the longitudinal direction (x-axis direction) and the lateral direction (y-axis direction) in the device coordinate system. For example, the first candidate reference direction may be the longitudinal direction in the device coordinate system, and the second candidate reference direction may be the lateral direction in the device coordinate system. The angle α between the extending direction and the first candidate reference direction, and the angle β between the extending direction and the second candidate reference direction can be calculated. If α is less than or equal to β, the first candidate reference direction is selected as the target reference direction, and the second candidate reference direction perpendicular to the target reference direction is selected as the target direction; if β is less than α, the second candidate reference direction is selected as the target reference direction, and the first candidate reference direction perpendicular to the target reference direction is selected as the target direction.

[0081] In this way, by determining at least two candidate reference directions based on the pose information of the mobile device, and selecting the candidate reference direction whose angle with the extension direction is closest to a right angle as the target direction, the target direction can be automatically selected as the lateral direction in straight road scenarios and dynamically adjusted to a candidate reference direction that is closer to the normal direction in curved road scenarios. This allows the geometric deviation between the sensing point and the reference point to be measured in a direction close to the normal on road segments with different curvatures, thereby improving the geometric accuracy of the evaluation results in complex road network scenarios such as curves.

[0082] In some examples, this step may determine one or more target point sets in the following ways: for each candidate reference point, if there is at least one first sensing point among the multiple candidate sensing points in the candidate sensing data set that satisfies a preset matching condition, then the candidate reference point may be used as the target reference point, and the first sensing point with the smallest target distance from the target reference point in the target direction among the at least one first sensing point may be determined as the target sensing point corresponding to the target reference point, and a target point set may be generated based on the target reference point and the target sensing point.

[0083] For example, the preset matching conditions may include: the straight-line distance between the first sensing point and the candidate reference point is less than or equal to a preset distance threshold, and / or, the sensing object corresponding to the first sensing point and the reference object corresponding to the candidate reference point have the same object type. The preset distance threshold can be set according to the actual application scenario; for example, it can be pre-determined by the user based on engineering experience, and the value range of the preset distance threshold can be from 0.5 meters to 3 meters. If the straight-line distance between the first sensing point and the candidate reference point is greater than the preset distance threshold, it is considered that the first sensing point and the candidate reference point are too far apart, and no matching is performed. The aforementioned object type may include lane line type, road boundary type, etc. If the sensing object corresponding to the first sensing point is a lane line type, and the reference object corresponding to the candidate reference point is a road boundary type, it is considered that the object types of the first sensing point and the candidate reference point are inconsistent, and no matching is performed. The above two conditions in the preset matching conditions can be used individually or in combination.

[0084] In this way, by pre-screening candidate sensing points with preset matching conditions, only candidate sensing points whose straight-line distance meets the threshold requirement and / or whose object type is consistent can be retained to participate in the subsequent target direction distance calculation. This reduces the amount of computation while effectively filtering out candidate sensing points that are too far away or whose types are inconsistent, avoiding cross-type incorrect matching and improving the accuracy and reliability of the matching results.

[0085] Step S330: Generate verification results for the target perception system based on the target point set.

[0086] In some examples, based on the target point set, unmatched baseline data points in the candidate baseline data set and unmatched sensing data points in the candidate sensing data set can be identified; the verification result of the target sensing system can be calculated based on the target point set, unmatched sensing data points, and unmatched baseline data points.

[0087] Among them, unmatched reference data points can correspond to missed detections, that is, points on the reference object that the target perception system failed to detect. Unmatched sensing data points can correspond to false detections, that is, points that the target perception system detected but do not have a corresponding point in the reference data.

[0088] In some examples, point-level evaluation metrics can be calculated based on the target point set, unmatched perceived data points, and unmatched baseline data points, and these metrics can be used as at least part of the validation results. These point-level evaluation metrics can include one or more of the following: point-level precision, point-level recall, and point-level accuracy error. Specifically: point-level precision characterizes the percentage of points correctly perceived by the target perception system, and can be calculated as the ratio of the number of target points to (the number of target points plus the number of unmatched perceived data points). For example, if the number of target points is 90 and the number of unmatched perceived data points is 10, then the point-level precision is 90 divided by (90 plus 10) equals 0.9. Point-level recall characterizes the percentage of points in the baseline data correctly perceived by the target perception system, and can also be calculated as the ratio of the number of target points to (the number of target points plus the number of unmatched baseline data points). For example, if the target point set has 90 points and the number of unmatched reference data points is 15, then the point-level recall rate is approximately 0.857 (90 divided by (90 plus 15)). Point-level precision error can characterize the geometric deviation between the target perceived point and the target reference point.

[0089] In this way, by distinguishing between the set of successfully matched target points and the data points that were not matched, TP, FN, and FP can be accurately counted separately. The number of target points corresponds to TP, the number of unmatched perceived data points corresponds to FP, and the number of unmatched baseline data points corresponds to FN. This enables accurate calculation of point-level precision and recall, avoiding the statistical distortion of TP, FP, and FN caused by substituting geometric matching with parameter differences in related technologies, and improving the statistical accuracy of validation metrics.

[0090] In some examples, object-level statistical indicators for each benchmark object can be calculated based on the target point set, unmatched perceived data points, and unmatched benchmark data points. These object-level statistical indicators can be used as at least part of the verification results. These object-level statistical indicators can also be called line-level statistical indicators.

[0091] For example, object-level statistical metrics may include one or more of the following: object-level accuracy, object-level recall, detection interruption rate, and morphological anomaly rate. Object-level accuracy reflects how many linear objects the target perception system correctly detects. Object-level recall reflects how many linear objects in the baseline data are correctly detected by the target perception system. Detection interruption rate reflects the frequency of discontinuous detection in lane line perception results, i.e., the situation where a baseline object is segmented into multiple segments in the perception results. Morphological anomaly rate reflects the proportion of local morphological anomalies in the perception results, i.e., the situation where the perceived object deviates excessively from the baseline object in certain areas.

[0092] For example, the benchmark object identifier associated with each candidate benchmark point can be used as the aggregation basis to aggregate the target point set, unmatched sensing data points, and unmatched benchmark data points belonging to the same benchmark object, and calculate the object-level statistical index of the benchmark object. For example, if a benchmark object contains 100 candidate benchmark points, of which 90 candidate benchmark points are successfully matched to form a target point set, and 10 candidate benchmark points are not matched to form unmatched benchmark data points, and the sensing object corresponding to the benchmark object contains 95 candidate sensing points, of which 90 candidate sensing points participate in matching to form a target point set, and 5 candidate sensing points are not matched to form unmatched sensing data points, then the object-level recall rate of the benchmark object is 90 divided by (90 plus 10) equals 0.9, and the object-level precision rate of the corresponding sensing object is 90 divided by (90 plus 5) approximately equals 0.947.

[0093] In this way, by aggregating from the point level to the object level, the ability of the perception system to detect complete linear objects can be evaluated from a macroscopic level, and object-level errors such as perception interruption and excessive local error can be identified, thereby enabling a more comprehensive evaluation of the performance of the target perception system.

[0094] In some examples, this step can also generate temporal statistical indicators based on the target point set corresponding to multiple frames of environmental data, and use these temporal statistical indicators as at least part of the verification results. Temporal statistical indicators may include inter-frame jitter, temporal detection stability indicators, etc. Inter-frame jitter can characterize the magnitude of change in the perception results of the target perception system between adjacent frames, while temporal detection stability indicators can characterize the stability and consistency of the target perception system in the temporal dimension.

[0095] By employing the methods described in steps S310 to S330, a target point set is constructed through point-by-point matching between candidate reference points and candidate sensing points. This method preserves the independent error information of each sampling point, avoids the smoothing masking of local deviations by polynomial curve fitting, and sets the target direction of each candidate reference point to the direction determined by the extension direction of the reference object at that point. The distance between the candidate sensing point and the candidate reference point in this target direction is used as the basis for matching and error measurement. This allows the target direction to dynamically change with the local orientation of the linear object, thus accurately reflecting the true geometric deviation of the sensing point relative to the reference point in various road network scenarios such as straight roads, curves, and ramps. Compared with the methods of measuring using straight-line distance or fixed direction in related technologies, the method of this embodiment can avoid the geometric deviation distortion problem caused by high curvature scenarios in related technologies, and improve the verification accuracy and reliability of the target sensing system's linear object sensing results in complex road network scenarios.

[0096] In some embodiments of this disclosure, step S330 may, for each set of target points, determine the reference object attribute information of the reference object corresponding to the target reference point, and the perception object attribute information of the perception object corresponding to the target perception point; if it is determined that the perception object attribute information and the reference object attribute information do not meet the preset attribute matching verification rules, the set of target points is determined to be an attribute error set; based on the number of attribute error sets and the number of target point sets, an attribute perception error index is calculated, and the attribute perception error index is used as at least a part of the verification result.

[0097] In some examples, the attribute matching verification rule may include: the attribute information of the perceived object is the same as the attribute information of the reference object. The same attribute information can include: the perceived object and the reference object having the same line type, color, or function type. For example, if the reference object's line type is solid and the perceived object's line type is dashed, then the line types are considered inconsistent, and the attribute matching verification rule is not met. If the reference object's color is white and the perceived object's color is yellow, then the colors are considered inconsistent, and the attribute matching verification rule is not met. If the reference object's function type is a double yellow line separating up and down traffic, and the perceived object's function type is a regular lane line, then the function types are considered inconsistent, and the attribute matching verification rule is not met.

[0098] In some examples, the attribute-aware error metric can be calculated as the ratio of the number of attribute error sets to the total number of target point sets. For instance, if the total number of target point sets is 100 and the number of attribute error sets is 5, then the attribute-aware error metric is 5%.

[0099] In this way, by further comparing the attribute information of the perceived objects in the target point set with the attribute information of the reference objects based on geometric matching, and marking the target point set that does not meet the attribute matching verification rules as the attribute error set, and then calculating the attribute perception error index, the perception accuracy of the perception system for the attributes such as line type, color, and function type of linear objects can be evaluated from the attribute dimension. This supplements the evaluation dimension other than the geometric deviation index and improves the comprehensiveness of the verification results.

[0100] In some embodiments of this disclosure, the implementation of determining the candidate sensing data set and the candidate benchmark data set based on environmental data in step S310 above may include the following steps S311 to S313.

[0101] Step S311: Based on the target perception system of the mobile device, the environmental data is processed to generate a raw perception data set including multiple raw perception points.

[0102] For example, the raw sensing data set can be the sensing results directly output by the target sensing system, where the raw sensing points can be sparsely or unevenly distributed. For instance, environmental data can be input into the target sensing system of a mobile device to obtain the raw sensing data set output by the target sensing system. This target sensing system can run online or offline; furthermore, the target sensing system can be deployed locally on the mobile device or deployed on a server or other electronic device. This embodiment does not limit this. Step S312: Obtain the original baseline data set corresponding to the environmental data. The original baseline data set includes multiple original baseline points.

[0103] For example, the original benchmark dataset can be generated by a benchmark data generation system, and the original benchmark points therein can be sparsely or unevenly distributed. The implementation of this benchmark data generation system can be found in the description of the foregoing embodiments of this disclosure, and will not be repeated here.

[0104] Step S313: Resample the original reference data set and the original sensing data set to generate a candidate reference data set including multiple candidate reference points and a candidate sensing data set including multiple candidate sensing points.

[0105] The resampling process can be either intensive or sparse; the specific method can be found in the previous description and will not be repeated here.

[0106] In some examples, resampling can be applied to both the baseline data and the sensed data simultaneously, giving them the same sampling density, thereby improving the accuracy of data point matching.

[0107] In this way, resampling can transform the original sparse or uneven data points into dense and structured candidate data points, laying the foundation for subsequent accurate matching. This improves the accuracy and robustness of the matching process.

[0108] It should be noted that the original sensing dataset obtained in step S311 can also be combined into a candidate sensing dataset, and the original benchmark dataset obtained in step S312 can be combined into a candidate benchmark dataset. Step S313 does not need to be executed.

[0109] In some examples, the resampling process for the original reference data set and the original sensed data set in step S313 above may include: determining resampling information for resampling based on the pose information of the mobile device when collecting environmental data; using the original reference data set or the original sensed data set as the original data set, and resampling the original data set based on the resampling information, each original data point in the original data set, and the original object corresponding to each original data point, to generate a candidate data set. The candidate data set may be a candidate reference data set or a candidate sensed data set; the original data point may be an original reference point or an original sensed point; the original object may be a reference object or a sensed object; the resampling information may include a first sampling direction, a second sampling direction, a first resampling distance interval corresponding to the first sampling direction, and a second resampling distance interval corresponding to the second sampling direction; the first sampling direction may be the longitudinal direction of the mobile device, and the second sampling direction may be perpendicular to the first sampling direction.

[0110] For example, the pose information mentioned above may include the travel direction, velocity vector, attitude angle, and other information of the mobile device. The first sampling direction may correspond to the longitudinal direction (x-axis direction) in the device coordinate system of the mobile device, and the second sampling direction may correspond to the lateral direction (y-axis direction). The first resampling distance interval and the second resampling distance interval can be set by the user based on engineering experience or actual needs. For example, both the first and second resampling distance intervals can be set to values ​​between 0.5 meters and 2 meters. For example, the first resampling distance interval can be set to 1 meter, and the second resampling distance interval can also be set to 1 meter. The values ​​of the first and second resampling distance intervals can be the same or different.

[0111] In this way, by determining the first sampling direction, the second sampling direction, and their corresponding resampling distance intervals based on the pose information of the mobile device when collecting environmental data, the longitudinal and lateral directions in the device coordinate system can be introduced into the resampling process, making the resampling direction match the local extension direction of the linear object. This allows it to adapt to different scenarios such as straight roads and curves. For example, it can perform equidistant sampling in a direction close to the normal (perpendicular to the extension direction) in different scenarios such as straight roads and curves, avoiding the sampling deviation caused by fixed single-direction sampling in high curvature scenarios, improving the accuracy of data point matching after data sampling, and also improving the accuracy of the perception system verification.

[0112] In some examples, the above-mentioned method of resampling the original data set based on resampling information, each original data point in the original data set, and the original object corresponding to each original data point to generate a candidate data set may include: performing target resampling processing on each original data point in the original data set to obtain sampled data points (e.g., sampled reference points or sampled sensing points) corresponding to the original data point (e.g., original reference points or original sensing points); and generating a candidate data set based on multiple original data points and the sampled data points corresponding to each original data point. For example, both the original data points and the sampled data points can be used as candidate data points in the candidate data set, such as both the original reference point and the sampled reference point being used as candidate reference points in the candidate reference data set, or both the original sensing point and the sampled sensing point being used as candidate sensing points in the candidate sensing data set. In this way, a candidate reference data set containing both the original reference point and the sampled reference point, and a candidate sensing data set containing both the original sensing point and the sampled sensing point can be obtained.

[0113] In some examples, the above-described target resampling process may be implemented by steps S11 to S13.

[0114] Step S11: Determine the extension direction and resampling interval corresponding to the original data points.

[0115] For example, the method for determining the extension direction can refer to the description in the foregoing embodiments of this disclosure. For instance, the extension direction of the original data point can be calculated based on the coordinates of the original data point and other adjacent original data points using a preset tangent direction determination algorithm.

[0116] The algorithm for determining the preset tangent direction can be, for example, the central difference method, the forward difference method, the backward difference method, the sliding window least squares fitting method, etc. This example does not limit this.

[0117] The other adjacent original data points can be the forward neighbors and / or backward neighbors of the original data points. For example, for the i-th original data point P iIts next neighbor is the (i-1)th original data point P. i-1 The forward neighbor is the (i+1)th original data point P. i+1 It should be noted that the forward neighbor point can be one of the K other original data points (e.g., P) that are forward neighbors of the original data point. i+1 P i+2 ... P i+K The backward neighbor point can also be one of the K other original data points that are backward neighbors of the original data point (e.g., P). i-1 P i-2 ... P i-K The extension direction of the original data point can be calculated based on the coordinates of the original data point and its forward and / or backward neighboring points.

[0118] The resampling interval mentioned above can be a linear interval between the original data point and other adjacent original data points. For example, the resampling interval corresponding to the i-th original data point can be from the i-th original data point P... i To the forward neighbor (i.e., the (i+1)th original data point P) i+1 A linear interval between points, or from the (i-1)th original data point P to the next adjacent point. i-1 ) to the i-th original data point P i The linear interval between them.

[0119] Step S12: Based on the extension direction, determine the direction with the smallest angle to the extension direction from the first sampling direction and the second sampling direction of the resampled information as the resampled direction corresponding to the original data point.

[0120] In some examples, the angle α between the extension direction and the first sampling direction, and the angle β between the extension direction and the second sampling direction can be calculated. If α is less than or equal to β, the first sampling direction is selected as the resampling direction; if β is less than α, the second sampling direction is selected as the resampling direction.

[0121] Step S13: Based on the resampling direction and the resampling distance interval corresponding to the resampling direction, resampling is performed within the resampling interval to obtain sampled data points.

[0122] In some examples, if the resampling direction is the first sampling direction, then the sampling distance interval is the first resampling distance interval; if the resampling direction is the second sampling direction, then the sampling distance interval is the second resampling distance interval. Starting from the original data point, a sampled data point can be generated every other sampling distance interval along the resampling direction until the end of the resampling interval is reached.

[0123] Figures 4 to 6This is a schematic diagram illustrating data point sampling in different scenarios provided by embodiments of this disclosure. In different scenarios, the longitudinal direction (x-axis direction) and the lateral direction (y-axis direction) can be determined based on the device coordinate system of the mobile device. The longitudinal direction can be the current forward direction of the mobile device, and the lateral direction can be a direction perpendicular to the longitudinal direction and pointing to the left side of the mobile device. Data point sampling is performed based on the longitudinal direction and / or the lateral direction.

[0124] Figure 4 This is a schematic diagram of data point sampling in a straight-line scene provided by an embodiment of this disclosure. For example... Figure 4 As shown, in a straight-line scene, the blue line represents the reference object, and the red line represents the sensing object. The reference object can be represented based on multiple original reference points, such as original reference point A on the blue line (shown as the blue dot). The sensing object can be represented based on multiple original sensing points, such as original sensing point B on the blue line (shown as the red dot). The arrow at original sensing point B in the figure indicates the extension direction of the sensing object. Based on this extension direction, the resampling direction can be determined as the longitudinal direction, i.e., the resampling direction can be selected as a longitudinal direction close to this extension direction (e.g., the first sampling direction). Based on the extension direction of the reference object (not shown in the figure), the resampling direction of the reference object can also be determined as the longitudinal direction. Based on the resampling direction and the pre-set resampling interval, the resampling cross-section shown by the horizontal dashed line can be obtained. Based on the intersection of this resampling cross-section and the reference object or sensing object, the sampled data points (e.g., sampled reference points or sampled sensing points) corresponding to the original data points (e.g., original reference points or original sensing points) can be obtained, thus obtaining a candidate reference data set containing original reference points and sampled reference points, and a candidate sensing data set containing original sensing points and sampled sensing points.

[0125] Figure 5 This is a schematic diagram of data point sampling in a curved scene provided by an embodiment of this disclosure. Figure 5 As shown, in a curve scenario, the blue curve represents the reference object, and the red curve represents the sensing object. The reference object can be represented based on multiple original reference points, such as original reference points A1 and A2 on the blue curve (shown as blue dots). The sensing object can be represented based on multiple original sensing points, such as original sensing points B1 and B2 on the red curve (shown as red dots). The arrows in the diagram indicate the extension directions of the sensing object at original sensing points B1 and B2, respectively.

[0126] For the original sensing point B1, if the direction of extension of the sensing object at that point is close to the longitudinal direction, then the first resampling direction is determined to be the longitudinal direction based on this extension direction. That is, the first resampling direction can be selected from a longitudinal direction close to this extension direction (e.g., the first sampling direction). Based on the resampling direction and a pre-set resampling interval, the resampling cross-section shown by the horizontal dashed line can be obtained. Based on the intersection of this resampling cross-section with the reference object or the sensing object, the sampled sensing points corresponding to the original sensing point B1 can be obtained, thus obtaining a candidate sensing data set containing the original sensing point B1 and its corresponding sampled sensing points. A candidate reference data set can also be obtained in the same way.

[0127] For the original sensing point B2, if the direction of extension of the sensing object at that point is close to the lateral direction, then the second resampling direction is determined to be the lateral direction based on this extension direction. That is, the second resampling direction can be selected from any lateral direction close to the extension direction (e.g., the second sampling direction). Based on the resampling direction and the pre-set resampling interval, the resampling cross-section shown by the vertical dashed line can be obtained. Based on the intersection of this resampling cross-section with the reference object or the sensing object, the sampled sensing points corresponding to the original sensing point B2 can be obtained, thus obtaining a candidate sensing data set containing the original sensing point B2 and its corresponding sampled sensing points. A candidate reference data set can also be obtained in the same way.

[0128] Thus, as Figure 5 As shown, in a curved road scenario, as the road curves and the extension direction of target objects such as lane lines changes, the resampling direction is also dynamically adjusted, thereby improving the precision and flexibility of sampling. This avoids the geometric deviation distortion problem caused by related technologies in high curvature scenarios and enhances the verification accuracy and reliability of the target perception system's linear object perception results in complex road network scenarios.

[0129] Figure 6 This is a schematic diagram of another curve scene data point sampling provided in an embodiment of this disclosure. For example... Figure 6As shown, in scenarios involving curves or complex trajectories, the black curve represents a linearly extending target object (which could be a reference object or a perceived object). This target object includes multiple raw data points indicated in red, such as a first raw data point, a second raw data point, and a third raw data point. For the first raw data point, its resampling direction can be determined as the longitudinal direction based on the tangent direction of the target object at that point. The resampling interval is set as the curve range between the first and second raw data points. Based on the resampling direction, the resampling interval, and the first resampling distance interval, the sampled data points indicated in orange between the first and second raw data points can be sampled. For the second raw data point, its resampling direction can be determined as the lateral direction based on the tangent direction of the target object at that point. The resampling interval is set as the curve range between the second and third raw data points. Based on the resampling direction, the resampling interval, and the second resampling distance interval, the sampled data points indicated in orange between the second and third raw data points can be obtained. Furthermore, a candidate data set containing the raw data points indicated in red and the sampled data points indicated in orange can be obtained.

[0130] The first resampling interval corresponds to the sampling interval in the longitudinal direction (x-axis direction), and the second resampling interval corresponds to the sampling interval in the lateral direction (y-axis direction). In straight-line scenarios, the extension direction is close to the longitudinal direction, so the resampling direction is selected in the lateral direction, and the sampling interval is the second resampling interval. In curved-line scenarios, the extension direction changes with the curvature, and the resampling direction is dynamically adjusted accordingly, with the sampling interval switching between the first and second resampling intervals. By using resampling information in both directions, the resampling direction can be dynamically selected based on the local extension direction of the linear object, adapting to different scenarios such as straight and curved lines, and improving the accuracy and adaptability of resampling.

[0131] In some embodiments of this disclosure, after resampling, the data point information corresponding to each sampled data point may include the location information of the sampled data, the object type information of the corresponding perceived object, the object attribute information, etc., so that the sampled data point can be associated with the original object for subsequent matching and aggregation operations.

[0132] Using the above method, in a curved scene, the extension direction changes with the curvature, and the resampling direction is also dynamically adjusted accordingly. By dynamically selecting the resampling direction with the smallest angle to the extension direction, the sampling points can accurately reflect the local geometric features of the linear object, avoid the smoothing distortion caused by polynomial fitting, and improve the accuracy and robustness of sampling.

[0133] Figure 7 This is a flowchart illustrating another sensing system verification method provided in an embodiment of this disclosure. Figure 7As shown, the method may include steps S710 to S740.

[0134] Step S710: Obtain environmental data.

[0135] For example, the environmental data could be environmental data collected by a mobile device. This includes, for instance, environmental data collected by a mobile device using sensors such as a camera, odometer, IMU, or LiDAR.

[0136] Step S720: Determine the candidate sensing data set and the candidate baseline data set based on the environmental data.

[0137] For example, in this step, environmental data can be input into both the baseline data generation system and the target perception system. The baseline data generation system can generate a raw baseline data set including multiple raw baseline points based on the environmental data, and the target perception system can generate a raw perception data set including multiple raw perception points based on the environmental data. For example, the raw baseline points and raw perception points have similar data formats and can be referred to as raw data points. The storage format of these raw data points can be: T represents the time when the environmental data for the original data point was collected, and k represents the object identifier of the linear object. This represents the information of the i-th original data point in the linear object identified as k obtained at time T, which may include the horizontal and vertical coordinates of that point in the device coordinate system. The object type of the target object corresponding to this point ( ), logo ( ),Function( Information such as lane lines can be included, and if the target object is a lane line, it can include the type, identification, and function of the lane line.

[0138] Furthermore, in this step, the original reference data set and the original sensing data set can be preprocessed based on the data preprocessing module. This preprocessing can include operations such as tangent direction calculation, resampling direction determination, and resampling processing, and output candidate reference data set and candidate sensing data set.

[0139] Methods for calculating the tangent direction can include: applying the original data points to... , can be based on Calculate its tangent direction on the curve. .

[0140] The resampling direction can be determined by: based on the tangent direction And set a threshold to determine the resampling direction for that point.

[0141] Resampling can be done through raster sampling, for example, by obtaining the sampling coordinates of the point on a predefined raster based on the resampling direction.

[0142] Thus, by preprocessing the original baseline dataset and the original sensed dataset using the data preprocessing module, the original data points can be obtained. sampling point set ,in In addition to containing the original data point information, it can also contain the key orientation corresponding to the point and its corresponding original object identifier (ID).

[0143] It should be noted that the specific implementation method of step S720 can also be found in [reference needed]. Figure 3 The description of step S310 in the illustrated embodiment will not be repeated here.

[0144] Step S730: Generate one or more target point sets through data matching.

[0145] For example, in this step, distance calculation, matching point pair extraction, matching point pair attribute checking and filtering can be performed based on the candidate baseline data set and the candidate sensing data set, and one or more target point sets can be output. Unmatched baseline data points and / or unmatched sensing data points can also be output.

[0146] The distance calculation and matching point pair extraction methods may include: based on the target direction of the i-th candidate reference point, calculating the straight-line distance and the projected distance in the target direction from all candidate sensing points to the i-th candidate reference point, taking the point with the straight-line distance less than the threshold and the smallest projected distance as the matching pair of the i-th candidate reference point; repeating this process to obtain the potential matching targets of all candidate reference points.

[0147] The methods for checking and filtering matching point pairs can include: checking the attributes of the matching targets based on set rules to filter out unreasonable matching pairs. For example, it can include: attribute conflict checking, such as lane line features and road boundary features (curbstones, road fences, etc.) should not be matched; lane function conflict checking, such as double yellow lines that separate up and down lanes should not be matched with ordinary lane lines.

[0148] It should be noted that the specific implementation of step S730 can also be found in [reference needed]. Figure 3 The description of step S320 in the illustrated embodiment will not be repeated here.

[0149] Step S740: Generate verification results for the target perception system based on the target point set.

[0150] For example, in this step, based on the target point set and any possible unmatched baseline data points and unmatched sensing data points, one or more operations such as point-level statistics, object-level statistics, and temporal statistics can be performed to generate verification results for the target sensing system. These verification results can include one or more of the following: point-level statistical results, object-level statistical results, and temporal statistical results. Point-level statistical results can include one or more of the following metrics: calculated point-level accuracy, recall, precision error, attribute accuracy, type accuracy, and color accuracy. Object-level statistical results can include one or more of the following metrics: object-level accuracy, object-level recall, detection interruption rate, and morphological anomaly rate, calculated based on object identifier aggregation. Temporal statistical results can include one or more of the following metrics: inter-frame jitter and temporal detection stability, calculated based on object identifier and temporal information aggregation.

[0151] It should be noted that the specific implementation of step S740 can also be found in [reference needed]. Figure 3 The description of step S330 in the illustrated embodiment will not be repeated here.

[0152] By employing the methods described in steps S710 to S740, a target point set is constructed through point-by-point matching between candidate reference points and candidate sensing points. This method preserves the independent error information of each sampling point, avoids the smoothing masking of local deviations by polynomial curve fitting, and sets the target direction of each candidate reference point to the direction determined by the extension direction of the reference object at that point. The distance between the candidate sensing point and the candidate reference point in this target direction is used as the basis for matching and error measurement. This allows the target direction to dynamically change with the local orientation of the linear object, thus accurately reflecting the true geometric deviation of the sensing point relative to the reference point in various road network scenarios such as straight roads, curves, and ramps. Compared with the methods in related technologies that use straight-line distance or fixed direction for measurement, the method in this embodiment avoids the geometric deviation distortion problem caused by high curvature scenarios in related technologies, and improves the verification accuracy and reliability of the target sensing system's linear object sensing results in complex road network scenarios.

[0153] This disclosure also provides a mobile device that can be deployed with a target perception system, which can be a perception system verified based on the perception system verification method in the foregoing embodiments of this disclosure.

[0154] For example, validated perception systems can be deployed in mobile devices to ensure the accuracy and reliability of perception during actual operation. Validated perception systems can accurately perceive linear objects such as lane lines and road boundaries in various scenarios, providing reliable input for functions such as localization, path planning, trajectory tracking, and lateral control of mobile devices. This improves the level of driving automation and safety of mobile devices.

[0155] In some examples, the mobile device can be Figure 1 or Figure 2 The mobile device shown.

[0156] Figure 8 This is a schematic diagram of the structure of a sensing system verification device provided in an embodiment of this disclosure. Figure 8 As shown, the sensing system verification device 1000 may include a memory 1010 and a processor 1020. The memory 1010 may be used to store computer instructions, and the processor 1020 may be used to retrieve computer instructions from the memory 1010 to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The processor may be one or more, and the one or more processors may execute instructions individually or jointly. Similarly, the memory may be one or more, and the one or more memories may store the aforementioned computer instructions individually or jointly.

[0157] In some examples, the sensing system verification device can be Figure 1 The server and / or mobile device in the sensing system. In other examples, the sensing system verification device can also be any electronic device.

[0158] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods in the foregoing embodiments of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0159] This disclosure also provides a chip that may include a processing unit, which can be used to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The chip may be in the form of an Application-Specific Integrated Circuit (ASIC), a System-on-Chip (SOC), a Field-Programmable Gate Array (FPGA), etc., and this embodiment is not limited to this. Optionally, the chip may further include a storage unit, which can be used to store computer instructions. The processing unit can be used to retrieve the computer instructions from the storage unit to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure.

[0160] This disclosure also provides a computer program product that may include a computer program that, when executed by a processor, can implement any of the methods described in the foregoing embodiments of this disclosure.

[0161] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0162] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0163] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0164] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and conventional procedural programming languages ​​(e.g., the "C" language or similar programming languages). The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, may execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0165] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0166] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that embodiments of the present disclosure may include some or all of the functions marked in the multiple blocks in the drawings, and may also include other functions not shown in the blocks in the drawings. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0169] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A verification method for a sensing system, characterized in that, The method includes: Based on environmental data collected by a mobile device, a candidate sensing data set including multiple candidate sensing points and a candidate benchmark data set including multiple candidate benchmark points are determined. Each candidate sensing point is one of multiple data points used to characterize a sensing object, and each candidate benchmark point is one of multiple data points used to characterize a benchmark object. The benchmark object and the sensing object are objects with a linearly extended shape. The candidate sensing data set is a data set obtained after processing the environmental data using the target sensing system of the mobile device. The candidate benchmark data set is used to verify the accuracy of the candidate sensing data set. Based on the distance between the candidate reference point and the candidate sensing point in the target direction, one or more sets of target points are determined; wherein each set of target points includes a target reference point determined from a plurality of candidate reference points and a target sensing point corresponding to the target reference point determined from a plurality of candidate sensing points, and the target direction of each candidate reference point is a direction determined based on the extension direction of the reference object at the candidate reference point; A verification result for the target perception system is generated based on the set of target points.

2. The method according to claim 1, characterized in that, For the first reference point among the candidate reference points, the target direction of the first reference point is determined based on the following method: Among the plurality of candidate reference points, a second reference point and a third reference point adjacent to the first reference point are determined; Based on a preset tangent direction determination algorithm, the tangent direction of the first reference point is calculated according to the coordinates of the first reference point, the second reference point, and the third reference point; The tangent direction is taken as the extension direction of the reference object at the first reference point; The target direction of the first reference point is determined based on the extended direction.

3. The method according to claim 2, characterized in that, Determining the target direction of the first reference point based on the extension direction includes: Based on the pose information of the mobile device when collecting the environmental data, at least two candidate reference directions are determined; Based on the extension direction, a candidate reference direction is determined from at least two candidate reference directions as the target reference direction of the first reference point; wherein, the target angle between the target reference direction and the extension direction is the minimum value of the angles between at least two candidate reference directions and the extension direction; The direction perpendicular to the target reference direction is taken as the target direction of the first reference point.

4. The method according to claim 1, characterized in that, The step of determining one or more target point sets based on the distance between the candidate reference points and the candidate sensing points in the target direction includes: For each candidate reference point, if there is at least one first sensing point among the multiple candidate sensing points in the candidate sensing data set that satisfies the preset matching conditions, then the candidate reference point is taken as the target reference point, and the first sensing point with the smallest target distance from the target reference point in the target direction is determined among at least one first sensing point as the target sensing point corresponding to the target reference point, and the target point set is generated based on the target reference point and the target sensing point. The preset matching conditions include: the straight-line distance between the first sensing point and the candidate reference point is less than or equal to a preset distance threshold, and / or the sensing object corresponding to the first sensing point is of the same object type as the reference object corresponding to the candidate reference point.

5. The method according to claim 4, characterized in that, The step of generating a verification result for the target perception system based on the target point set includes: For each set of target points, the baseline object attribute information of the baseline object corresponding to the target baseline point and the perception object attribute information of the perception object corresponding to the target perception point are determined; if it is determined that the perception object attribute information and the baseline object attribute information do not meet the preset attribute matching verification rules, the set of target points is determined to be an attribute error set. Based on the number of attribute error sets and the number of target point sets, an attribute-aware error index is calculated, and the attribute-aware error index is used as at least a part of the verification result.

6. The method according to claim 1, characterized in that, The step of generating a verification result for the target perception system based on the target point set includes: Based on the target point set, determine the unmatched benchmark data points in the candidate benchmark data set and the unmatched sensing data points in the candidate sensing data set. The verification result of the target perception system is calculated based on the target point set, the unmatched sensing data points, and the unmatched reference data points.

7. The method according to claim 6, characterized in that, The step of calculating the verification result of the target perception system based on the target point set, the unmatched sensing data points, and the unmatched reference data points includes: Based on the target point set, unmatched perception data points, and unmatched benchmark data points of each benchmark object, an object-level statistical index of the benchmark object is calculated, and the object-level statistical index is used as at least a part of the index of the verification result.

8. The method according to any one of claims 1 to 7, characterized in that, The environmental data collected by the mobile device determines a candidate sensing data set including multiple candidate sensing points and a candidate reference data set including multiple candidate reference points, including: Based on the target perception system of the mobile device, the environmental data is processed to generate a raw perception data set including multiple raw perception points. Obtain the original benchmark data set corresponding to the environmental data, wherein the original benchmark data set includes multiple original benchmark points; The original reference data set and the original sensing data set are resampled to generate a candidate reference data set including multiple candidate reference points and a candidate sensing data set including multiple candidate sensing points.

9. The method according to claim 8, characterized in that, The original baseline data set and the original sensed data set are resampled, including: Based on the pose information of the mobile device when collecting the environmental data, resampling information for resampling is determined. The resampling information includes a first sampling direction, a second sampling direction, a first resampling distance interval corresponding to the first sampling direction, and a second resampling distance interval corresponding to the second sampling direction. The first sampling direction is the longitudinal direction of the mobile device, and the second sampling direction is perpendicular to the first sampling direction. The original reference data set or the original sensing data set are respectively used as the original data set. Based on the resampling information, each original data point in the original data set, and the original object corresponding to each original data point, the original data set is resampled to generate a candidate data set. The original data point is the original reference point or the original sensing point, the original object is the reference object or the sensing object, and the candidate data set is the candidate reference data set or the candidate sensing data set.

10. The method according to claim 9, characterized in that, Based on the resampling information, each original data point in the original dataset, and the original object corresponding to each original data point, the original dataset is resampled to generate a candidate dataset, including: For each original data point in the original dataset, target resampling is performed to obtain sampled data points corresponding to the original data points; The candidate data set is generated based on the multiple original data points and the sampled data points corresponding to each original data point. The target resampling process includes: Determine the extension direction and resampling interval corresponding to the original data point; wherein, the resampling interval is the linear interval between the original data point and other adjacent original data points where the original object is located; Based on the extension direction, the direction with the smallest angle to the extension direction is determined from the first sampling direction and the second sampling direction of the resampling information as the resampling direction corresponding to the original data point; Based on the resampling direction and the resampling distance interval corresponding to the resampling direction, resampling is performed within the resampling interval to obtain the sampled data points.

11. A mobile device, characterized in that, The mobile device is equipped with a target perception system, which is a perception system verified based on the perception system verification method according to any one of claims 1 to 10.

12. A sensing system verification device, characterized in that, The method includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method of any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

Images