Space-time labeling and automatic analysis method and system for automatic driving test problem

Abstract

The invention discloses a space-time labeling and automatic analysis method and system for an automatic driving test problem, and relates to the technical field of automatic driving. The space-time marking and automatic analysis method comprises the following steps: synchronously collecting time sequence data of a multi-source perception sensor on an automatic driving vehicle; performing data fusion processing on the time sequence data to generate lane-level structured positioning data with time-space alignment; based on the structured positioning data, multi-modal feature extraction and hierarchical classification identification are carried out by using a deep learning model, and a classification result of the test problem is obtained; calling a high-precision map service, associating a classification result with geographic position information in the structured positioning data, and generating time-space annotation data; and performing visual rendering on the spatio-temporal annotation data on the high-precision map to generate a test problem map interface. Through the synergistic effect of three core technologies of multi-source data fusion, intelligent analysis and space-time visualization, full-process automation and intelligentization of discovery, positioning, attribution and presentation of test problems are realized.

Description

A spatiotemporal annotation and automated analysis method and system for autonomous driving testing problems Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a spatiotemporal annotation and automated analysis method and system for autonomous driving testing problems. Background Technology

[0002] Autonomous driving testing and verification is a core component in ensuring the safety and reliability of advanced intelligent driving systems. As systems evolve from single-function verification to a systematic evaluation of the entire "perception-decision-control" chain and multi-sensor fusion performance, the industry is committed to building a comprehensive testing system that combines simulation, closed-course testing, and open road testing to address the challenges of long-tail scenarios. However, traditional manual methods are inefficient when processing massive amounts of multi-dimensional data. Furthermore, issues such as spatiotemporal alignment deviations between multi-source sensors, automatic and accurate fault location, and the disconnect between test result presentation and analysis collectively limit testing efficiency.

[0003] Existing solutions have improved automation and data processing efficiency to some extent by introducing high-precision integrated navigation, automated scripts, and visualization tools. However, these solutions still have significant limitations: the lack of integration between tools makes it difficult to form an end-to-end process from data collection and intelligent analysis to result presentation; multi-sensor data fusion often remains at a loosely coupled level, resulting in insufficient positioning accuracy in complex environments and an inability to support lane-level problem labeling; problem analysis relies too heavily on preset rules and lacks the ability to automatically extract features from multimodal data using deep learning; and the weak integration of test problems with high-precision maps limits the ability to gain macroscopic insights into patterns and mine data value from a spatiotemporal perspective. Summary of the Invention

[0004] The purpose of this invention is to overcome the technical problems of low testing efficiency, insufficient accuracy, and slow iterative optimization caused by the lack of accuracy of multi-sensor data fusion, lack of intelligent analysis capabilities, and limited visualization depth in the existing autonomous driving testing methods, and to provide a spatiotemporal annotation and automated analysis method and system for autonomous driving testing problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, the present invention provides a method for spatiotemporal annotation and automated analysis of autonomous driving test problems, comprising the following steps: synchronously collecting time-series data from multi-source perception sensors on an autonomous vehicle; performing data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; based on the structured positioning data, using a deep learning model to perform multimodal feature extraction and hierarchical classification recognition to obtain a classification result for the test problem; calling a high-precision map service to associate the classification result with the geographical location information in the structured positioning data to generate spatiotemporal annotation data; and visually rendering the spatiotemporal annotation data on the high-precision map to generate a test problem map interface.

[0006] A further improvement of the present invention is that the multi-source sensing sensor includes at least a GNSS receiver, an IMU inertial measurement unit, and a camera; the synchronous acquisition is achieved through a high-precision clock synchronization protocol.

[0007] A further improvement of the present invention is that the data fusion processing includes: fusing GNSS and IMU data using a tightly coupled filtering model; and further fusing visual SLAM data based on a federated filtering architecture to generate the structured positioning data.

[0008] A further improvement of the present invention is that the hierarchical classification and recognition includes three levels of processing: the first level classifies scene features based on visual data, the second level fuses vehicle state parameters to determine driving / parking modes, and the third level uses the corresponding temporal model or graph network to perform fine-grained problem classification based on the mode.

[0009] A further improvement of the present invention is that the logic for determining the driving / parking mode includes: when the vehicle speed is continuously lower than a set threshold and the vehicle is located in a parking area, it is determined to be in parking mode; otherwise, it is determined to be in driving mode.

[0010] A further improvement of the present invention is that, after obtaining the classification result, it also includes a similarity matching step: calculating the multi-dimensional weighted similarity between the new question and historical cases, and performing automated attribution, triggering manual review, or marking it as a new question based on the similarity threshold.

[0011] A further improvement of the present invention is that the visualization rendering includes: generating a spatiotemporal heatmap based on the spatiotemporal labeled data using a kernel density estimation algorithm, so as to visually display the distribution pattern of the test problem.

[0012] Secondly, this invention provides a spatiotemporal annotation and automated analysis system for autonomous driving test problems, comprising the following modules: a data acquisition module for synchronously acquiring time-series data from multi-source perception sensors on an autonomous vehicle; a data fusion module for performing data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; a problem analysis module for performing multimodal feature extraction and hierarchical classification and recognition based on the structured positioning data using a deep learning model to obtain classification results for test problems; a spatiotemporal annotation module for calling a high-precision map service to associate the classification results with the geographical location information in the structured positioning data to generate spatiotemporal annotation data; and a visualization rendering module for visually rendering the spatiotemporal annotation data on the high-precision map to generate a test problem map interface.

[0013] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-mentioned method for spatiotemporal annotation and automated analysis of autonomous driving test problems.

[0014] Fourthly, the present invention provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned method for spatiotemporal annotation and automated analysis of autonomous driving test problems.

[0015] Compared with existing technologies, this invention has the following beneficial effects: This invention provides a spatiotemporal annotation and automated analysis method for autonomous driving test problems. First, it synchronously collects time-series data from multiple source perception sensors and uses data fusion processing to generate spatiotemporally aligned lane-level structured positioning data. This effectively solves the problem of inaccurate positioning caused by occlusion and signal loss in complex scenarios using traditional single sensors, improving positioning accuracy to the centimeter level and providing a reliable data foundation for subsequent analysis. Based on this, it utilizes a deep learning model for multimodal feature extraction and hierarchical classification and recognition, achieving automated attribution and analysis of test problems. This significantly reduces reliance on manual interpretation and solves the problems of low efficiency and strong subjectivity in traditional methods when processing massive amounts of test data. Simultaneously, the hierarchical classification strategy enhances the ability to identify complex scenarios and long-tail problems. Furthermore, by calling a high-precision map service, the classification results are associated with geographic location information to generate spatiotemporal annotation data, which is then visualized and rendered on a high-precision map, ultimately forming an intuitive test problem map interface. This allows test engineers to quickly locate the precise spatiotemporal points where problems occur, understand the distribution patterns of problems, and thus accelerate fault diagnosis and algorithm iteration optimization. In summary, this method, through the synergistic effect of three core technologies—multi-source data fusion, intelligent analysis, and spatiotemporal visualization—achieves full-process automation and intelligence in the discovery, localization, attribution, and presentation of test problems. This significantly improves the efficiency and accuracy of autonomous driving testing and provides strong technical support for the safety verification of autonomous driving systems. Attached Figure Description

[0016] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the invention in any way. Furthermore, the shapes and proportions of the components in the drawings are merely illustrative to aid in understanding the invention and do not specifically limit the shapes and proportions of the components of the invention.

[0017] Figure 1 is a schematic diagram of the spatiotemporal annotation and automated analysis method for autonomous driving test problems according to the present invention; Figure 2 is a flowchart of a specific method according to an embodiment of the present invention; Figure 3 is an automated analysis and identification model for problems according to an embodiment of the present invention; Figure 4 is a sequence diagram of the specific system interaction process according to an embodiment of the present invention; Figure 5 is a schematic diagram of the spatiotemporal annotation and automated analysis system for autonomous driving test problems according to the present invention; Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0020] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0021] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0023] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings: Example 1 As shown in Figure 1, the present invention provides a spatiotemporal annotation and automated analysis method for autonomous driving test problems, including the following steps: S1, synchronously collecting time-series data from multi-source perception sensors on an autonomous vehicle; S2, performing data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; S3, based on the structured positioning data, using a deep learning model to perform multimodal feature extraction and hierarchical classification recognition to obtain the classification result of the test problem; S4, calling a high-precision map service to associate the classification result with the geographical location information in the structured positioning data to generate spatiotemporal annotation data; S5, visually rendering the spatiotemporal annotation data on the high-precision map to generate a test problem map interface.

[0025] The flowchart shown in Figure 2 clearly illustrates the complete technical chain and modular architecture of this method from data acquisition to result visualization. It comprehensively demonstrates the collaborative workflow of the three core modules corresponding to the above steps: "data acquisition and fusion", "intelligent analysis and recognition" and "spatiotemporal annotation visualization".

[0026] In practice, the synchronous acquisition step is accomplished by a sensor group consisting of a GNSS receiver (such as NovAtel PwrPak7), an IMU (such as ADIS16488), and a global shutter camera (such as FLIR BFS-U3-16S2C-C) configured in the vehicle. The acquisition process employs the IEEE 1588 v2 precision clock synchronization protocol to achieve microsecond-level time synchronization, ensuring data timing consistency. Data fusion processing uses an extended Kalman filter algorithm to establish a tightly coupled model, with the state update formula as follows:

[0027] In the formula, For state estimators, For the current moment, For Kalman gain, These are actual measured values. The measurement prediction function is dynamically estimated using a state vector containing 9 parameters such as position, velocity, and attitude. Then, a federated filtering architecture is used to fuse visual SLAM feature point data, outputting structured positioning data with centimeter-level accuracy. The deep learning model employs a multimodal fusion architecture combining a ResNet-18-based visual feature extraction network and a bidirectional LSTM positioning data processing network. Problem identification is achieved through a hierarchical classification process as shown in Figure 3: this flowchart clearly illustrates the decision-making path from environmental perception to specific problem attribution. The first-level classification identifies scene features (such as weather, lighting, and road type) based on visual data; the second-level classification fuses vehicle state parameters to determine driving or parking modes; the third-level classification is further divided according to mode: driving scenarios use an LSTM network to analyze driving behavior features (such as ELK and ACC), while parking scenarios use a graph convolutional network (GCN) to process ultrasonic point cloud data (such as APA and AVP). The high-precision map service is implemented by calling the Amap API V5.0. After converting the WGS84 coordinate system to the GCJ-02 coordinate system, it performs nearest road search in the directed graph G=(V,E) of the road network based on the improved A* algorithm. To achieve sub-meter level accuracy matching, the system introduces a lane-level matching confidence model to score candidate lanes. This model is as follows:

[0028] in, This represents the confidence level of the match between the vehicle's location and a specific candidate lane, with a value range of (0,1). This refers to the lateral distance deviation between the positioning point and the lane centerline. The angular deviation between the vehicle's heading angle and the lane direction line; 0.5 and 0.3 are weighting parameters calibrated using historical data, used to balance the impact of lateral and heading deviations on the matching results. The system selects... The lane with the highest value is selected as the final matching result. Subsequently, the system generates GeoJSON formatted annotation data with a timestamp. Visualization rendering utilizes WebGL technology, employing the Mapbox GL JS engine to dynamically overlay annotation points and generate heatmaps.

[0029] The specific system interaction flow is shown in the sequence diagram in Figure 4: After the tester triggers manual marking, the system sequentially calls the positioning system (to obtain multi-sensor fused coordinates) and the high-precision map service (to perform reverse geocoding), ultimately storing the problem-time data with precise semantic location in a structured manner, providing a high-quality, traceable data foundation for subsequent analysis. This method solves the problem of spatiotemporal inconsistency of sensor data through synchronous acquisition and fusion processing of multi-source data; it achieves automated classification of test problems through a hierarchical deep learning model; and it realizes precise positioning and visualization of problem points through the high-precision map service. A complete technical chain from data acquisition, fusion analysis to visualization presentation has been established, forming a closed-loop processing flow. After implementation, it can achieve the technical effects of reducing manual dependence, improving testing efficiency, and achieving centimeter-level positioning accuracy.

[0030] In some embodiments, the multi-source sensing sensors include at least a GNSS receiver, an IMU (Inertial Measurement Unit), and a camera; synchronous acquisition is achieved through a high-precision clock synchronization protocol. Specifically, the NovAtelPwrPak7 GNSS receiver is selected, which supports dual-frequency signal reception and effectively suppresses ionospheric errors; the ADIS16488 IMU is selected, which has six degrees of freedom measurement capability and low temperature drift characteristics; the FLIR BFS-U3-16S2C-C global shutter camera is selected to avoid motion distortion caused by rolling shutter. Synchronous acquisition adopts the IEEE 1588 v2 network precision clock synchronization protocol, achieving a synchronization accuracy of less than 1 microsecond between sensors through hardware timestamps. In practical applications, other sensor models can also be selected, and this application embodiment does not limit this. This scheme ensures the accuracy and synchronization of raw data acquisition by selecting a combination of sensors with specific performance and a synchronization protocol.

[0031] Data fusion processing includes: First, spatiotemporal alignment and coordinate system standardization of multi-source data. A body coordinate system (ISO-8855 standard) is established with the vehicle's center of mass as the origin. Global or local coordinates provided by sensors such as GNSS and visual SLAM are uniformly transformed to this body coordinate system using a homogeneous transformation matrix. This step ensures that all subsequent fusion operations are performed under a consistent spatiotemporal reference. The core transformation is as follows:

[0032] in, This is the transformation matrix from the local map coordinate system to the WGS84 geodetic coordinate system. This is the transformation matrix from the WGS84 geodetic coordinate system to the geocentric Earth-fixed coordinate system. Let be the transformation matrix from the Earth-centered Earth-fixed coordinate system to the local Northeast-Sky coordinate system. This is the transformation matrix from the global coordinate system to the vehicle body coordinate system.

[0033] Based on a unified coordinate system, a tightly coupled filtering model is used to fuse GNSS and IMU data; and further, visual SLAM data is fused based on a federated filtering architecture to generate structured positioning data. The tightly coupled filtering adopts the extended Kalman filter algorithm, and the state vector contains 15-dimensional parameters such as position, velocity, and attitude. 100Hz high-frequency data fusion is achieved through the NovAtel SPAN-CPT hardware platform. The federated filtering architecture sets up GNSS / IMU sub-filters and visual SLAM sub-filters, and dynamically adjusts the weights of each sensor through adaptive information allocation coefficients. When the GNSS signal is lost, the fusion weights are automatically switched to the visual SLAM system. Visual SLAM adopts the VINS-Mono framework, and constructs the spatiotemporal consistency constraint equation through the Lie group SE(3) method to estimate the extrinsic parameter transformation matrix between the camera and IMU in real time; where the spatiotemporal consistency constraint equation is:

[0034] In the formula, For To optimize the variables, minimize them. For projection function, To make the coordinate system Transform to coordinate system The transformation matrix, World coordinate system To coordinate system The inverse transformation matrix, For the first The coordinates of a 3D spatial point For the first The target value for each point.

[0035] The hierarchical classification and recognition process comprises three levels of processing: the first level classifies scene features based on visual data; the second level integrates vehicle state parameters to determine driving / parking modes; and the third level uses corresponding temporal models or graph networks for fine-grained problem classification based on the mode. The first-level classification employs an improved ResNet-18 network, with input data being a normalized sequence of video frames. A spatial attention mechanism enhances the representation of key region features, outputting scene features such as weather, lighting, and road type. The second-level classification uses a two-layer fully connected network, inputting parameters such as vehicle speed, steering angle, and gear position obtained from the vehicle's CAN bus. When the vehicle speed is consistently below 5 km / h and the location is within a parking area, it is determined to be in parking mode. The third-level classification uses an LSTM temporal network to analyze driving behavior features over three consecutive seconds for driving modes, and a GCN graph convolutional network to process ultrasonic point cloud data for parking modes.

[0036] The logic for determining driving / parking modes includes: when the vehicle speed is consistently below a set threshold and the vehicle is located within a parking area, it is determined to be in parking mode; otherwise, it is determined to be in driving mode. Specifically, the threshold is set to 5 km / h, and the average vehicle speed of 10 consecutive sampling points is detected using a sliding window algorithm. The location area determination combines POI information provided by the Gaode Map API; when the vehicle is within the polygonal geofence of the parking lot, parking mode determination is triggered. For boundary situations such as low-speed maneuvering, an additional threshold of 15 degrees for the standard deviation of the steering angle and an ultrasonic radar detection distance of <1.5 meters are introduced as auxiliary judgment conditions.

[0037] After obtaining the classification results, a similarity matching step is also included: calculating the multi-dimensional weighted similarity between the new question and historical cases, and performing automated attribution, triggering manual review, or marking it as a new question based on the similarity threshold. Specifically, the multi-dimensional weighted similarity calculation formula is as follows:

[0038] in, To assess overall similarity, These are the weighting coefficients for spatial similarity. For spatial similarity components, These are the weighting coefficients for time similarity. For time similarity components, These are the weighting coefficients for sensor similarity. For sensor similarity components, These are the weighting coefficients for environmental similarity. This is the environmental similarity component. Specifically, spatial similarity uses the Haversine formula to calculate geographical distance, temporal similarity uses a time decay function, sensor similarity uses a dynamic time warping algorithm, and environmental similarity is based on weather condition matching. Thresholds are set: when similarity > 0.8, historical cases are automatically associated for attribution; between 0.5 and 0.8, manual review is triggered; and when similarity < 0.5, it is marked as a new problem type for specialized analysis.

[0039] The visualization rendering includes generating a spatiotemporal heatmap based on spatiotemporally labeled data using a kernel density estimation algorithm to visually represent the distribution patterns of the test questions. Specifically, the kernel density estimation function is:

[0040] in, In order to be in The true probability density function at point The estimated value, For sample size, The bandwidth parameter is used to control smoothness. For the first Data points of each sample, This is the kernel function.

[0041] Specifically, bandwidth parameters Adaptive selection of kernel function according to the Silverman criterion. Employing a Gaussian kernel, the visualization engine uses an H3 geographic grid index combined with R*-Tree to achieve fast spatiotemporal data retrieval. Real-time rendering of heatmaps is achieved through WebGL shaders, and the display granularity can be dynamically adjusted by sliding windows over time.

[0042] As shown in Figure 5, Example 2 of this invention also provides a spatiotemporal annotation and automated analysis system for autonomous driving test problems, comprising the following modules: a data acquisition module for synchronously acquiring time-series data from multi-source perception sensors on an autonomous vehicle; a data fusion module for performing data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; a problem analysis module for performing multimodal feature extraction and hierarchical classification and recognition based on the structured positioning data using a deep learning model to obtain the classification results of the test problems; a spatiotemporal annotation module for calling a high-precision map service to associate the classification results with the geographical location information in the structured positioning data to generate spatiotemporal annotation data; and a visualization rendering module for visually rendering the spatiotemporal annotation data on a high-precision map to generate a test problem map interface.

[0043] The data acquisition module is deployed in the vehicle's industrial control computer, connecting to various sensors via a CAN bus and Ethernet interface. The data fusion module uses a real-time processing program written in C++, running on the vehicle's computing unit (NVIDIA DRIVE AGX Orin). The problem analysis module uses a deep learning model implemented in the PyTorch framework, deployed on an edge server. The spatiotemporal annotation module calls the Amap (Gaode Maps) service via a RESTful API, and the visualization rendering module develops a web front-end interface based on Vue.js + Mapbox GL. All modules communicate with each other through data topics established using the ROS 2 communication framework.

[0044] This system addresses the issue of data processing chain disruptions through modular design. By establishing standardized data interfaces and communication protocols, it ensures collaborative operation among modules, further achieving fully automated processing from data acquisition to visualization.

[0045] Example 3: Referring to Figure 6, the present invention also provides an electronic device 100 for spatiotemporal labeling and automated analysis of autonomous driving test problems; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0046] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the spatiotemporal annotation and automated analysis method for autonomous driving test problems described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0047] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0048] The memory 101 in the electronic device 100 stores multiple instructions to implement a spatiotemporal annotation and automated analysis method for autonomous driving test problems. The processor 102 can execute the multiple instructions to achieve: synchronously collecting time-series data from multi-source perception sensors on the autonomous vehicle; performing data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; based on the structured positioning data, using a deep learning model to perform multimodal feature extraction and hierarchical classification recognition to obtain the classification result of the test problem; calling a high-precision map service to associate the classification result with the geographical location information in the structured positioning data to generate spatiotemporal annotation data; and visually rendering the spatiotemporal annotation data on the high-precision map to generate a test problem map interface.

[0049] Example 4: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0050] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0055] Many embodiments and applications beyond the examples provided will be apparent to those skilled in the art upon reading the foregoing description. Therefore, the scope of this teaching should not be determined by reference to the foregoing description, but rather by reference to the foregoing claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the foregoing claims is not intended as a waiver of that subject matter, nor should it be construed as an indication that the applicant has not considered that subject matter as part of the disclosed inventive subject matter.

[0056] The above content provides a further detailed description of the present invention. It should not be construed that the specific embodiments of the present invention are limited to this. For those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection of the present invention as defined by the submitted claims.

Claims

1. A spatiotemporal annotation and automated analysis method for autonomous driving testing problems, characterized in that, Includes the following steps: Synchronously collect time-series data from multi-source perception sensors on autonomous vehicles; The time-series data is fused to generate spatiotemporally aligned lane-level structured positioning data; Based on the structured localization data, a deep learning model is used to extract multimodal features and perform hierarchical classification and recognition to obtain the classification results of the test question. The high-precision map service is invoked to associate the classification results with the geographic location information in the structured positioning data to generate spatiotemporal annotation data; The spatiotemporal annotation data is visualized and rendered on the high-precision map to generate a test problem map interface.

2. The spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 1, characterized in that, The multi-source sensing sensor includes at least a GNSS receiver, an IMU inertial measurement unit, and a camera; the synchronous acquisition is achieved through a high-precision clock synchronization protocol.

3. A spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 1 or 2, characterized in that, The data fusion process includes: fusing GNSS and IMU data using a tightly coupled filtering model; and further fusing visual SLAM data based on a federated filtering architecture to generate the structured positioning data.

4. The spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 1, characterized in that, The hierarchical classification and recognition includes three levels of processing: the first level classifies scene features based on visual data, the second level integrates vehicle state parameters to determine driving / parking modes, and the third level uses the corresponding temporal model or graph network to perform fine-grained problem classification based on the modes.

5. The spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 4, characterized in that, The logic for determining driving / parking modes includes: when the vehicle speed is continuously below a set threshold and the vehicle is located in a parking area, it is determined to be in parking mode; otherwise, it is determined to be in driving mode.

6. The spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 1, characterized in that, After obtaining the classification results, a similarity matching step is also included: calculating the multi-dimensional weighted similarity between the new question and historical cases, and performing automated attribution, triggering manual review, or marking it as a new question based on the similarity threshold.

7. The spatiotemporal annotation and automated analysis method for autonomous driving testing problems according to claim 1, characterized in that, The visualization rendering includes: generating a spatiotemporal heatmap based on the spatiotemporal labeled data using a kernel density estimation algorithm to visually display the distribution patterns of the test questions.

8. A spatiotemporal annotation and automated analysis system for autonomous driving testing problems, characterized in that, It includes the following modules: a data acquisition module, used to synchronously acquire time-series data from multi-source perception sensors on autonomous vehicles; The data fusion module is used to perform data fusion processing on the time-series data to generate spatiotemporally aligned lane-level structured positioning data; The problem analysis module is used to extract multimodal features and perform hierarchical classification and recognition based on the structured localization data using a deep learning model, so as to obtain the classification results of the test problem. The spatiotemporal annotation module is used to call high-precision map services, associate the classification results with the geographic location information in the structured positioning data, and generate spatiotemporal annotation data; The visualization rendering module is used to visualize and render the spatiotemporal annotation data on the high-precision map, generating a test problem map interface.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the spatiotemporal annotation and automated analysis method for autonomous driving test problems as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the spatiotemporal labeling and automated analysis method for the autonomous driving test problem as described in any one of claims 1 to 7.

Images