Simulation test scene parameter generation method of automatic driving system and electronic equipment

Abstract

The invention provides a simulation test scene parameter generation method of an automatic driving system and electronic equipment. The method comprises the following steps: acquiring road condition information transmitted by at least one road monitoring device in a target road in real time; the road condition information comprises a long-tail problem identification result; reconstructing road scene parameters based on pre-stored road attribute data and road condition information; searching a target vehicle running on a target road in an automatic driving mode by accessing an automatic driving system, and acquiring sensor configuration parameters of the target vehicle; and generating simulation test scene parameters according to the sensor configuration parameters and the road scene parameters, so as to generate a simulation test scene for the target road, the target vehicle and the automatic driving system based on the simulation test scene parameters. According to the invention, the simulation test scene covers a wide range of driving environments and emergencies, and the diversity and comprehensiveness of the scene are ensured, so that the method is used for realizing the efficient offline verification of the automatic driving system.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method for generating simulation test scenario parameters for an autonomous driving system and an electronic device. Background Art

[0002] At present, autonomous driving technology has gradually entered the field testing stage. However, in actual road testing, some long-tail problems may be encountered (such as traffic accidents, congestion, special-shaped vehicles, etc.), which are difficult to fully cover within the limited testing time and scope. The existing autonomous driving verification method mainly relies on field testing, which is subject to limitations such as high cost and high risk, and it is difficult to fully and systematically verify the safety and reliability of the autonomous driving system. Summary of the invention

[0003] The embodiments of the present application provide a method and electronic device for generating simulation test scenario parameters for an autonomous driving system to solve one or more of the above-mentioned technical problems.

[0004] In a first aspect, an embodiment of the present application provides a method for generating simulation test scenario parameters of an autonomous driving system, which is applied to a cloud computing platform, and the method includes: obtaining road condition information transmitted in real time by at least one road monitoring device in a target road; the road condition information includes a long-tail problem recognition result; the long-tail problem recognition result is obtained by the road monitoring device identifying the road condition information by calling a visual text large model, and the visual text large model is deployed in the road monitoring device or a local server of the autonomous driving system; reconstructing road scene parameters based on pre-stored road attribute data and the road condition information; the road condition information includes at least one of the state data of dynamic objects, the state data of static objects and environmental data in the road, and the road attribute data includes at least one of the orientation, lane data and geographic location of the road; by accessing the autonomous driving system, searching for a target vehicle traveling on the target road in an autonomous driving mode, and obtaining sensor configuration parameters of the target vehicle; generating simulation test scenario parameters according to the sensor configuration parameters and the road scene parameters, so as to generate a simulation test scenario for the target road, the target vehicle and the autonomous driving system based on the simulation test scenario parameters.

[0005] In a second aspect, an embodiment of the present application provides a method for generating simulation test scenario parameters of an autonomous driving system, which is applied to a road monitoring device, the method comprising: collecting road condition information of a target road in real time, the road condition information comprising at least one of status data of dynamic objects in the road, status data of static objects and environmental data; calling a large visual text model to detect the road condition information and identify a long-tail problem recognition result; wherein the large visual text model is deployed in the road monitoring device or a local server of the autonomous driving system; sending the road condition information including the long-tail problem recognition result to the cloud, so that the cloud generates simulation scenario parameters according to the road condition information including the long-tail problem recognition result, pre-stored road attribute data and sensor configuration parameters, and generates a simulation test scenario for the target road, target vehicle and the autonomous driving system based on the simulation test scenario parameters; wherein the road attribute data comprises at least one of the orientation of the road, lane data and geographic location, and the sensor configuration parameters are sensor configuration parameters of the target vehicle traveling on the target road in autonomous driving mode.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0008] Compared with the related art, this application has the following advantages:

[0009] According to an embodiment of the present application, the road condition information transmitted in real time by at least one road monitoring device in the target road is obtained; the road condition information includes a long-tail problem identification result; the long-tail problem identification result is obtained by the road monitoring device by calling a visual text big model to identify the road condition information, and the visual text big model is deployed in the local server of the road monitoring device or the autonomous driving system, so as to obtain the long-tail problems on the road automatically detected by the visual text big model transmitted by the road detection device, so as to convert these actual road conditions into scenes that can be used for simulation. Afterwards, the road scene parameters are reconstructed based on the pre-stored road attribute data and the road condition information; the road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data, and the road attribute data includes at least one of the orientation, lane data and geographical location of the road; by accessing the autonomous driving system, the target vehicle traveling on the target road in the autonomous driving mode is searched, and the sensor configuration parameters of the target vehicle are obtained; the simulation test scene parameters are generated according to the sensor configuration parameters and the road scene parameters, so as to generate the simulation test scene for the target road, the target vehicle and the autonomous driving system based on the simulation test scene parameters. The embodiment of the present application integrates road-side and cloud data, reconstructs road scene parameters, and then combines the sensor configuration parameters on the vehicle side to obtain simulation test scene parameters, so that the simulation test scene covers a wide range of driving environments and emergencies, ensuring the diversity and comprehensiveness of the scene, and then used to achieve efficient offline verification of the autonomous driving system.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 A flowchart of a method for generating simulation test scenario parameters for an autonomous driving system provided in an embodiment of the present application is shown;

[0013] Figure 2 A flowchart of another method for generating simulation test scenario parameters for an autonomous driving system provided in an embodiment of the present application is shown;

[0014] Figure 3A structural block diagram of a simulation test scenario parameter generation device for an autonomous driving system provided in an embodiment of the present application is shown;

[0015] Figure 4 A structural block diagram of another device for generating simulation test scenario parameters for an autonomous driving system provided in an embodiment of the present application is shown;

[0016] Figure 5 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0018] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0019] First, the terms involved are explained.

[0020] Vehicle-Road-Cloud Collaboration: Vehicle-Road-Cloud Collaboration refers to the integration of computing and data processing capabilities of the vehicle, road, and cloud to form a unified autonomous driving system verification architecture. The vehicle is responsible for perception and execution, the road is responsible for real-time data collection and preliminary processing, and the cloud is responsible for scene reconstruction and simulation testing.

[0021] Closed-Loop Simulation Verification: Closed-Loop Simulation Verification refers to the full-process simulation test of the autonomous driving system in a simulation environment, including perception, decision-making, control and other links. The system runs in a closed-loop form in the simulation, simulating the real driving process, and recording and analyzing its performance.

[0022] Mean Failure Per Million Kilometers (MPI): MPI is an important indicator for measuring the safety of autonomous driving systems, indicating the average number of failures per million kilometers of driving. A lower MPI value means a higher system reliability.

[0023] In different types of complex road environments such as urban roads, rural roads, and highways, the autonomous driving system needs to face a variety of road conditions and traffic conditions, including emergencies, bad weather, and traffic flow fluctuations. These environments often have long-tail problems, such as sudden traffic accidents, sudden congestion, and the identification and response of special-shaped vehicles (such as over-height and over-width vehicles). When planning specific autonomous driving routes, such as distribution routes in logistics parks and specific bus routes in the city center, relevant departments or enterprises need to verify and evaluate the performance of the autonomous driving system in these specific areas. When the autonomous driving system is upgraded, new algorithms, strategies, or sensor configurations are usually introduced. In order to verify the stability and security of the new version, it needs to be tested in as many scenarios as possible. Although field road testing can provide real road test data, it is often accompanied by high costs and potential safety risks. Especially when facing long-tail problems, traditional field testing methods are difficult to cover all possible scenarios. Therefore, how to conduct comprehensive and systematic verification of the autonomous driving system through efficient offline simulation methods combined with actual road test data has become an urgent problem to be solved.

[0024] In the process of developing and verifying the autonomous driving system, the following methods are traditionally used to verify and test the system performance:

[0025] The first one is real vehicle verification: real vehicle verification is one of the main methods for testing autonomous driving systems at present, that is, testing in a real road environment and evaluating the performance of the system through actual driving data. The advantage of this method is that the test environment is real and can directly reflect the performance of the system in actual use. However, there are several problems with real vehicle verification: High cost: Due to the cost of a large number of vehicles, drivers, equipment and road test sites, the cost of real vehicle verification is very high. Safety risks: Real vehicle testing in complex road environments may face potential risks of traffic accidents, especially when verifying long-tail problems (such as emergencies, accidents, etc.), it is even more difficult to ensure safety. Limited scenario coverage: With limited testing time and resources, real vehicle verification is difficult to cover all possible driving scenarios, especially some low-probability but high-risk long-tail problems, which are often difficult to verify effectively.

[0026] The second type is artificially created scenarios: Artificially created scenarios refer to the manual design and creation of specific test scenarios through simulation platforms to verify the performance of the autonomous driving system in these scenarios. The advantage of this method is that specific scenarios can be created for testing in a targeted manner, but it also has the following limitations: Scenario design is complex and time-consuming: Designing a high-quality test scenario requires a lot of time and manual intervention, especially when it comes to complex traffic environments, the difficulty and cost of scene production will increase significantly. Limited number and diversity of scenarios: The number of artificially created scenarios is usually limited, and it is difficult to cover all possible road conditions and long-tail problems, which can easily lead to incomplete testing. Difficult to reflect actual road conditions: Although artificially designed scenarios can be used to test specific problems, due to the lack of support from actual road data, it is often difficult to reflect the complex situations in the real world, resulting in a lack of reality in the test results.

[0027] The third method is scenario playback based on historical data: Another verification method is scenario playback based on historical road test data, that is, by replaying the data collected in past actual tests or operations, the system's performance in a specific scenario is verified. This method can effectively utilize existing data resources, but there are the following problems: Data limitations: Historical data is limited by past testing and operating environments and may not cover the latest road conditions or long-tail problems that have never been encountered. Scenarios are not customizable: The replayed scenarios are often fixed and difficult to adjust or customize according to new requirements, resulting in poor test flexibility.

[0028] Fourth: Rule-based simulation: Rule-driven simulation systems generate test scenarios based on preset traffic rules and vehicle behavior models. These systems are usually used for large-scale simulation tests. Although rule-driven simulation can generate a large amount of test data, its main problems are: The complexity of simulation scenarios is limited: preset rules are difficult to cover all variables in reality, especially when dealing with unexpected situations or long-tail problems, the simulation results may deviate greatly from the actual situation. Lack of realism: rule-driven simulation scenarios usually lack details and complexity, and it is difficult to fully simulate the road environment and traffic behavior in reality.

[0029] In summary, the existing autonomous driving system verification methods have their own advantages and disadvantages, but they generally have problems such as high cost, high risk, and incomplete scenario coverage, especially when verifying long-tail problems.

[0030] Based on the above problems, the present application provides a method and electronic device for generating simulation test scenario parameters for an autonomous driving system. The method detects these long-tail problems through road-side equipment and reconstructs the scene in the cloud to provide accurate test scenarios for offline simulation, which can effectively verify the response capabilities of the autonomous driving system in the development stage. The method can generate corresponding simulation scenarios based on the actual road conditions on the target route through the collected large-scale data, and conduct closed-loop simulation tests to help evaluate the various situations that the autonomous driving system may encounter in actual operation and ensure the safety and stability of the system. The method can generate rich simulation scenarios through vehicle-road-cloud collaboration, covering various long-tail problems, ensuring that the new version of the system can be fully verified in an offline environment before being put into practical application to reduce risks.

[0031] This method replaces some on-site road tests with offline simulation, which can reduce testing costs without affecting the quality of system verification and effectively reduce potential road test risks. This method can be applied to the development, testing, and evaluation of autonomous driving systems to improve efficiency, reduce costs, and ensure the safety and reliability of the system in practical applications. This method is an offline verification solution based on vehicle-road-cloud collaboration, which aims to solve the limitations of existing solutions through efficient scenario generation and simulation technology, and provide a more comprehensive, flexible and secure verification method.

[0032] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] The present application embodiment provides a method for generating simulation test scenario parameters for an autonomous driving system, which is applied to a cloud computing platform. Figure 1 The flowchart shown is a method for generating simulation test scenario parameters of an autonomous driving system according to an embodiment of the present application. The method may include:

[0034] Step S101, obtaining road condition information transmitted in real time by at least one road monitoring device in the target road; the road condition information includes a long-tail problem recognition result; the long-tail problem recognition result is obtained by the road monitoring device by calling a visual text big model to recognize the road condition information, and the visual text big model is deployed in the road monitoring device or a local server of the autonomous driving system.

[0035] In this possible implementation, the target road is a pre-selected route, such as a city road, a highway, etc. A plurality of road-end devices, i.e., road detection devices, may be installed on the selected route, and these devices include, but are not limited to, sensing devices such as cameras and radars, for real-time monitoring of road conditions. The road detection device continuously collects video streams, image data, and other sensor data, and uploads these data to a local server for preliminary processing. Using the visual text big model deployed on the road detection device or the local server, the collected images and videos are analyzed in real time to detect and identify potential long-tail problems, such as traffic accidents, congestion, and special-shaped vehicles (such as over-high or over-wide vehicles), etc. The cloud obtains the road condition information sent in real time by the road detection device, i.e., obtains the road condition information transmitted in real time by at least one road monitoring device in the target road; the road condition information includes the long-tail problem recognition result; the long-tail problem recognition result is obtained by the road monitoring device by calling the visual text big model to identify the road condition information, and the visual text big model is deployed in the road monitoring device or the local server of the autonomous driving system.

[0036] It should be noted that in the field of autonomous driving, long-tail issues refer to special scenarios or events that occur less frequently but have higher risks in actual road driving, such as traffic accidents, sudden congestion, and special-shaped vehicles. These issues are difficult to fully cover in limited field tests due to their low frequency but high risk characteristics. In this step, long-tail issues are detected in real time through road monitoring equipment and a large visual text model, and are used as an important scenario for verifying the autonomous driving system.

[0037] Once a long-tail problem is detected, it can be automatically labeled and classified and archived to obtain the long-tail problem identification results. The labeled content at least includes key information such as event type, occurrence time, and geographic location.

[0038] Step S102, reconstructing road scene parameters based on pre-stored road attribute data and the road condition information; the road condition information includes at least one of status data of dynamic objects in the road, status data of static objects and environmental data, and the road attribute data includes at least one of the road's orientation, lane data, and geographic location.

[0039] In this possible implementation, the pre-stored road attribute data may be historically acquired data, which may be obtained through continuous updating and improvement. The road attribute data includes at least one of the road orientation, lane data, and geographic location. The specific content of the road attribute data may be obtained according to actual needs, and the present application embodiment does not specifically limit this.

[0040] Road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data. Among them, static objects include fixed structures in the road environment, such as roads, guardrails, buildings, trees, etc. Dynamic objects include traffic participants such as vehicles and pedestrians that generate motion trajectories. Environmental data includes weather environment data, geographical environment data, etc. State data is used to characterize the position, posture and motion state of dynamic objects or static objects.

[0041] Road scene parameters are used to describe the scene and environment of the road, which can specifically cover different weather conditions, lighting environments, and traffic flow conditions.

[0042] Step S103, by accessing the automatic driving system, searching for a target vehicle traveling on the target road in an automatic driving mode, and acquiring sensor configuration parameters of the target vehicle.

[0043] In this possible implementation, the autonomous driving system refers to a system that uses computers, sensors, and artificial intelligence technology to enable the vehicle to autonomously perform driving operations without the intervention of a human driver. The goal of the autonomous driving system is to improve traffic safety, reduce traffic congestion, improve transportation efficiency, and provide users with a more convenient travel experience. Sensor configuration parameters refer to parameters that need to be set or adjusted when using sensors to ensure that the sensors can work accurately and stably. Different types of sensors have different configuration parameters. Autonomous driving mode refers to the working state and capabilities of the autonomous driving system under different conditions and scenarios. Depending on the degree of automation and application scenarios, autonomous driving modes can be divided into multiple levels and types.

[0044] In this step, the target vehicle traveling on the target road in the autonomous driving mode is searched by accessing the autonomous driving system, and the sensor configuration parameters of the target vehicle are obtained to ensure that it can be detected and processed by sensors with different configurations (such as cameras, radars, lidars, etc.).

[0045] Step S104, generating simulation test scenario parameters according to the sensor configuration parameters and the road scene parameters, so as to generate a simulation test scenario for the target road, the target vehicle and the autonomous driving system based on the simulation test scenario parameters.

[0046] In this possible implementation, the pre-stored road attribute data and road condition information may be recorded according to the world coordinate system or the fixed road coordinate system. Therefore, the road scene parameters reconstructed based on the two are also data in the world coordinate system or the fixed road coordinate system. The perception data of the vehicle sensor is generally based on the vehicle's own coordinate system (body coordinate system). Therefore, the simulation test scene parameters in the body coordinate system can be generated according to the sensor configuration parameters and the road scene parameters. Then, based on the simulation test scene parameters, a simulation test scene for the target road, the target vehicle and the automatic driving system is generated. In this step, the reconstructed static objects, dynamic objects and road-end sensor data are converted into sensor signals from the perspective of the vehicle in order to simulate the real-time environmental information that the sensor will receive during the vehicle's driving process. This process involves multiple aspects such as coordinate conversion, multi-sensor fusion, and simulated sensor characteristics.

[0047] Generate simulation test scenarios for autonomous driving systems, i.e., scenario reconstruction. Scenario reconstruction refers to processing data on actual roads (such as video clips, image data, etc.) through algorithms to generate high-fidelity virtual scenarios that can run in a simulation environment. These scenarios restore real road conditions as much as possible in order to test the performance of the autonomous driving system in simulation. In this step, the actual long-tail problems detected are converted into simulation scenarios through the scenario reconstruction technology in the cloud to ensure the authenticity and reliability of the test results.

[0048] In this step, the road-side equipment is responsible for real-time detection and identification of long-tail problems on the road, the cloud is responsible for data processing, scene reconstruction and simulation scene generation, and then the vehicle-side performs simulation testing and system performance verification. Relying on the vehicle-road-cloud collaborative architecture, efficient offline verification of the autonomous driving system is achieved, reducing dependence on field testing and improving the comprehensiveness and safety of verification. Furthermore, closed-loop simulation verification is used to comprehensively evaluate the performance of the autonomous driving system in complex scenarios, which helps to discover and solve potential problems in the system in advance.

[0049] According to an embodiment of the present application, the road condition information transmitted in real time by at least one road monitoring device in the target road is obtained; the road condition information includes a long-tail problem identification result; the long-tail problem identification result is obtained by the road monitoring device by calling a visual text big model to identify the road condition information, and the visual text big model is deployed in the local server of the road monitoring device or the autonomous driving system, so as to obtain the long-tail problems on the road automatically detected by the visual text big model transmitted by the road detection device, so as to convert these actual road conditions into scenes that can be used for simulation. Afterwards, the road scene parameters are reconstructed based on the pre-stored road attribute data and the road condition information; the road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data, and the road attribute data includes at least one of the orientation, lane data and geographical location of the road; by accessing the autonomous driving system, the target vehicle traveling on the target road in the autonomous driving mode is searched, and the sensor configuration parameters of the target vehicle are obtained; the simulation test scene parameters are generated according to the sensor configuration parameters and the road scene parameters, so as to generate the simulation test scene for the target road, the target vehicle and the autonomous driving system based on the simulation test scene parameters. The embodiment of the present application integrates road-side and cloud data, reconstructs road scene parameters, and then combines the sensor configuration parameters on the vehicle side to obtain simulation test scene parameters, so that the simulation test scene covers a wide range of driving environments and emergencies, ensuring the diversity and comprehensiveness of the scene, and then used to achieve efficient offline verification of the autonomous driving system.

[0050] In a possible implementation, reconstructing road scene parameters based on pre-stored road attribute data and the road condition information includes: reconstructing the motion trajectories of static objects and dynamic objects in the road scene respectively based on the road condition information; and adding first reconstructed data of the static objects and second reconstructed data of the motion trajectories of the dynamic objects to the pre-stored road attribute data.

[0051] In this possible implementation, since the motion states of static objects and dynamic objects are different, the data processing processes of the two are different. Therefore, the motion trajectories of static objects and dynamic objects in the road scene can be reconstructed respectively according to the characteristics of static objects and dynamic objects. After that, the reconstructed static objects, i.e., the first reconstructed data, and the motion trajectories of dynamic objects, i.e., the second reconstructed data, are added to the pre-stored road attribute data. It should be noted that the motion trajectories of dynamic objects include dynamic objects.

[0052] In a possible implementation, based on the road condition information, the motion trajectories of static objects and dynamic objects in the road scene are reconstructed separately, which can be performed according to the following steps: performing point cloud segmentation processing on the point cloud data in the road condition information to determine the point cloud data of the static object, and reconstructing the static object based on the point cloud data of the static object; using a deep learning model to identify dynamic objects from the video sequence of the road condition information, performing multi-target tracking on the identified dynamic objects, obtaining trajectory estimation results based on the tracked targets, and determining the motion trajectory of the dynamic object based on the trajectory estimation results.

[0053] In this possible implementation, when the road condition information includes point cloud data, the point cloud data is subjected to point cloud filtering processing, that is, the original point cloud data collected by the laser radar is filtered to remove noise points and outliers and retain valid information. After that, point cloud segmentation processing is performed, and a clustering algorithm (such as Euclidean distance clustering) is used to segment the point cloud data into different object areas, and static objects such as roads, buildings, and guardrails are preliminarily separated. After that, point cloud and image fusion can also be performed: that is, the point cloud data and image data are aligned, and the texture and color information in the image are used to enrich the features of the point cloud and improve the precision of the modeling of static objects. Finally, point cloud splicing processing can be performed, that is, a more complete three-dimensional scene is obtained by splicing multiple frames of point cloud data. Point cloud data at different times are aligned using a point cloud registration algorithm, such as the ICP (Iterative Closest Point) algorithm, to construct a more detailed static object model.

[0054] When the road condition information includes a video sequence, target detection processing is performed on the video sequence, that is, a deep learning model is used, for example, including but not limited to: YOLO (You Only Look Once, a real-time target detection system), SSD (Single Shot MultiBox Detector, a real-time target detection model), Mask R-CNN (a model for target detection and instance segmentation), to detect dynamic objects in real time from the image data of the video sequence, and obtain their categories, positions and bounding boxes. Afterwards, multiple targets are tracked for dynamic objects using methods such as Kalman filtering and target association (such as the Hungarian algorithm) to obtain the continuous position change of each object in time, that is, multiple targets are tracked for the identified dynamic objects. Afterwards, a filtering algorithm (such as an extended Kalman filter and a particle filter) is used to estimate the motion trajectory of the tracked target, and the position, velocity and acceleration information of the target in the global coordinate system are recorded, that is, the trajectory estimation result is obtained based on the tracked target. Finally, the discrete target positions are interpolated to generate a continuous motion trajectory. A smoothing algorithm (such as B-spline interpolation) is used to eliminate jitter in the trajectory and obtain a smoother motion path, that is, the motion trajectory of the dynamic object is determined based on the trajectory estimation result. Based on this step, the dynamic object is detected and tracked in real time, and its motion trajectory in space is reconstructed.

[0055] In an embodiment of the present application, the reconstructed dynamic object trajectories can be superimposed on the static scene model, and the dynamic changes in the scene can be updated in real time to construct a comprehensive three-dimensional traffic environment model.

[0056] In a possible implementation, adding the static object to the pre-stored road attribute data can be performed according to the following steps: fusing data of the same static object reconstructed in different time periods on the same road section to obtain a fused static object; and adding the fused static object to the pre-stored road attribute data.

[0057] In this possible implementation, data fusion includes: integrating point cloud data collected at different time points on the same road section, and accumulating these point cloud data into a more complete and detailed three-dimensional scene through feature matching and coordinate alignment. This process can use a point cloud splicing algorithm (such as the ICP algorithm) for data fusion to eliminate errors caused by changes in sensor positions.

[0058] Before adding the fused static object to the pre-stored road attribute data, the following operations may be performed, including but not limited to: Data redundancy elimination: Identify and eliminate redundant point cloud information through multiple scans and data fusion. For example, if there is a high degree of consistency between multiple data points of an object, its redundant data can be eliminated and only the most representative part can be retained to reduce data noise and improve reconstruction efficiency. Comprehensive reconstruction of street scenes: After accumulating data multiple times, the system can perform batch optimization processing to build a more complete road structure and street scene. Batch optimization can not only reconstruct the three-dimensional form of the object, but also accurately restore detailed structures such as lane lines and curbs. This optimization process usually uses global map stitching technology to integrate all available moment fragment data into a global model. Detail enhancement: Through the accumulation of information from multiple perspectives and multiple moments, the details of static objects can be better identified and reconstructed. For example, in some areas with severe occlusion, the data at a single time point may not fully capture the shape of the object. However, through multiple data accumulation, the system can obtain complete information about the object from different angles in different time periods to improve the precision of modeling. Global consistency: Batch optimization can ensure that the reconstruction of road structures and objects is globally consistent. Compared with single-frame reconstruction, the fusion of multi-time information can correct local deviations and generate a more accurate global three-dimensional scene model. This not only helps to reconstruct static objects, but also significantly improves the accuracy of static elements such as lane lines, curbs and marking lines, providing a reliable map foundation for autonomous driving. Robustness enhancement: Multi-time information accumulation can also improve the robustness of the reconstruction system to noise and dynamic interference. For example, temporary obstacles on certain roads (such as construction fences and temporarily parked vehicles) appear in non-fixed positions in multiple scans. These interference factors can be identified and eliminated through temporal data fusion, retaining only the real static structure. Dynamic data update: Road-side equipment can continuously monitor the same road section and collect data regularly to continuously update and improve the scene model. Through each data accumulation and batch optimization, the reconstruction system can dynamically adapt to long-term changes in the environment (such as road reconstruction, building changes, etc.) and maintain the accuracy of reconstruction of static objects.

[0059] In this possible implementation, accumulating information on static objects at multiple times and perspectives can improve scene reconstruction performance. By collecting static data of the same road section at different time points, more comprehensive road structure and surrounding environment information can be obtained, effectively improving the reconstruction accuracy of static objects.

[0060] In a possible implementation, simulation test scene parameters are generated according to the sensor configuration parameters and the road scene parameters, including: generating a vehicle coordinate system according to the sensor configuration parameters; and performing coordinate transformation processing on the road scene parameters according to a coordinate transformation relationship between the road scene coordinate system and the vehicle coordinate system to transform them into the vehicle coordinate system.

[0061] In this possible implementation, the position (usually located at the center of the vehicle or the center of the front axle) and direction of the origin of the vehicle's coordinate system must first be determined, that is, the vehicle coordinate system is generated according to the sensor configuration parameters. The road scene parameters include the first reconstruction data corresponding to the static object and the second reconstruction data of the dynamic object. The first reconstruction data and the second reconstruction data are converted from the coordinate system of the road monitoring device, that is, the road scene coordinate system to the vehicle coordinate system. The real-time posture information (position and orientation) of the vehicle is required to determine the coordinate conversion relationship between the road scene coordinate system and the vehicle coordinate system, and then complete the coordinate conversion.

[0062] First, coordinate transformation of reconstructed data: For each three-dimensional coordinate point (x, y, z) of a static object, coordinate transformation is performed using the vehicle's position information to obtain its position (x', y', z') in the vehicle coordinate system. Second, coordinate transformation of reconstructed data: For the trajectory data of a dynamic object, the corresponding vehicle position is used to transform it at each time point to obtain the motion trajectory of the dynamic object from the vehicle's perspective.

[0063] In this possible implementation, the data of static objects and dynamic objects in the vehicle coordinate system are used to project the three-dimensional scene onto the two-dimensional image plane of the camera through the internal and external parameters of the camera to generate the camera image from the vehicle's perspective. The camera's internal parameters (focal length, principal point position, etc.) and external parameters (camera position and orientation in the vehicle coordinate system) are used to construct a projection matrix to project the three-dimensional point (x', y', z') into the image coordinate system.

[0064] According to the installation position, scanning angle and range of the vehicle's laser radar, the reconstructed three-dimensional obstacle scene is scanned in the vehicle coordinate system to generate a laser radar point cloud. The laser radar's transmission and reception process within a certain angle and distance range is simulated to obtain point cloud data from the vehicle's perspective. Millimeter-wave radar is mainly used to detect dynamic obstacles, so the reconstructed dynamic obstacle trajectory information is extracted in the vehicle coordinate system. According to the radar's detection range and angle, the interaction between the radar wave and the dynamic obstacle from the vehicle's perspective is calculated to simulate the target's reflection signal. The processed sensor signal from the vehicle's perspective is output to form input data that can be used for offline simulation and autonomous driving system verification. These data include: Image data: image sequence from the perspective of the vehicle's camera, including environmental texture, obstacles and other information. Point cloud data: three-dimensional point cloud from the perspective of the vehicle's laser radar, including the spatial distribution and characteristics of obstacles. Radar data: target reflection signal from the perspective of the millimeter-wave radar, including the speed and position information of dynamic obstacles. Through these steps, the reconstructed scene collected by the road-side equipment can be converted into a multi-sensor signal from the vehicle's perspective, providing real perception data for the vehicle's autonomous driving system for simulation testing and verification.

[0065] The implementation process of the method is described below with reference to a specific embodiment.

[0066] Step 1: Detection of long-tail problems on road-side equipment. Deployment of road-side equipment: Install multiple road-side equipment on selected routes (such as urban roads, highways, etc.). These equipment include cameras, radars and other sensing devices for real-time monitoring of road conditions. Data collection: Road-side equipment continuously collects video streams, image data and other sensor data, and uploads these data to the local server for preliminary processing. Detection of long-tail problems: Use the visual text large model deployed on the road-side equipment or local server to analyze the collected images and videos in real time to detect and identify potential long-tail problems, such as traffic accidents, congestion, and special-shaped vehicles (such as over-high or over-wide vehicles). Event labeling and classification: Once a long-tail problem is detected, the system will automatically label it and classify it for archiving. The labeling content includes key information such as event type, time of occurrence, and geographic location.

[0067] Step 2: Cloud-based scene reconstruction and simulation scene generation. Data upload: Upload the detected long-tail problem-related data (such as video clips, image data, annotation information, etc.) to the cloud server for further processing and analysis. Scene reconstruction: The cloud server combines the uploaded actual road data with other existing road models through an integrated scene reconstruction algorithm to reconstruct high-fidelity road scenes. These scenes can cover different weather conditions, lighting environments, and traffic flow conditions. Simulation scene generation: Based on the given vehicle-side device sensor configuration, the cloud system automatically generates simulation scene files. These scene files accurately reflect the long-tail problems on the actual road and ensure that they can be detected and processed by sensors with different configurations (such as cameras, radars, lidars, etc.). Scene verification and optimization: Perform preliminary verification of the generated simulation scene to ensure that it can run effectively in the simulation environment. If problems are found, the system will automatically adjust and optimize the scene file to ensure its high consistency with the actual road scene.

[0068] Step 3: Offline closed-loop simulation verification. Simulation environment preparation: Load the generated scenario file on the offline simulation platform, and prepare the version of the autonomous driving system that needs to be verified. The simulation platform should support the simulation input of multiple sensor data to ensure consistency with the actual vehicle hardware configuration. Closed-loop simulation test: Start the closed-loop simulation test. The system simulates the driving process of the vehicle in different scenarios through the simulation platform to verify the response ability and processing effect of the autonomous driving system to various long-tail problems. The closed-loop simulation includes the simulation of the entire process of perception, decision-making, control, etc., and records every step of the system operation and results. Test result analysis: Collect all data generated during the simulation test, including the system's response time to different scenarios, processing results, potential risk points, etc. By analyzing these data, the performance of the autonomous driving system in various long-tail problem scenarios is evaluated.

[0069] Step 4: Road test performance estimation and indicator calculation. Large-scale data collection: The system collects data for a long time and multiple times on the selected route to ensure the diversity and integrity of the scene data. All data will be used to continuously enrich and optimize the scene database in the cloud. System performance evaluation: Based on the simulation test results, the system will calculate the possible performance of the autonomous driving system in the actual road test. The evaluation content includes key indicators such as system stability, response speed, and decision accuracy. MPI (million-kilometer failure rate) and other indicator calculations: Based on the simulation results, the system calculates important safety indicators such as the MPI of the autonomous driving system. The MPI indicator measures the reliability of the system by counting the number of potential failures per million kilometers during the simulation process. The MPI indicator of the autonomous driving system is calculated through simulation testing to evaluate the stability and safety of the system so as to make necessary improvements. Verification report generation: The system automatically generates a complete verification report, which covers the result analysis of each test scenario, the overall performance evaluation of the system, and early warnings and suggestions for potential risks.

[0070] Step 5: Verification result feedback and system optimization. Feedback and adjustment: Feedback the results of the simulation test to the autonomous driving system development team to help them identify weaknesses and improvements in the system. System optimization: Based on the feedback results, optimize and adjust the algorithms, sensor configurations, etc. of the autonomous driving system. The optimized system version will be re-invested in simulation verification to form a closed-loop development and verification process. Continuous verification and improvement: This application supports continuous verification and optimization. With the collection of new road data and long-tail problem scenarios, the simulation scenario library is continuously enriched to ensure that the autonomous driving system can always cope with the latest road conditions challenges.

[0071] Based on the above steps, comprehensive scenario coverage can be achieved: Automated scenario generation: This application uses a large visual text model to automatically detect long-tail problems on the road through a vehicle-road-cloud collaboration, and converts these actual road conditions into scenarios that can be used for simulation. This automated scenario generation method can cover a wide range of driving environments and emergencies, including long-tail problems such as traffic accidents, congestion, and special-shaped vehicles, ensuring the diversity and comprehensiveness of the scenarios. Customized simulation scenarios: The cloud can generate customized simulation scenarios based on the sensor configuration of the vehicle-side device. This means that no matter how the system is upgraded or the sensor configuration is changed, the corresponding test scenarios can be quickly generated, so as to more flexibly adapt to the verification requirements of different versions of the autonomous driving system. Efficient verification process: Reduced dependence on field testing: Compared with traditional real vehicle verification, the offline simulation of this application greatly reduces the dependence on field testing and reduces the cost and risk of testing. Through the automatic generation and verification of a large number of simulation scenarios, the comprehensive test of the system can be completed in a shorter time, greatly improving the verification efficiency. Closed-loop simulation verification: This application can fully verify various versions of the autonomous driving system through offline closed-loop simulation. This closed-loop simulation can not only test the performance of the system in a safe and controllable environment, but also simulate complex road environments and long-tail problems to ensure the stability and reliability of the system in different scenarios. Higher safety: Avoiding the risk of long-tail problems in the field: In traditional field tests, verifying long-tail problems (such as traffic accidents or emergencies) is often accompanied by higher safety risks.

[0072] This application simulates these long-tail problems in an offline environment by means of simulation scenarios, avoiding safety hazards in field tests and ensuring the safety of the verification process. High-fidelity simulation: By utilizing actual road data and the powerful computing power of the cloud, this application can generate high-fidelity simulation scenarios, making the verification results closer to the actual road conditions, thereby improving the accuracy and credibility of the verification. Low cost and efficient resource utilization. Reduced manual intervention: Compared with manually produced scenarios, the automated scenario generation of this application greatly reduces the need for manual intervention, reducing labor costs and time costs. In addition, based on the computing power of the cloud, a large number of scenarios can be generated and verified at the same time, improving resource utilization efficiency. Large-scale data-driven: This application relies on large-scale actual road test data for scenario generation and verification, which can better reflect the complex situations in the real world, maximize test coverage using existing data resources, and reduce the repetitive costs of traditional field road tests. Dynamic adaptability: Real-time response and update: Since this application relies on the real-time detection capabilities of road-side equipment, it can dynamically update the simulation scenario according to real-time road conditions to ensure that the verification content is always consistent with the current actual road environment. This dynamic adaptability is particularly important in a rapidly changing traffic environment and helps improve the system's adaptability and response capabilities.

[0073] This application solves the problems of insufficient scene coverage, long verification cycle, and inability to effectively handle complex or extreme scenes in existing autonomous driving system verification methods. This application proposes a systematic verification method based on vehicle-road-cloud collaboration, which captures key events on the road in real time, converts them into simulatable scenes, and conducts large-scale offline simulation verification in the cloud to estimate the potential performance of the autonomous driving system on a specific route. Compared with existing methods such as real-vehicle verification, artificial scene creation, and playback based on historical data, this application provides a more comprehensive, efficient, safe, and lower-cost means of verifying autonomous driving systems, especially in dealing with long-tail problems and complex road scenes, showing obvious technical superiority.

[0074] The present application also provides a method for generating simulation test scenario parameters for an autonomous driving system, which can be applied to road monitoring equipment. Figure 2 The flowchart shown is another method for generating simulation test scenario parameters of an autonomous driving system according to an embodiment of the present application. The method may include:

[0075] Step S201 : collecting the road condition information of the target road in real time, wherein the road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data.

[0076] In an embodiment of the present application, the target road is a pre-selected route, such as a city road, a highway, etc. A plurality of road-end devices, namely road detection devices, may be installed on the selected route. These devices include but are not limited to: sensing devices such as cameras and radars, which are used to monitor road conditions in real time. The road detection equipment continuously collects video streams, image data, and other sensor data, and uploads these data to a local server for preliminary processing. Using a large visual text model deployed on the road detection equipment or the local server, the collected images and videos are analyzed in real time to detect and identify potential long-tail problems, such as traffic accidents, congestion, and special-shaped vehicles (such as over-high or over-wide vehicles).

[0077] Road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data. Among them, static objects include fixed structures in the road environment, such as roads, guardrails, buildings, trees, etc. Dynamic objects include traffic participants such as vehicles and pedestrians that generate motion trajectories. Environmental data includes weather environment data, geographical environment data, etc. State data is used to characterize the position, posture and motion state of dynamic objects or static objects.

[0078] Step S202, calling the visual text big model to detect the road condition information and identify the long-tail problem recognition result; wherein the visual text big model is deployed in the road monitoring device or the local server of the autonomous driving system.

[0079] In the embodiment of the present application, the visual text model is a pre-trained model. Based on the trained visual text model, a long-tail scenario file (model output scenario description text, risk factor, and sensor data of the corresponding time period) can be automatically generated, that is, the visual text model is called to detect the road condition information and identify the long-tail problem recognition result.

[0080] It should be noted that expanding the recognition capabilities of road monitoring equipment requires the use of rich quasi-long-tail data, combined with manual screening and annotation, in order to build a higher-quality training data set. Through continuous scene collection and description, the training model can be optimized and the ability to accurately recognize complex long-tail scenes can be improved.

[0081] It should be noted that in the field of autonomous driving, long-tail issues refer to special scenarios or events that occur less frequently but have higher risks in actual road driving, such as traffic accidents, sudden congestion, and special-shaped vehicles. These issues are difficult to fully cover in limited field tests due to their low frequency but high risk characteristics. In this step, long-tail issues are detected in real time through road monitoring equipment and a large visual text model, and are used as an important scenario for verifying the autonomous driving system.

[0082] In this step, the visual text big model is used to detect long-tail problems in real time on the road-side equipment, and the detection results are uploaded to the cloud so that the cloud can reconstruct the scene of the uploaded actual road data through algorithms and generate high-fidelity simulation scene files for offline simulation testing.

[0083] Step S203, sending the road condition information including the long-tail problem identification result to the cloud, so that the cloud generates simulation scenario parameters according to the road condition information including the long-tail problem identification result, pre-stored road attribute data and sensor configuration parameters, and generates a simulation test scenario for the target road, target vehicle and the autonomous driving system based on the simulation test scenario parameters; wherein the road attribute data includes at least one of the orientation, lane data and geographic location of the road, and the sensor configuration parameters are sensor configuration parameters of the target vehicle traveling on the target road in autonomous driving mode.

[0084] In the embodiment of the present application, the pre-stored road attribute data may be data acquired historically, and may be obtained through continuous updating and improvement. The road attribute data includes at least one of the road orientation, lane data, and geographic location. The specific content of the road attribute data may be obtained according to actual needs, and the embodiment of the present application does not specifically limit this.

[0085] An autonomous driving system refers to a system that uses computers, sensors, and artificial intelligence technologies to enable a vehicle to autonomously perform driving operations without human driver intervention. The goal of an autonomous driving system is to improve traffic safety, reduce traffic congestion, improve transportation efficiency, and provide users with a more convenient travel experience. Sensor configuration parameters refer to parameters that need to be set or adjusted when using sensors to ensure that the sensors can work accurately and stably. Different types of sensors have different configuration parameters. Autonomous driving mode refers to the working state and capabilities of an autonomous driving system under different conditions and scenarios. Depending on the degree of automation and application scenarios, autonomous driving modes can be divided into multiple levels and types. By accessing the autonomous driving system, searching for a target vehicle traveling on the target road in autonomous driving mode, and obtaining the sensor configuration parameters of the target vehicle, it is ensured that detection and processing can be performed by sensors with different configurations (such as cameras, radars, lidars, etc.).

[0086] The pre-stored road attribute data and road condition information may be recorded according to the world coordinate system or the fixed road coordinate system. Therefore, the road scene parameters reconstructed based on the two are also data in the world coordinate system or the fixed road coordinate system. The perception data of the vehicle sensor is generally based on the vehicle's own coordinate system (body coordinate system). Therefore, the simulation test scene parameters in the body coordinate system can be generated according to the sensor configuration parameters and the road scene parameters. Then, based on the simulation test scene parameters, a simulation test scene for the target road, the target vehicle and the automatic driving system is generated. In this step, the reconstructed static objects, dynamic objects and road-side sensor data are converted into sensor signals from the perspective of the vehicle in order to simulate the real-time environmental information that the sensor will receive during the vehicle's driving process. This process involves multiple aspects such as coordinate conversion, multi-sensor fusion, and simulated sensor characteristics.

[0087] Generate simulation test scenarios for autonomous driving systems, i.e., scenario reconstruction. Scenario reconstruction refers to processing data on actual roads (such as video clips, image data, etc.) through algorithms to generate high-fidelity virtual scenarios that can run in a simulation environment. These scenarios restore real road conditions as much as possible in order to test the performance of the autonomous driving system in simulation. In this step, the actual long-tail problems detected are converted into simulation scenarios through the scenario reconstruction technology in the cloud to ensure the authenticity and reliability of the test results.

[0088] In this step, offline closed-loop simulation testing is introduced to fully verify the performance of the autonomous driving system in complex scenarios by running pre-generated long-tail problem scenarios on the simulation platform. The closed-loop simulation covers the entire process from perception, decision-making to control, ensuring that the system can respond correctly to various road conditions that may be encountered. Based on the simulation test results, the system can infer the performance of the autonomous driving system in actual road tests and calculate key safety indicators such as MPI. Through continuous data collection and scenario optimization, this application supports dynamic adjustment and improvement of simulation scenarios and verification schemes to ensure that the system is always in the best condition.

[0089] The embodiment of the present application integrates road-side and cloud data, reconstructs road scene parameters, and then combines the sensor configuration parameters on the vehicle side to obtain simulation test scene parameters, so that the simulation test scene covers a wide range of driving environments and emergencies, ensuring the diversity and comprehensiveness of the scene, and then used to achieve efficient offline verification of the autonomous driving system.

[0090] In one possible implementation, a large visual text model is called to detect the road condition information and identify long-tail problem recognition results, including: using a pre-trained large visual text model to detect the road condition information, identifying one or more events including traffic accidents, congestion and irregular vehicles, at least marking the type, occurrence time, geographic location and risk factor of the event, and obtaining a long-tail problem recognition result.

[0091] In this possible implementation, the specific practices that can be used when pre-training a large model are: 1. Screen out clips containing possible long-tail scenes, such as traffic accidents, vehicle breakdowns, congestion, etc. This step does not require in-depth and detailed annotations, but only needs to determine whether there are potential long-tail scenes in the data, thereby reducing screening time and labor costs; 2. Use natural language to describe the scene in detail, for example: "There was a car accident ahead, the left lane was closed, and the vehicle slowed down to avoid it." After that, a unified data storage system is established to pair video clips with text descriptions to ensure that each video clip has an accurate natural language description. Use timestamps and event identifiers to accurately locate specific events in the video to ensure data consistency and relevance.

[0092] In this step, once a long-tail problem is detected, it can be automatically labeled and classified and archived to obtain the long-tail problem identification result. The labeled content at least includes key information such as event type, occurrence time, geographical location and risk factor.

[0093] In a possible implementation, the road condition information and the long-tail problem identification result are sent to the cloud, including: determining the time when the long-tail problem occurs based on the long-tail problem identification result; determining a time period including the time of occurrence, and sending the road condition information collected within the time period and the long-tail problem identification result to the cloud.

[0094] In this possible implementation, the road condition information corresponding to the time period including the time when the long-tail problem occurs and the long-tail problem identification result can be sent to the cloud instead of sending all the road condition information to the cloud, which greatly reduces the amount of data sent, saves resources and improves data processing efficiency. The length of the time period can be set according to actual needs, and the embodiment of the present application does not specifically limit this.

[0095] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a simulation test scenario parameter generation device for an autonomous driving system. Figure 3 The figure is a structural block diagram of a simulation test scenario parameter generation device for an automatic driving system according to an embodiment of the present application, and the device may include:

[0096] An acquisition module 301 is provided for acquiring road condition information transmitted in real time by at least one road monitoring device in a target road; the road condition information includes a long-tail problem recognition result; the long-tail problem recognition result is obtained by the road monitoring device by calling a large visual text model to recognize the road condition information, and the large visual text model is deployed in the local server of the road monitoring device or the autonomous driving system; a reconstruction module 302 is provided for reconstructing road scene parameters based on pre-stored road attribute data and the road condition information; the road condition information includes at least one of the state data of dynamic objects, the state data of static objects and environmental data in the road, and the road attribute data includes at least one of the orientation, lane data and geographical location of the road; a search module 303 is provided for searching a target vehicle traveling on the target road in an autonomous driving mode by accessing the autonomous driving system, and obtaining the sensor configuration parameters of the target vehicle; a generation module 304 is provided for generating simulation test scene parameters according to the sensor configuration parameters and the road scene parameters, so as to generate a simulation test scene for the target road, the target vehicle and the autonomous driving system based on the simulation test scene parameters.

[0097] According to an embodiment of the present application, the road condition information transmitted in real time by at least one road monitoring device in the target road is obtained; the road condition information includes a long-tail problem identification result; the long-tail problem identification result is obtained by the road monitoring device by calling a visual text big model to identify the road condition information, and the visual text big model is deployed in the local server of the road monitoring device or the autonomous driving system, so as to obtain the long-tail problems on the road automatically detected by the visual text big model transmitted by the road detection device, so as to convert these actual road conditions into scenes that can be used for simulation. Afterwards, the road scene parameters are reconstructed based on the pre-stored road attribute data and the road condition information; the road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data, and the road attribute data includes at least one of the orientation, lane data and geographical location of the road; by accessing the autonomous driving system, the target vehicle traveling on the target road in the autonomous driving mode is searched, and the sensor configuration parameters of the target vehicle are obtained; the simulation test scene parameters are generated according to the sensor configuration parameters and the road scene parameters, so as to generate the simulation test scene for the target road, the target vehicle and the autonomous driving system based on the simulation test scene parameters. The embodiment of the present application integrates road-side and cloud data, reconstructs road scene parameters, and then combines the sensor configuration parameters on the vehicle side to obtain simulation test scene parameters, so that the simulation test scene covers a wide range of driving environments and emergencies, ensuring the diversity and comprehensiveness of the scene, and then used to achieve efficient offline verification of the autonomous driving system.

[0098] In a possible implementation, reconstructing road scene parameters based on pre-stored road attribute data and the road condition information includes: reconstructing the motion trajectories of static objects and dynamic objects in the road scene respectively based on the road condition information; and adding first reconstructed data of the static objects and second reconstructed data of the motion trajectories of the dynamic objects to the pre-stored road attribute data.

[0099] In a possible implementation, simulation test scene parameters are generated according to the sensor configuration parameters and the road scene parameters, including: generating a vehicle coordinate system according to the sensor configuration parameters; and performing coordinate transformation processing on the road scene parameters according to a coordinate transformation relationship between the road scene coordinate system and the vehicle coordinate system to transform them into the vehicle coordinate system.

[0100] In one possible implementation, based on the road condition information, the motion trajectories of static objects and dynamic objects in the road scene are reconstructed separately, including: performing point cloud segmentation processing on the point cloud data in the road condition information to determine the point cloud data of the static object, and reconstructing the static object based on the point cloud data of the static object; using a deep learning model to identify dynamic objects from a video sequence of the road condition information, performing multi-target tracking on the identified dynamic objects, obtaining trajectory estimation results based on the tracked targets, and determining the motion trajectory of the dynamic object based on the trajectory estimation results.

[0101] In a possible implementation, adding the static object to the pre-stored road attribute data includes: fusing data of the same static object reconstructed in different time periods on the same road section to obtain a fused static object; and adding the fused static object to the pre-stored road attribute data.

[0102] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a simulation test scenario parameter generation device for an autonomous driving system. Figure 4 The figure is a structural block diagram of a simulation test scenario parameter generation device for an automatic driving system according to an embodiment of the present application, and the device may include:

[0103] The acquisition module 401 is used to acquire the road condition information of the target road in real time, wherein the road condition information includes at least one of the status data of dynamic objects in the road, the status data of static objects and environmental data; the recognition module 402 is used to call the visual text large model to detect the road condition information and identify the long-tail problem recognition result; wherein the visual text large model is deployed in the road monitoring device or the local server of the autonomous driving system; the sending module 403 is used to send the road condition information including the long-tail problem recognition result to the cloud, so that the cloud generates simulation scenario parameters according to the road condition information including the long-tail problem recognition result, pre-stored road attribute data and sensor configuration parameters, and generates a simulation test scenario for the target road, the target vehicle and the autonomous driving system based on the simulation test scenario parameters; wherein the road attribute data includes at least one of the road orientation, lane data and geographical location, and the sensor configuration parameters are the sensor configuration parameters of the target vehicle traveling on the target road in autonomous driving mode.

[0104] The embodiment of the present application integrates road-side and cloud data, reconstructs road scene parameters, and then combines the sensor configuration parameters on the vehicle side to obtain simulation test scene parameters, so that the simulation test scene covers a wide range of driving environments and emergencies, ensuring the diversity and comprehensiveness of the scene, and then used to achieve efficient offline verification of the autonomous driving system.

[0105] In one possible implementation, a large visual text model is called to detect the road condition information and identify long-tail problem recognition results, including: using a pre-trained large visual text model to detect the road condition information, identifying one or more events including traffic accidents, congestion and irregular vehicles, at least marking the type, occurrence time, geographic location and risk factor of the event, and obtaining a long-tail problem recognition result.

[0106] In a possible implementation, the road condition information and the long-tail problem identification result are sent to the cloud, including: determining the time when the long-tail problem occurs based on the long-tail problem identification result; determining a time period including the time of occurrence, and sending the road condition information collected within the time period and the long-tail problem identification result to the cloud.

[0107] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0108] Figure 5 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 5 As shown, the electronic device includes: a memory 501 and a processor 502. The memory 501 stores a computer program that can be run on the processor 502. When the processor 502 executes the computer program, the method in the above embodiment is implemented. The number of the memory 501 and the processor 502 can be one or more.

[0109] The electronic device also includes:

[0110] The communication interface 503 is used to communicate with external devices and perform data exchange transmission.

[0111] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the memory 501, the processor 502 and the communication interface 503 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0112] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0113] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0114] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0115] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0116] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0117] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0118] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0119] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0121] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0122] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0123] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0124] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0125] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0126] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0127] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating simulation test scenario parameters of an autonomous driving system, applied to a cloud computing platform, the method comprising: Obtaining road condition information transmitted in real time by at least one road monitoring device on the target road; The traffic information includes long-tail problem identification results; The long-tail problem recognition result is obtained by the road monitoring device by calling a large visual text model to recognize the road condition information, and the large visual text model is deployed in the road monitoring device or a local server of the autonomous driving system; Reconstructing road scene parameters based on pre-stored road attribute data and the road condition information; the road condition information includes at least one of state data of dynamic objects in the road, state data of static objects and environmental data, and the road attribute data includes at least one of the direction of the road, lane data and geographical location; By accessing the autonomous driving system, searching for a target vehicle traveling on the target road in an autonomous driving mode, and obtaining sensor configuration parameters of the target vehicle; Generate simulation test scenario parameters according to the sensor configuration parameters and the road scenario parameters, so as to generate a simulation test scenario for the target road, the target vehicle and the autonomous driving system based on the simulation test scenario parameters.

2. The method according to claim 1, wherein: Reconstructing road scene parameters based on pre-stored road attribute data and the road condition information includes: Based on the road condition information, reconstruct the motion trajectories of static objects and dynamic objects in the road scene respectively; The first reconstructed data of the static object and the second reconstructed data of the motion trajectory of the dynamic object are added to the pre-stored road attribute data.

3. The method according to claim 1, wherein: Generating simulation test scene parameters according to the sensor configuration parameters and the road scene parameters includes: generating a vehicle coordinate system according to the sensor configuration parameters; According to the coordinate conversion relationship between the road scene coordinate system and the vehicle coordinate system, coordinate conversion processing is performed on the road scene parameters to convert them into the vehicle coordinate system.

4. The method according to claim 2, wherein: Based on the road condition information, reconstructing the motion trajectories of static objects and dynamic objects in the road scene respectively includes: Performing point cloud segmentation processing on the point cloud data in the road condition information to determine point cloud data of a static object, and reconstructing the static object based on the point cloud data of the static object; A deep learning model is used to identify dynamic objects from the video sequence of the road condition information, multi-target tracking is performed on the identified dynamic objects, trajectory estimation results are obtained based on the tracked targets, and the motion trajectory of the dynamic objects is determined based on the trajectory estimation results.

5. The method according to claim 2, wherein: Adding the static object to the pre-stored road attribute data includes: Perform data fusion on the same static object reconstructed in different time periods on the same road section to obtain a fused static object; The fused static object is added to the pre-stored road attribute data.

6. A method for generating simulation test scenario parameters of an autonomous driving system, applied to a road monitoring device, the method comprising: Collecting the road condition information of the target road in real time, wherein the road condition information includes at least one of the state data of dynamic objects in the road, the state data of static objects and environmental data; Calling a large visual text model to detect the road condition information and identify a long-tail problem recognition result; wherein the large visual text model is deployed in the road monitoring device or a local server of the autonomous driving system; The road condition information including the long-tail problem identification result is sent to the cloud, so that the cloud generates simulation scenario parameters according to the road condition information including the long-tail problem identification result, pre-stored road attribute data and sensor configuration parameters, and generates a simulation test scenario for the target road, target vehicle and the autonomous driving system based on the simulation test scenario parameters; wherein the road attribute data includes at least one of the direction of the road, lane data, and geographic location, and the sensor configuration parameters are sensor configuration parameters of the target vehicle traveling on the target road in autonomous driving mode.

7. The method according to claim 6, wherein: The visual text model is called to detect the road condition information and identify the long-tail problem recognition results, including: The road condition information is detected using a pre-trained visual text model to identify one or more events including traffic accidents, congestion and irregular vehicles, and at least the type, occurrence time, geographic location and risk factor of the event are marked to obtain a long-tail problem recognition result.

8. The method according to claim 6, wherein: Sending the traffic information and the long-tail problem identification result to the cloud includes: Based on the long-tail problem identification result, determine the time when the long-tail problem occurs; Determine a time period including the occurrence time, and send the road condition information collected within the time period and the long-tail problem identification result to the cloud.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images