Simulation test method and related device

Abstract

The invention provides a simulation testing method and a related device, and relates to the field of simulation testing. The method comprises the following steps: executing positioning operation by utilizing a simulation map to obtain initial positioning information, subsequently performing path planning operation to obtain initial navigation information, converting the initial navigation information and the initial positioning information into target navigation information and target positioning information, and performing simulation operation on a driving scene of a simulation vehicle according to the initial positioning information. And the follow-up target NOA controller controls the simulation vehicle to run to the destination by using the target navigation information, the target positioning information and the driving scene. And subsequently, the target NOA controller can be used for controlling the control information of the operation of the simulation vehicle and the operation information of the simulation vehicle to realize the test of the target NOA controller. Besides, the simulation map is obtained through scene generalization, various complex conditions possibly encountered in the driving process can be reflected, then the method can be applied to complex and large-amount city scenes, and simulation testing of the city NOA is achieved.

Description

Technical Field

[0001] This application relates to the field of simulation testing, and more specifically, to a simulation testing method and related apparatus. Background Technology

[0002] City NOA (Navigate on Autopilot), as a higher level of L2 autonomous driving, has become a key research direction for major intelligent driving manufacturers. In order to improve the control level of city NOA, simulation testing has become an indispensable and important step.

[0003] In practical applications, the scenarios that need to be handled during urban NOA testing are not only numerous but also highly complex. Therefore, how to achieve simulation testing of urban NOA is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, this application provides a simulation testing method and related apparatus to solve the problem of the urgent need to realize simulation testing of urban NOA.

[0005] To solve the above-mentioned technical problems, this application adopts the following technical solution:

[0006] The first aspect of this application provides a simulation testing method, including:

[0007] A simulation map is generated; the simulation map is obtained through scene generalization operations.

[0008] In response to the navigation command for the destination, a positioning operation is performed using the simulation map to obtain the initial positioning information of the simulation vehicle;

[0009] Based on the simulation map, the initial positioning information, and the destination, a route planning operation is performed to obtain the initial navigation information of the simulation vehicle.

[0010] The initial navigation information and the initial positioning information are converted into target navigation information and target positioning information that can be recognized by the target navigation and autopilot NOA controller, and the target navigation information and the target positioning information are sent to the target NOA controller;

[0011] Based on the initial positioning information, the driving scenario of the simulated vehicle is simulated, and the simulated driving scenario is output to the target NOA controller, so that the target NOA controller uses the target navigation information, the target positioning information, and the driving scenario to control the simulated vehicle to the destination; the control information of the target NOA controller to control the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

[0012] In one possible implementation, generating the simulation map includes:

[0013] Acquire data from multiple sensors;

[0014] Process the collected data according to the corresponding processing method;

[0015] The data after the processing is performed is fused to obtain map scene tile data. During the data fusion operation, point cloud data is overlaid with semantic layers from the collected image information in the same coordinate system. Combined with inertial navigation data, the 3D scene is reconstructed to obtain map scene tile data.

[0016] A scene generalization operation is performed on the map scene tile data to obtain a simulation map.

[0017] In one possible implementation, a scene generalization operation is performed on the map scene tile data to obtain a simulation map, including:

[0018] A scene generalization algorithm is used to perform scene generalization operations on the map scene tile data to obtain a simulation map;

[0019] The operating logic of the scenario generalization algorithm is as follows:

[0020] Random scenarios are generated using constraints, and simulation maps are constructed using these random scenarios.

[0021] In one possible implementation, a positioning operation is performed using the simulation map to obtain the initial positioning information of the simulated vehicle, including:

[0022] Acquire environmental simulation images of the simulated vehicle;

[0023] The environmental simulation image is subjected to feature extraction to obtain multi-scale features of the environmental simulation image;

[0024] Using the multi-scale features, candidate regions where the simulated vehicle is located are determined from the simulation map;

[0025] The candidate regions are classified and regressed to obtain the initial positioning information of the simulated vehicle.

[0026] In one possible implementation, a route planning operation is performed based on the simulation map, the initial positioning information, and the destination to obtain the initial navigation information of the simulation vehicle, including:

[0027] Obtain the road network of the simulation map;

[0028] Using vertex separators, a coverage map including the initial positioning information and the destination is determined from the road network;

[0029] The vertex separator is divided into small subsets of vertices to decompose the cover graph into multiple balanced units;

[0030] Using the multiple balancing units, a path planning operation is performed to obtain the initial navigation information of the simulated vehicle; the starting point of the initial navigation information is the initial positioning information and the ending point is the destination.

[0031] In one possible implementation, the multiple balancing units are used to perform path planning operations to obtain the initial navigation information of the simulated vehicle, including:

[0032] The navigation route is determined through path planning operations, passing through at least one balancing unit; the starting point of the navigation route is the initial positioning information and the ending point is the destination.

[0033] The driving route within the balance unit through which the navigation route passes is determined, thereby obtaining the initial navigation information of the simulated vehicle.

[0034] In one possible implementation, the initial navigation information and the initial positioning information are converted into target navigation information and target positioning information that can be recognized by the target NOA controller, including:

[0035] Obtain the target data format used by the target NOA controller;

[0036] The initial navigation information and the initial positioning information are converted into target navigation information and target positioning information in the target data format.

[0037] In one possible implementation, based on the initial positioning information, a simulation operation is performed on the driving scenario of the simulated vehicle, including:

[0038] Based on the initial positioning information and the map within a preset range of the initial positioning information, render the environmental information within the preset range of the simulated vehicle;

[0039] The simulation includes the vehicle's operational information and sensor scene information.

[0040] The driving scenario of the simulated vehicle is constructed based on the environmental information, the operational information, and the sensor scene information.

[0041] A second aspect of this application provides a simulation testing apparatus, comprising:

[0042] The map generation module is used to generate a simulation map; the simulation map is obtained through scene generalization operation.

[0043] The positioning module is used to perform positioning operations using the simulation map in response to navigation instructions for the destination, and to obtain the initial positioning information of the simulation vehicle.

[0044] The navigation module is used to perform route planning based on the simulation map, the initial positioning information, and the destination to obtain the initial navigation information of the simulation vehicle.

[0045] The conversion module is used to convert the initial navigation information and the initial positioning information into target navigation information and target positioning information that can be recognized by the target NOA controller, and send the target navigation information and the target positioning information to the target NOA controller;

[0046] The testing module is used to simulate the driving scenario of the simulated vehicle based on the initial positioning information, and output the simulated driving scenario to the target NOA controller, so that the target NOA controller uses the target navigation information, the target positioning information and the driving scenario to control the simulated vehicle to the destination; the control information of the target NOA controller to control the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

[0047] A third aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0048] The memory is used to store computer programs;

[0049] The processor is used to execute the computer program so that the electronic device can implement the above-described simulation testing method.

[0050] This application provides a simulation testing method and related apparatus. Responding to a navigation command to a destination, a positioning operation is performed using a simulation map to obtain initial positioning information of the simulated vehicle. Based on the simulation map, the initial positioning information, and the destination, a path planning operation is performed to obtain initial navigation information of the simulated vehicle. The initial navigation information and the initial positioning information are converted into target navigation information and target positioning information that can be recognized by the target NOA controller. The target navigation information and the target positioning information are sent to the target NOA controller. Based on the initial positioning information, a driving scenario of the simulated vehicle is simulated, and the driving scenario is output to the target NOA controller, enabling the target NOA controller to control the simulated vehicle to travel to the destination using the target navigation information, the target positioning information, and the driving scenario. Subsequently, the control information and driving information of the simulated vehicle controlled by the target NOA controller can be used to test the target NOA controller. Furthermore, the simulation map in this application is obtained through scenario generalization, which can reflect various complex situations that may be encountered during driving, and thus can be applied to complex and large-scale urban scenarios to achieve simulation testing of urban NOA. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 A flowchart illustrating a simulation testing method provided in this application embodiment;

[0053] Figure 2 A module interaction diagram provided for an embodiment of this application;

[0054] Figure 3 A scenario generalization flowchart provided for an embodiment of this application;

[0055] Figure 4 A positioning flowchart provided for an embodiment of this application;

[0056] Figure 5 A navigation flowchart provided for an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the structure of a simulation testing device provided in an embodiment of this application;

[0058] Figure 7This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Highway NOA (Navigate on Autopilot) typically combines features such as adaptive cruise control, lane keeping assist, and automatic lane changing. Compared to highway NOA, urban NOA (Navigate on Autopilot), representing a higher level of Level 2 autonomous driving, has become a key research focus for major autonomous driving manufacturers. To improve the control level of urban NOA, simulation testing has become an indispensable and crucial step. Furthermore, the increasing demand for mass production of urban NOA has also created a need for its simulation testing.

[0061] In practical applications, urban NOA (Noise, Arrangement, and Automation) needs to handle not only a large number of scenarios but also high complexity. For example, intersections are not necessarily straight lines but may be oddly shaped. To achieve mass production, they need to be tested in numerous real-world logical scenarios. Currently used methods such as HIL (hardware-in-the-loop simulation) and SIL (software-in-the-loop simulation) mostly involve manually customized simulation scenarios, resulting in a small number of scenarios that are difficult to meet the demands of large-scale and highly complex scenarios. Therefore, how to achieve simulation testing for urban NOA is a technical problem that urgently needs to be solved by those skilled in the art.

[0062] To address this, this application provides a simulation testing method and related apparatus. Responding to a navigation command from a destination, a positioning operation is performed using a simulation map to obtain the initial positioning information of the simulated vehicle. Based on the simulation map, the initial positioning information, and the destination, a path planning operation is performed to obtain the initial navigation information of the simulated vehicle. The initial navigation information and initial positioning information are converted into target navigation information and target positioning information that can be recognized by the target NOA controller. The target navigation information and target positioning information are sent to the target NOA controller. Based on the initial positioning information, a simulation operation is performed on the driving scenario of the simulated vehicle, and the driving scenario is output to the target NOA controller, enabling the target NOA controller to control the simulated vehicle to reach its destination using the target navigation information, target positioning information, and driving scenario. Subsequently, the control information and driving information of the simulated vehicle controlled by the target NOA controller can be used to test the target NOA controller. Furthermore, the simulation map in this application is obtained through scenario generalization, which can reflect various complex situations that may be encountered during driving, and thus can be applied to complex and large-scale urban scenarios to achieve simulation testing of urban NOA.

[0063] The simulation process of this application can employ SIL simulation technology.

[0064] In one implementation of this application, a simulation testing method is provided, wherein the execution subject can be a simulation controller.

[0065] Reference Figure 1 A simulation testing method may include:

[0066] S11. Generate simulation map.

[0067] The simulation map is obtained through scene generalization operations and can reflect various complex situations that may be encountered during driving. Therefore, it can be applied to complex and large-scale urban scenarios to realize simulation testing of urban NOA (Noise of Alertness).

[0068] In one implementation, refer to Figure 2 The simulation controller includes a cloud map scene generalization module 10, a NOA scheme generalization simulation module 20, a software vehicle-in-the-loop simulation module 30, and a NOA algorithm under test module 40. Each module performs a different function.

[0069] The cloud map scene generalization module 10 is deployed on a private cloud server hardware and software platform. It uses a medium-power computing server as the hardware carrier, and the software platform is divided into a scheduling layer, a high-performance computing layer, and a storage cluster as carriers.

[0070] In the early stages of NOA algorithm development, the test vehicle collects and outputs various sensor information, primarily including video data, LiDAR data, and inertial navigation data. The cloud-based map scene generalization module 10 can identify key features, extract scenes, and mine scenes from various sensor information. Combining this with knowledge, experience, and regulations, and processed by the simulation scene generalization algorithm, it outputs a large number of simulation maps required for NOA testing. These simulation maps reflect various complex situations that may be encountered during urban driving, generating XML (Extensible Markup Language) and xosc files for the simulation maps, which are then uniformly saved and managed within the cloud-based map scene generalization module 10. The cloud-based map scene generalization module 10 can also output simulation maps to the NOA solution generalization simulation module 20 and the software vehicle-in-the-loop simulation module 30.

[0071] S12. In response to the navigation command of the destination, perform a positioning operation using the simulation map to obtain the initial positioning information of the simulation vehicle.

[0072] The NOA solution generalization simulation module 20 is deployed on a local graphical workstation running a Linux system. It includes an AI (Artificial Intelligence) model inference environment, with the AI ​​model inference algorithm code stored in an image container. It also configures the AI ​​inference dependency, model conversion dependency, and third-party library dependency environments. The graphical workstation is equipped with a 4G network card and connects to the cloud-based map scene generalization module 10 via an APN (Access Point Name) dedicated line. The graphical workstation receives the simulation map generated by the cloud-based map scene generalization module 10. Based on the simulation map, the NOA solution generalization simulation module 20 performs positioning and navigation operations, feeding back the obtained positioning and navigation information to the cloud-based map scene generalization module 10. The cloud-based map scene generalization module 10 can filter maps within a preset range (e.g., a 10-kilometer radius) based on the positioning information and provide this to the software vehicle-in-the-loop simulation module 30. The cloud-based map scene generalization module 10 can also send the positioning and navigation information to the NOA algorithm under test module 40.

[0073] The software vehicle-in-the-loop simulation module 30 is stored on local graphics workstations deployed across multiple Linux systems. Each local workstation is equipped with a 4G network card. These workstations are connected to the cloud-based map scene generalization module 10 via an APN leased line, employing a divide-and-conquer approach to interact with massive amounts of data. One set of workstations runs scene simulation software (such as VTD (VIRES Virtual Test Drive)) and downloads the NOA simulation map opendrive .xodr format file from the cloud-based map scene generalization module 10 via the APN leased line, then uses the scene simulation software for scene rendering. Another set of workstations runs vehicle dynamics simulation software based on a Windows environment. This set of workstations communicates with the VTD-running workstations via Ethernet cables and routers, exchanging vehicle dynamics model information and sensor scene information to complete the generation and rendering of the NOA simulation map within the simulation scene.

[0074] The NOA (Noise, Arrival, and Occupation) test algorithm module 40 is equipped with a NOA controller under test. The NOA controller undergoes corresponding testing before being installed in the vehicle in mass production. The test algorithm is deployed on a Linux system environment graphical workstation, written in C++ code, and interconnected with the cloud map scene generalization module 10 and the software vehicle-in-the-loop simulation module 30 via Ethernet cable and router. It receives map scene, sensor information, and operation information from the software vehicle-in-the-loop simulation module 30, and receives navigation and positioning information from the cloud map scene generalization module 10. It outputs the vehicle's motion control information to the software vehicle-in-the-loop simulation module 30 to realize simulated vehicle navigation in the simulation scene and complete the urban NOA function simulation test.

[0075] Based on the functional descriptions of the modules above, a local graphics workstation running the NOA solution generalization simulation module 20 is constructed. This workstation features a simulated in-vehicle navigation map selection interface. Users can search for a destination on this interface and then click the navigation button to receive navigation instructions based on that destination. Subsequently, the NOA solution generalization simulation module 20 uses the simulated map to perform positioning operations, obtaining the current location information of the simulated autonomous vehicle. This current location information serves as the initial positioning information of the simulated vehicle.

[0076] In real-world scenarios, the initial positioning information takes into account the errors of the positioning device, making the initial positioning information close to the actual positioning result of the positioning device.

[0077] S13. Based on the simulation map, initial positioning information, and destination, perform path planning to obtain the initial navigation information of the simulation vehicle.

[0078] In this embodiment, given the known simulation map, the current position of the simulated vehicle (i.e., initial positioning information), and the destination of the simulated vehicle, the NOA scheme generalization simulation module 20 can be used for navigation planning. Specifically, the NOA scheme generalization simulation module 20 performs global path planning and local path planning. The trajectory planning generates a sequence from the current state of the autonomous vehicle to the next target state. This sequence specifies the evolution of the vehicle's state over time, satisfies the kinematic and dynamic constraints of the simulated vehicle, and avoids collisions with static and moving obstacles in the simulation environment.

[0079] The results of global and local path planning performed by the NOA scheme generalization simulation module 20 constitute the initial navigation information of the simulated vehicle. This initial navigation information includes all states of the simulated vehicle from its current location to its destination.

[0080] S14. Convert the initial navigation information and initial positioning information into target navigation information and target positioning information that the target NOA controller can recognize, and send the target navigation information and target positioning information to the target NOA controller.

[0081] In this embodiment, current NOA implementation schemes from various manufacturers differ significantly in their core positioning and navigation planning technologies. Some manufacturers rely on pre-configured maps for positioning and navigation. These pre-configured maps can be high-precision maps, lightweight maps, or crowdsourced maps, etc. Other manufacturers do not pre-configure maps. For positioning and navigation without pre-configured maps, they generally use a memory map obtained by fusing multi-sensor data. This memory map is then used for positioning and navigation. During navigation, camera data can be used to estimate the autonomous vehicle's position relative to the map. The process of obtaining the memory map by fusing multi-sensor data involves fusing data collected by radar (LIDAR) and camera data, and then using SLAM (Simultaneous Localization and Mapping) to obtain the memory map.

[0082] As can be seen from the above, due to the different map formats or generation methods used by various R&D manufacturers, the formats and contents of their maps differ. Therefore, when testing NOA controllers from different R&D manufacturers, the target NOA controller needs to use the map data of the R&D manufacturer in which it is located during operation. Thus, the initial navigation information and initial positioning information constructed in this application need to be converted to the map format used by the corresponding R&D manufacturer.

[0083] In one implementation, the target data format used by the target NOA controller can be obtained. This target data format is the map format used by the R&D manufacturer that develops the target NOA controller. This format can be any of the general formats.

[0084] Then, format conversion technology is applied to convert the initial navigation information and initial positioning information into target navigation information and target positioning information in the target data format. Since the obtained target navigation information and target positioning information are in the target data format of the map used by the manufacturer of the target NOA controller, the target NOA controller can correctly identify and use the target navigation information and target positioning information, and send the target navigation information and target positioning information to the target NOA controller in the NOA test algorithm module 40, so that it can use the target navigation information and target positioning information to perform automatic vehicle control operations.

[0085] S15. Based on the initial positioning information, perform simulation operations on the driving scenario of the simulated vehicle, and output the simulated driving scenario to the target NOA controller, so that the target NOA controller can use the target navigation information, target positioning information and driving scenario to control the simulated vehicle to run to the destination.

[0086] Among them, the control information of the target NOA controller controlling the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

[0087] In this embodiment, in addition to target navigation information and target positioning information, the target NOA controller also needs to simulate the driving scenario of the simulated vehicle. The driving scenario is simulated by the software vehicle-in-the-loop simulation module 30 using the initial positioning information. The driving scenario generally includes the vehicle's driving environment, operating information such as vehicle speed and acceleration, as well as data collected by sensors, etc., and then outputs the driving scenario to the target NOA controller.

[0088] In a real-world scenario, after the NOA solution generalization simulation module 20 generates the planned path, clicking the "Smart Driving" button activates the NOA controller's city NOA function. Based on the destination set by the driver on the map, the city NOA function uses the simulation map and vehicle positioning to plan a navigation route, controlling the simulated vehicle to automatically drive along the navigation route within the simulation scenario. It outputs the vehicle's motion control information to the software vehicle-in-the-loop simulation module 30, thus achieving simulated vehicle navigation within the simulation scenario. During automatic driving, it can perform operations such as cruise control, following the vehicle in front, automatic lane changing, or stopping to avoid obstacles, and provides real-time road condition updates to the NOA test algorithm module 40. When approaching the end of the road covered by the current NOA function, the NOA controller exits the NOA function.

[0089] During operation, the target NOA controller collects control information and operational information about the simulated vehicle. This collected information serves as reference data for testing the target NOA controller. Subsequently, this data can be compared with the corresponding standard information in the test cases. If they match, the city NOA function is normal, and the test passes. If they do not match, the city NOA function is abnormal, and the test fails.

[0090] In this embodiment, in response to a navigation command to the destination, a positioning operation is performed using a simulated map to obtain the initial positioning information of the simulated vehicle. Based on the simulated map, the initial positioning information, and the destination, a path planning operation is performed to obtain the initial navigation information of the simulated vehicle. The initial navigation information and initial positioning information are converted into target navigation information and target positioning information that the target NOA controller can recognize. The target navigation information and target positioning information are then sent to the target NOA controller. Based on the initial positioning information, a driving scenario simulation operation is performed on the simulated vehicle, and the driving scenario is output to the target NOA controller. This allows the target NOA controller to control the simulated vehicle to its destination using the target navigation information, target positioning information, and driving scenario. Subsequently, the control information and driving information of the simulated vehicle controlled by the target NOA controller can be used to test the target NOA controller.

[0091] In addition, the simulation map in this application is obtained through scene generalization, which can reflect various complex situations that may be encountered during the driving process, and thus can be applied to complex and large-scale urban scenarios to realize the simulation test of urban NOA.

[0092] Based on any of the above embodiments, in one implementation, referencing Figure 3 The generation of the simulation map includes:

[0093] S21. Acquire data collected by multiple sensors.

[0094] Specifically, the cloud-based map scene generalization module 10 includes a crowdsourced map tile scene processing module and a generalization algorithm simulation map scene generation module. The crowdsourced map tile scene processing module is used to generate map scene tile data.

[0095] The crowdsourced map tile scene processing module is configured in the scene tiling tool, and the crowdsourced map tile scene processing module is configured with corresponding scene editor and map editor software.

[0096] The test vehicle will collect and output various sensor information. The sensor data mainly includes video data collected by the camera, point cloud data collected by the lidar, positioning data collected by the Global Positioning System (GPS), and inertial navigation data collected by the IMU (Inertial Measurement Unit).

[0097] S22. Process the collected data according to the processing method corresponding to the collected data.

[0098] The video data processing methods include distortion correction and projection transformation. The point cloud data processing method for LiDAR acquisition involves: obtaining motion information of the target object from the raw LiDAR point cloud; performing data preprocessing, calibration, stitching, filtering and downsampling, ground point removal, point cloud segmentation, clustering, and coordinate alignment to segment and classify the target point cloud; and then using an improved BP (BackPropagation) neural network to classify the target point cloud. The improvement to the BP neural network mainly involves filtering the characteristics of the output target to obtain the important elements and target information needed for scene generation. Important elements and target information include, but are not limited to:

[0099] Vector information and 3D model information such as road surveying data: standard data identifiers for urban main roads and roads, including ID, type, duration (Age), coordinates (x, y, z), and dimensions (width W, height H, length L).

[0100] Data streams from roadside trees, green belts, vehicle spacing, pedestrian crossings, guardrails, traffic lights, signs, buildings, etc.

[0101] S23. Perform data fusion on the data after the processing operation to obtain map scene tile data.

[0102] In the data fusion operation, point cloud data and semantic layers from collected image information are overlaid in the same coordinate system, and combined with inertial navigation data, a 3D scene is reconstructed to obtain map scene tile data.

[0103] Specifically, RTK (Real-Time Kinematic positioning data) positioning data is used as the comparison data for LiDAR target recognition. After multi-sensor coordinate transformation, multi-sensor target fusion is performed. During fusion, GPS time is used to synchronize video data, point cloud data, and inertial navigation data, accurately outputting the positional relationships between target information. The synchronized data undergoes distributed data processing via Java scripts, followed by multi-source data fusion, road line fusion, trajectory fusion, and other operations. In one implementation, during fusion, the LiDAR point cloud data can be overlaid with the semantic layer of the image information acquired by the camera in the same coordinate system. Combined with the inertial navigation data, a three-dimensional scene is reconstructed. This three-dimensional scene is the map scene tile data in this embodiment. The map scene tile data includes a map scene over a period of time. In one embodiment, the map scene may include the aforementioned important elements and target information.

[0104] S24. Perform scene generalization operation on the map scene tile data to obtain the simulation map.

[0105] In one implementation, a scene generalization algorithm is used to perform scene generalization operations on the map scene tile data to obtain a simulation map. The operating logic of the scene generalization algorithm is as follows: random scenes are generated using constraints, and the simulation map is constructed using these random scenes.

[0106] In this embodiment, various manufacturers developing the target NOA controller can implement the NOA algorithm using either a pre-configured map or one without. Alternatively, the NOA algorithm can be implemented using pure vision or a combination of sensor perception. Therefore, to accommodate the NOA algorithm requirements of different manufacturers, the simulation tools' scene generalization capability and the NOA solution's simulation generalization capability are required to be high.

[0107] In this embodiment, the generalization algorithm simulation map scene generation module is deployed in the AI ​​model inference environment of a private cloud server. The image container stores the algorithm code for AI model inference, which can be mapped to the AI ​​inference environment to implement relevant calls, providing scene and NOA map processing engine and generalization algorithm. It receives map scene tile data output by the crowdsourced map tile scene processing module and processes the map scene tile data using a large model of the scene generalization algorithm.

[0108] In one implementation, the large model of the scenario generalization algorithm is trained with a large amount of real vehicle data. The real vehicle data is obtained from the enterprise cloud server through information exchange with the enterprise cloud server to ensure that the quality and quantity of the obtained data are high.

[0109] The computational logic of the scenario generalization algorithm in the large model is as follows: under certain constraints (product definition, functional safety, and accident boundaries, etc.), random scenarios are generated. These generated random scenarios can realistically reflect various complex situations that may be encountered during driving and are highly representative. The generated scenarios include, but are not limited to, diverse urban road conditions, various urban NOA operating conditions under different weather conditions (urban NOA operating conditions refer to scenarios required for NOA functional testing; for example, if NOA has entrance and exit ramps, the scenario needs to include ramps), and various types of urban traffic signs.

[0110] Subsequently, a simulation map containing a large number of urban NOA test scenarios was constructed using randomized scenarios. This simulation map can also be called a simulated urban NOA map.

[0111] The generated simulation map is converted into OpenX (OpenScenario and OpenDrive) data, which is then output to the cloud-connected map transmission and conversion module in the cloud map scene generalization module 10. The cloud-connected map transmission and conversion module converts the data format. In addition, the simulation map can also be transmitted to the software vehicle-in-the-loop simulation module 30 for use via the network.

[0112] In one implementation, because the NOA (Noise, Arrival, and Availability) navigation functions of NOA algorithm developers are mostly based on high-precision maps, lightweight maps, crowdsourced maps, or memorized maps stored in their respective automakers' cloud servers, and the interaction between the map and the NOA algorithm module is achieved through the enterprise's cloud server, the formats or content of the maps used by different developers may differ. Therefore, the simulated map in this embodiment may differ in format or content from the maps used by the developers, and thus cannot be directly used by the NOA algorithms of the developers. Analysis revealed that the maps used by different developers... Figure 1 Generally, there are several common formats. Therefore, the simulation map in the embodiments of this application can be converted into a map supported by the R&D manufacturer through format conversion.

[0113] In this embodiment, the cloud-connected map transmission and conversion module uses format conversion technology to convert the simulation map in this embodiment into a map supported by the R&D manufacturer. The converted simulation map is then transmitted to the R&D manufacturer's vehicle cloud, where the NOA algorithm can use the map. This establishes the interaction between the simulated city NOA map and the NOA algorithm. The NOA controller uses the simulation map to achieve positioning and navigation motion planning.

[0114] In this embodiment, the simulation map is converted into a unique map format supported by the NOA (Noise, Arrival, and Positioning) algorithm of the car manufacturer, so that the NOA algorithm running in the car manufacturer's cloud can correctly use the converted map for navigation and positioning operations.

[0115] In this embodiment, an innovative simulation toolchain architecture and core simulation testing algorithm suitable for urban NOA are constructed. A large number of high-quality simulation maps derived from real-world scene slice data are generated through a scene generalization algorithm, which is suitable for urban scenarios with large volume and high complexity.

[0116] Based on any of the above embodiments, in one implementation, a positioning operation is performed using a simulation map to obtain the initial positioning information of the simulation vehicle, referring to... Figure 4 ,include:

[0117] S31. Obtain environmental simulation images of the simulated vehicle.

[0118] In this embodiment, when the simulated vehicle starts running, an environmental image of the simulated vehicle is simulated in advance. This image is called the environmental simulation image. The environmental simulation image represents the environmental image of the current location of the simulated vehicle, and the environmental image may contain information such as pedestrian crossings.

[0119] S32. Perform feature extraction on the environmental simulation image to obtain multi-scale features of the environmental simulation image.

[0120] In this embodiment, the NOA scheme generalized simulation module 20 includes a simulation map generalized navigation planning module and a simulation map generalized positioning module. The simulation map generalized positioning module is responsible for estimating the vehicle state, including attitude, linear velocity, angular velocity, etc., based on the parsed simulation map, and calculating the position of the simulation vehicle relative to the map or road. This position is the initial positioning information.

[0121] During positioning, a positioning scheme can be implemented with or without a pre-configured map. Furthermore, some manufacturers use only visual sensors for positioning, while others employ multi-sensor fusion. The difference between visual sensors and multi-sensor fusion lies in whether or not a LiDAR is included. Regardless of the scheme, the core of positioning ultimately boils down to the problem of positioning on a simulated map. NOA simulation maps, lacking some elements of real maps and with targets exhibiting diverse shapes and sizes, are more complex. Therefore, they require higher levels of scene generalization ability from the simulation tool and higher simulation generalization ability from the NOA scheme. In this application, for urban NOA simulation maps, automatic labeling (e.g., the location of pedestrian crossings or traffic lights) or editing (e.g., removing non-static objects captured by sensors) is required after processing by a generalized positioning algorithm to achieve positioning on the urban NOA map.

[0122] The localization algorithm in this embodiment comprises two parts: a feature extraction neural network and a target detection neural network. The feature extraction neural network employs a combination of a basic convolutional neural network, multi-scale convolutional modules, and a feature pyramid neural network to extract multi-scale features from the environmental simulation image. The basic convolutional neural network can be a CNN (Convolutional Neural Network, an improved convolutional neural network model). The core of a CNN is to extract and classify features of targets in the input environmental simulation image by stacking convolutional layers, pooling layers, and fully connected layers. Its core lies in using deep learning algorithms to classify and locate the extracted features.

[0123] Multi-scale convolution modules can extract more detailed and comprehensive features from target regions (such as those near the center and edges) by designing convolution kernels of different sizes. Through feature separation and fusion mechanisms, they combine the abstract semantic features extracted by deep convolutional layers with the detailed texture features retained by shallow convolutional layers, effectively enhancing feature representation capabilities.

[0124] Feature pyramid neural networks, on the other hand, construct a multi-layered feature pyramid structure to generate feature maps with hierarchical relationships at different scales. This design can comprehensively cover targets of different sizes, and through cross-layer connections and feature fusion, it can achieve accurate classification and localization of targets in a multi-dimensional space, significantly improving the robustness of the detection model to scale changes.

[0125] After the feature extraction operation of the feature extraction neural network described above, multi-scale features of the environmental simulation image can be obtained.

[0126] S33. Using multi-scale features, determine the candidate areas where the simulated vehicles are located from the simulation map.

[0127] In one implementation, the process of constructing the simulation map can be referred to the above description.

[0128] The object detection neural network uses extracted multi-scale features to determine the candidate regions where the simulated vehicle is located from the simulation map through a region proposal network.

[0129] Specifically, the region proposal network takes multi-scale features as input and generates a series of candidate regions containing simulated vehicles in the simulation map through a sliding window and anchor box mechanism. These candidate regions are initially selected based on preset anchor boxes of different scales and aspect ratios, combined with semantic information on the feature map, with the aim of covering potential target locations as comprehensively as possible.

[0130] S34. Classify and regress the candidate regions to obtain the initial positioning information of the simulated vehicle.

[0131] In this application, the candidate regions are then classified and regressed, and the target category and location information can be output to realize the localization of the simulated vehicle and obtain the initial localization information of the simulated vehicle.

[0132] In this embodiment, the target detection neural network ultimately outputs precise location coordinates (coordinates of the upper left and lower right corners of the bounding box), enabling high-precision positioning of the simulated vehicle in a complex environment and providing a reliable basis for subsequent intelligent decision-making such as path planning and behavior prediction.

[0133] In this embodiment, a simulation map generalization positioning algorithm is used to simulate the vehicle's positioning data in real time, thereby providing positioning data to the target NOA controller so that the target NOA controller can use the positioning data to perform vehicle control operations during testing.

[0134] Based on any of the above embodiments, after knowing the initial positioning information of the simulated vehicle, a route planning operation can be performed based on the simulation map, the initial positioning information, and the destination to obtain the initial navigation information of the simulated vehicle. Specifically, refer to... Figure 5 Based on the simulation map, initial location information, and destination, a route planning operation is performed to obtain the initial navigation information of the simulated vehicle, which may include:

[0135] S41. Obtain the road network from the simulation map.

[0136] In this embodiment, after the NOA function is activated, the navigation planning and positioning module in the host performs positioning and navigation operations based on the cloud map, and then sends navigation and positioning information to the NOA algorithm module through the vehicle manufacturer's cloud map module to complete the NOA function. Therefore, the core of the simulation needs to design a generalized navigation planning and positioning algorithm for the simulation map, tailored to the characteristics of the urban NOA scheme, to simulate the function of the navigation planning and positioning module in the host, and to realize the real-time updating of navigation path planning and positioning data of the simulation map. The simulation map generalized navigation planning module and simulation map generalized positioning module in this application are developed and implemented for this purpose.

[0137] The simulation map generalized navigation planning module is used to perform global and local path planning and behavior selection (such as searching for a destination and selecting a generated route) on the parsed simulation map. A corresponding visual interface has also been developed for this module.

[0138] In one implementation, the navigation path planning function is:

[0139] The calculation of the path or trajectory from the current state of the simulated vehicle to the next local target state defined by the behavior selection is essentially the simulation calculation of the route through the road network from the initial position of the simulated vehicle to the defined final destination. This planning process relies on a city NOA map. However, in real-world scenarios, simulated city NOA maps differ significantly from real maps: firstly, simulated city NOA maps lack some elements compared to real maps, failing to fully replicate real geographical details; secondly, due to the need to test extreme scenarios, simulation maps introduce more complex road condition elements and environmental variables, making simulated city NOA maps more complex than real map scenarios. Furthermore, different manufacturers add unique map elements and rules according to their own needs. Therefore, to ensure the effectiveness of the planning algorithm in diverse map environments, it is necessary to employ algorithms with strong generalization capabilities, achieving accurate adaptation to various simulated map scenarios through dynamic adjustment and adaptive learning.

[0140] For complex road networks in simulated urban NOA (Normally Indicator) maps, existing shortest path calculation algorithms are not suitable. For example, algorithms like Dijkstra's algorithm are too complex, slow, and demanding on hardware computing power. Furthermore, these algorithms are unsuitable for handling highly complex urban NOA simulation maps, especially when there are significant differences between the simulation and real maps, such as missing or unavailable attribute data. Unavailable attributes include, but are not limited to, data unavailability due to differences in data format such as guardrail attributes, road boundary representation, median strips, and curvature description methods. Missing attribute data includes, but is not limited to, issues related to changes in the number of lanes, road type changes, or road segmentation at intersections (where intersections exist, the road needs to be broken and mapped to have a unique lane ID).

[0141] To this end, this application improves the generalized navigation planning algorithm for urban NOA simulation maps, which is not only applicable to highly complex urban NOA simulation maps, but also improves upon traditional algorithms in terms of query time, preprocessing time, space utilization, and robustness to input changes.

[0142] In the generalized navigation planning algorithm for urban NOA simulation maps in this application, a weighted directed graph is constructed to analyze and model the road network, thereby achieving an effective representation of complex simulation maps and providing a better solution for path planning.

[0143] In one implementation, parsing the generation of a weighted directed graph to represent the road network may include the following steps:

[0144] a. Initialize the directed graph structure: Use a suitable graph data structure to store the topology information of the road network in the simulation map. Initialize an empty directed graph; the nodes and edges of the road network will be added to this graph later.

[0145] b. Add nodes (intersections and road endpoints): Traverse each intersection and road endpoint in the road network, assigning each node a unique identifier (such as a string ID). Record the location information of each node (such as coordinates (x, y) on a 2D plane). This location information helps to accurately display the node's position on the map during subsequent visualization. Additionally, you can add some extra attributes to the nodes, such as the node type (e.g., traffic hub node, ordinary intersection node, etc.).

[0146] c. Adding Road Segments (Directed Edges): For each road segment in the road network, determine its two endpoints (start node and end node). Assign a unique ID to each road segment. Calculate the relevant attributes of the road segment, such as road length, speed limit, number of lanes, road type (e.g., highway, urban road), and whether it is one-way. Calculate the time required to traverse the road segment based on its length and speed limit, and add this time value as the edge weight (weight in a weighted directed graph) to the directed graph. Add directed edges from the start node to the end node in the directed graph, and store the calculated road segment attributes as edge attributes in the directed graph.

[0147] d. Handling Special Cases and Complex Road Network Features: Consider special cases in the road network, such as road merging, branching, and roundabouts. For these special cases, nodes and edges need to be added and connected appropriately to accurately represent the road network topology. Handle situations where road attributes are unavailable or missing. For example, when certain road attributes (such as guardrail attributes, curvature descriptions, etc.) are unavailable, default values ​​can be used or reasonable inferences can be made based on the attributes of surrounding roads.

[0148] S42. Using vertex separators, determine the overlay map from the road network, including initial positioning information and destination.

[0149] In this embodiment, after knowing the initial location information and the destination, it is necessary to navigate a route starting from the initial location information and ending at the destination. During navigation, to improve efficiency, only the overlay map surrounding the initial location information and the destination is processed.

[0150] The process of determining the coverage map includes the following steps:

[0151] 1. Data preparation and road network loading:

[0152] Load detailed information about the road network from the simulation map data, including the location coordinates of nodes (such as intersections), the connection relationships of edges (such as road segments), and the attributes of the roads (such as road type, number of lanes, speed limit, etc.).

[0153] Define major roads clearly, for example, by selecting them based on road type (such as highways, arterial roads, etc.) and road importance (such as traffic volume).

[0154] 2. Construct the backbone subnet:

[0155] Traverse the loaded road network. For each road, use the inherent hierarchical structure (physical layer, association layer, and topology layer) of the simulated map road network to determine whether it belongs to the main road.

[0156] For edges and nodes belonging to the main road network, add them to a trunk subnet. When adding nodes and edges, preserve their attribute information from the original road network.

[0157] 3. Define vertex separators:

[0158] Define appropriate vertex separators based on the characteristics and requirements of the road network. Vertex separators can be identifiers used to distinguish different types of nodes or different hierarchical structures.

[0159] For example, vertex separators can be defined based on node location information (such as using different separators for nodes in different regions) and road type (such as using different separators for highway nodes and ordinary road nodes).

[0160] 4. Divide the network into subnet regions based on vertex separators:

[0161] Traverse the nodes in the main subnet and divide the nodes into different regions according to the vertex separator.

[0162] For example, for a trunk subnetwork containing major roads in different areas of a city, nodes can be divided into city center area nodes, suburban area nodes, etc., based on vertex separators. The nodes and edges in each area form a subgraph.

[0163] 5. Calculate the coverage map:

[0164] For each subnet region, based on the nodes within the region, use a breadth-first search or depth-first search algorithm to traverse the nodes and edges within that region.

[0165] During the search process, based on the definition of the vertex separator, only nodes and edges related to the current region are considered.

[0166] The found nodes and edges are added to a new graph structure (overlay graph) while preserving their attribute information.

[0167] For example, within a region, starting from a certain initial node, a breadth-first search algorithm is used to find all nodes and edges connected to that node and add them to the cover graph until all reachable nodes within the region have been traversed.

[0168] 6. Integrated Coverage Map:

[0169] If there are multiple subnet areas, the coverage maps of each area will be integrated.

[0170] For the connecting edges between different regions, reasonable processing is carried out during the integration process according to the rules of vertex separators to ensure the integrity and accuracy of the overlay map.

[0171] The resulting overlay map contains information about the main roads and is calculated and constructed based on vertex separators. It can better reflect the hierarchical structure and key information of the simulated map's road network and can be used for subsequent navigation, path planning and other tasks.

[0172] S43. Divide the vertex separator into smaller subsets of vertices to decompose the covering graph into multiple balanced units.

[0173] In practice, the vertex separator is divided into a small subset of vertices. After the separator is removed, the overlay graph is decomposed into multiple balanced units.

[0174] After decomposing the cover graph into multiple balancing units, the connection structure between these units can be further optimized. Specifically, with the goal of minimizing the number of cutting edges between the boundary vertices of different balancing units, the connections between balancing units are adjusted while ensuring reasonable distances between boundary vertices within each unit. This process reduces complex connections between units, lowers computational complexity, and ensures structural integrity and connectivity within each unit. This results in a decomposition of the entire cover graph that satisfies balance requirements while being more concise and efficient in its topological structure, providing a better foundational model for subsequent tasks such as path planning and network analysis.

[0175] In one implementation, step S43 includes the following steps:

[0176] 1. Filtering vertices based on delimiters:

[0177] Traverse all vertices in the cover graph and classify them according to the established vertex separator rules. Vertices with the same separator are grouped together to form an initial subset of vertices. For example, for a cover graph containing road nodes of various urban areas, nodes located in the city center (with a specific separator) are selected to form one subset of vertices; nodes located in the suburbs (with another separator) are selected to other corresponding subsets.

[0178] 2. Evaluate the balance of vertex subsets:

[0179] For each initially formed subset of vertices, relevant attributes are calculated to assess its balance. The assessment can consider factors such as the number of vertices, the number of edges, and the geographical coverage. For example, the number of vertices in each subset can be counted to determine the degree of difference in vertex count between different subsets; or the total road length covered by each subset can be calculated to assess the balance of road resources among the subsets.

[0180] 3. Adjust vertex subsets:

[0181] If imbalances are found in certain subsets of vertices, the vertices are redistributed based on the evaluation results. For example, if a subset has too many vertices while neighboring subsets have fewer, some vertices from the overweight subset can be moved to neighboring subsets (based on factors such as their connectivity with neighboring subsets) to achieve vertex balance. Simultaneously, edge connectivity is considered to ensure that the connecting edges between subsets are reasonably preserved during vertex adjustments, without disrupting the overall structure of the covering graph.

[0182] 4. Construct a balancing unit:

[0183] For each adjusted subset of vertices, it is combined with its connected edges to form a balanced unit. When constructing the balanced unit, relevant attributes of the vertices and edges are preserved, such as road length, speed limit, and number of lanes. For example, a subset of vertices and its connected edges constitute a relatively independent road network substructure, i.e., a balanced unit, where road nodes and edges have close connections and certain regional characteristics.

[0184] 5. Verify and improve the balancing unit:

[0185] The generated balanced cells are validated to check if they meet the requirements for cover graph decomposition. For example, it is confirmed that the boundaries between each balanced cell are clear and there is no duplicate assignment of vertices or edges; the connecting edges between balanced cells are checked to ensure that they accurately reflect the overall topological structure of the cover graph. If problems are found, the balanced cells are further adjusted and improved to ensure that the final balanced cells can reasonably represent the structure of the cover graph, providing a valid basis for subsequent path planning.

[0186] It should be noted that the above steps S41-S43 can be preprocessed in advance by the preprocessing module.

[0187] S44. Using multiple balancing units, perform path planning operations to obtain the initial navigation information of the simulated vehicle.

[0188] The starting point of the initial navigation information is the initial positioning information, and the ending point is the destination.

[0189] In this application, the problem of calculating initial navigation information with the starting point as the initial positioning information and the ending point as the destination is transformed into the problem of finding the shortest path in a weighted directed graph.

[0190] In one implementation, step S44 may include:

[0191] 1) Determine a navigation route that passes through at least one balance unit through path planning operations.

[0192] The starting point of the navigation route is the initial positioning information, and the ending point is the destination.

[0193] In practical scenarios, once the balancing cells in the overlay graph are known, clique edges can be calculated by combining bottom-up and parallel processing. For the overlay graph, a bidirectional search algorithm is used for searching. During the search process, if the source vertex (initial positioning information) and the target vertex (destination) are far apart, the bidirectional search algorithm only scans the vertices of the balancing cells to obtain a navigation route that passes through at least one balancing cell. This navigation route is the clique edge.

[0194] In this application, the bidirectional search algorithm only scans the vertices of the balancing unit, which can effectively reduce unnecessary computational overhead.

[0195] 2) Determine the driving route within the balance unit through which the navigation route passes, and obtain the initial navigation information of the simulated vehicle.

[0196] In this application, after knowing the traversed balance unit, it is necessary to determine the driving route inside the balance unit. At this time, a bidirectional search algorithm can be used for path planning to obtain the driving route of the simulated vehicle inside the balance unit, thereby obtaining the initial navigation information of the simulated vehicle.

[0197] It should be noted that if the shortest path from the initial location information to the destination is unique in the overlay map and contains the vertices to be compressed, then the least important vertices are removed from the map and shortcuts are created between each pair of adjacent vertices, thereby adapting the simulated city NOA map with missing map elements for navigation planning.

[0198] In addition, since urban NOA scenarios are diverse and change in real time, map scenario data also needs to simulate real vehicle-to-everything (V2X) cloud connectivity to update map scenario data and broadcast traffic conditions in real time. For example, after encountering congestion or accidents, the urban NOA map positioning information needs to be updated in real time, and navigation route planning also needs to be updated and broadcast in real time to provide real-time positioning and navigation information.

[0199] The optimized path planning algorithm proposed in this application has improved in terms of query time, preprocessing time, space utilization and robustness to input changes. Even for large road networks, it can plan the direction of travel within milliseconds.

[0200] Based on any of the above embodiments, according to the initial positioning information, a simulation operation is performed on the driving scenario of the simulated vehicle, including:

[0201] 1) Based on the initial positioning information and the map within the preset range of the initial positioning information, render the environmental information within the preset range of the simulation vehicle.

[0202] In this application, after obtaining the initial positioning information and initial navigation information, the NOA scheme generalization simulation module 20 can send the initial positioning information and initial navigation information to the cloud map scene generalization module 10. The cloud map scene generalization module 10 uses the initial positioning information to determine the map within the preset range of the initial positioning information (such as within 10 kilometers).

[0203] Then, the initial positioning information and the map within the preset range of the initial positioning information are sent to the vehicle-in-the-loop simulation module 30 of the software for scene rendering.

[0204] Specifically, the software vehicle-in-the-loop simulation module 30 accesses the cloud map scene generalization module 10 via an APN dedicated line, and downloads the city NOA simulation map within the preset range of the initial positioning information from the cloud map scene generalization module 10. The simulation map in .xodr format from OpenDrive is loaded into the scene simulation software (such as VTD) on a graphics workstation to realize the NOA simulation map scene generation and rendering. The rendered scene includes environmental information within the preset range of the simulation vehicle.

[0205] 2) Simulate the operating information of the simulated vehicle and the sensor scene information.

[0206] In this application, the software vehicle-in-the-loop simulation module 30 runs a simulated vehicle model on another Windows graphics workstation to simulate the vehicle's operational information and sensor scene information. The two graphics workstations in the software vehicle-in-the-loop simulation module 30 achieve synchronous simulation via a router and Ethernet.

[0207] The simulated vehicle's operational information can include vehicle speed (linear velocity and angular velocity), acceleration, etc. Sensor scene information can be data detected by sensors, such as pedestrians and obstacles.

[0208] 3) Construct a driving scenario for the simulated vehicle based on environmental information, operational information, and sensor scene information.

[0209] In this application, by combining environmental information, operational information, and sensor scene information, the driving scene of the simulated vehicle can be obtained. This driving scene is then sent to the NOA (Noise, Arrival, and Activation) algorithm module 40 to test the NOA controller within the module.

[0210] In one implementation, the entire testing process is as follows:

[0211] The NOA solution generalization simulation module 20 receives the simulation map output by the generalization algorithm simulation map scene generation module. After the simulation map generalization navigation planning module performs positioning operations and the simulation map generalization positioning module performs navigation operations, the positioning and navigation are updated to the simulation vehicle navigation selection interface on the local graphics workstation display. Roads and targets are visualized synchronously, and map positioning information is updated in real time. At the same time, the navigation planning and positioning information are sent to the cloud interconnection map transmission and conversion module in real time. The cloud interconnection map transmission and conversion module converts the data into navigation and positioning information that can be recognized by the vehicle manufacturer's cloud map on the enterprise cloud server through format conversion. The vehicle manufacturer's cloud map broadcasts the real-time updated navigation and positioning information to the NOA algorithm under test module 40 in real time.

[0212] Meanwhile, the software vehicle-in-the-loop simulation module 30 renders the driving scene and outputs environmental, vehicle, and sensor information to the NOA algorithm under test module 40 via Ethernet and CAN (Controller Area Network) formats. The NOA algorithm under test module 40 outputs vehicle control information to the software vehicle-in-the-loop simulation module 30 to control the vehicle to drive according to the navigation planning path.

[0213] Because the map in the car manufacturer's cloud server is synchronized with the NOA algorithm module under test in real time, it can complete NOA functions such as motion planning and control, realize the complete closed loop of urban NOA function simulation test, and enable the simulation test and verification of urban NOA algorithm.

[0214] Based on the embodiments of the above simulation testing method, another implementation of this application provides a simulation testing apparatus, referring to... Figure 6 ,include:

[0215] The map generation module 101 is used to generate a simulation map; the simulation map is obtained through a scene generalization operation.

[0216] The positioning module 102 is used to perform positioning operations using a simulation map in response to navigation commands from the destination, thereby obtaining the initial positioning information of the simulated vehicle; the simulation map is obtained through scene generalization operations.

[0217] The navigation module 103 is used to perform path planning based on the simulation map, initial positioning information and destination to obtain the initial navigation information of the simulation vehicle.

[0218] The conversion module 104 is used to convert the initial navigation information and initial positioning information into target navigation information and target positioning information that can be recognized by the target NOA controller, and send the target navigation information and target positioning information to the target NOA controller.

[0219] The test module 105 is used to simulate the driving scenario of the simulated vehicle based on the initial positioning information, and output the simulated driving scenario to the target NOA controller, so that the target NOA controller can use the target navigation information, target positioning information and driving scenario to control the simulated vehicle to the destination; the control information of the target NOA controller to control the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

[0220] Among them, the map generation module 101 is configured with the cloud map scene generalization module 10 mentioned above, the positioning module 102 can be configured with the NOA scheme generalization simulation module 20 mentioned above, the navigation module 103 can be configured with the NOA scheme generalization simulation module 20 mentioned above, the conversion module 104 can be configured with the cloud interconnected map transmission conversion module in the cloud map scene generalization module 10 mentioned above, and the testing module 105 can be configured with the software vehicle-in-the-loop simulation module 30 mentioned above and the NOA algorithm under test module 40.

[0221] In one implementation, the map generation module 101 includes:

[0222] The data acquisition submodule is used to acquire data collected by various sensors.

[0223] The data processing submodule is used to process the collected data according to the processing method corresponding to the collected data.

[0224] The data fusion submodule is used to perform data fusion operations on the data after the processing operations are performed to obtain map scene tile data. Specifically, during the data fusion operation, point cloud data is overlaid with the semantic layer in the collected image information in the same coordinate system, and combined with inertial navigation data to reconstruct the 3D scene and obtain map scene tile data.

[0225] The scene generalization submodule is used to perform scene generalization operations on the map scene tile data to obtain a simulation map.

[0226] In one implementation, the scenario generalization submodule is specifically used for:

[0227] A scene generalization algorithm is used to perform scene generalization operations on the map scene tile data to obtain a simulation map;

[0228] The operating logic of the scenario generalization algorithm is as follows:

[0229] Random scenarios are generated using constraints, and simulation maps are constructed using these random scenarios.

[0230] In one implementation, the positioning module 102 includes:

[0231] The image acquisition submodule is used to acquire environmental simulation images of the simulated vehicle.

[0232] The feature extraction submodule is used to perform feature extraction operations on the environmental simulation image to obtain multi-scale features of the environmental simulation image;

[0233] The region determination submodule is used to determine the candidate region where the simulation vehicle is located from the simulation map by utilizing multi-scale features;

[0234] The localization submodule is used to classify and regress candidate regions to obtain the initial localization information of the simulated vehicle.

[0235] In one implementation, the navigation module 103 includes:

[0236] The road network acquisition submodule is used to acquire the road network of the simulation map;

[0237] The overlay determination submodule is used to determine an overlay map from the road network, including initial positioning information and destination, using vertex separators;

[0238] The decomposition submodule is used to divide the vertex separator into smaller subsets of vertices to decompose the cover graph into multiple balanced units;

[0239] The planning submodule is used to perform path planning operations using multiple balancing units to obtain the initial navigation information of the simulated vehicle; the starting point of the initial navigation information is the initial positioning information and the ending point is the destination.

[0240] In one implementation, the planning submodule is specifically used for:

[0241] The navigation route is determined through path planning, passing through at least one balancing unit. The starting point of the navigation route is the initial positioning information and the ending point is the destination. The driving route inside the balancing unit through which the navigation route passes is determined, and the initial navigation information of the simulated vehicle is obtained.

[0242] In one implementation, the conversion module 104 is specifically used for:

[0243] Obtain the target data format used by the target NOA controller, and convert the initial navigation information and initial positioning information into target navigation information and target positioning information in the target data format.

[0244] In one implementation, the test module 105 includes:

[0245] The environment simulation submodule is used to render the environmental information within a preset range of the simulation vehicle based on the initial positioning information and the map within the preset range of the initial positioning information.

[0246] The vehicle simulation module is used to simulate the operating information and sensor scene information of the aforementioned simulated vehicle.

[0247] The scene construction submodule is used to construct the driving scene of the simulated vehicle based on environmental information, operational information, and sensor scene information.

[0248] In this application, in response to a navigation command to the destination, a positioning operation is performed using a simulation map to obtain the initial positioning information of the simulated vehicle. Based on the simulation map, the initial positioning information, and the destination, a path planning operation is performed to obtain the initial navigation information of the simulated vehicle. The initial navigation information and initial positioning information are converted into target navigation information and target positioning information that the target NOA controller can recognize. The target navigation information and target positioning information are then sent to the target NOA controller. Based on the initial positioning information, a driving scenario simulation operation is performed on the simulated vehicle, and the driving scenario is output to the target NOA controller, enabling the target NOA controller to control the simulated vehicle to reach the destination using the target navigation information, target positioning information, and driving scenario. Subsequently, the control information and driving information of the target NOA controller can be used to test the target NOA controller. In addition, the simulation map in this application is obtained through scenario generalization, which can reflect various complex situations that may be encountered during driving, and thus can be applied to complex and large-scale urban scenarios to realize the simulation testing of urban NOA.

[0249] It should be noted that the working process of each module and sub-module in this application is described in the corresponding descriptions in the above embodiments, and will not be repeated here.

[0250] This application also provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0251] Memory is used to store computer programs;

[0252] The processor is used to execute computer programs so that electronic devices can implement the simulation test methods described above.

[0253] refer to Figure 7The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0254] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0255] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0256] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the simulation testing methods provided in this application.

[0257] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the simulation testing methods provided in this application.

[0258] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A simulation testing method, characterized in that, include: Generate simulation maps; The simulation map is obtained through scene generalization operation; In response to the navigation command for the destination, a positioning operation is performed using the simulation map to obtain the initial positioning information of the simulation vehicle; Based on the simulation map, the initial positioning information, and the destination, a route planning operation is performed to obtain the initial navigation information of the simulation vehicle. The initial navigation information and the initial positioning information are converted into target navigation information and target positioning information that can be recognized by the target navigation and autopilot NOA controller, and the target navigation information and the target positioning information are sent to the target NOA controller; Based on the initial positioning information, the driving scenario of the simulated vehicle is simulated, and the simulated driving scenario is output to the target NOA controller, so that the target NOA controller uses the target navigation information, the target positioning information, and the driving scenario to control the simulated vehicle to the destination; the control information of the target NOA controller to control the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

2. The simulation testing method according to claim 1, characterized in that, The generation of the simulation map includes: Acquire data from multiple sensors; Process the collected data according to the corresponding processing method; The data after the processing is performed is fused to obtain map scene tile data. During the data fusion operation, point cloud data is overlaid with semantic layers from the collected image information in the same coordinate system. Combined with inertial navigation data, the 3D scene is reconstructed to obtain map scene tile data. A scene generalization operation is performed on the map scene tile data to obtain a simulation map.

3. The simulation testing method according to claim 2, characterized in that, The map scene tile data is subjected to scene generalization operation to obtain a simulation map, including: A scene generalization algorithm is used to perform scene generalization operations on the map scene tile data to obtain a simulation map; The operating logic of the scenario generalization algorithm is as follows: Random scenarios are generated using constraints, and simulation maps are constructed using these random scenarios.

4. The simulation testing method according to claim 1, characterized in that, Using the simulation map, a positioning operation is performed to obtain the initial positioning information of the simulated vehicle, including: Acquire environmental simulation images of the simulated vehicle; The environmental simulation image is subjected to feature extraction to obtain multi-scale features of the environmental simulation image; Using the multi-scale features, candidate regions where the simulated vehicle is located are determined from the simulation map; The candidate regions are classified and regressed to obtain the initial positioning information of the simulated vehicle.

5. The simulation testing method according to claim 1, characterized in that, Based on the simulation map, the initial positioning information, and the destination, a route planning operation is performed to obtain the initial navigation information of the simulation vehicle, including: Obtain the road network of the simulation map; Using vertex separators, a coverage map including the initial positioning information and the destination is determined from the road network; The vertex separator is divided into small subsets of vertices to decompose the cover graph into multiple balanced units; Using the multiple balancing units, a path planning operation is performed to obtain the initial navigation information of the simulated vehicle; the starting point of the initial navigation information is the initial positioning information and the ending point is the destination.

6. The simulation testing method according to claim 5, characterized in that, Using the multiple balancing units, path planning is performed to obtain the initial navigation information of the simulated vehicle, including: The navigation route is determined through path planning operations, passing through at least one balancing unit; the starting point of the navigation route is the initial positioning information and the ending point is the destination. The driving route within the balance unit through which the navigation route passes is determined, thereby obtaining the initial navigation information of the simulated vehicle.

7. The simulation testing method according to claim 1, characterized in that, Converting the initial navigation information and the initial positioning information into target navigation information and target positioning information that the target NOA controller can recognize includes: Obtain the target data format used by the target NOA controller; The initial navigation information and the initial positioning information are converted into target navigation information and target positioning information in the target data format.

8. The simulation testing method according to claim 1, characterized in that, Based on the initial positioning information, a simulation operation is performed on the driving scenario of the simulated vehicle, including: Based on the initial positioning information and the map within a preset range of the initial positioning information, render the environmental information within the preset range of the simulated vehicle; The simulation includes the vehicle's operational information and sensor scene information. The driving scenario of the simulated vehicle is constructed based on the environmental information, the operational information, and the sensor scene information.

9. A simulation testing device, characterized in that, include: The map generation module is used to generate simulation maps; The simulation map is obtained through scene generalization operation; The positioning module is used to perform positioning operations using the simulation map in response to navigation instructions for the destination, and to obtain the initial positioning information of the simulation vehicle. The navigation module is used to perform route planning based on the simulation map, the initial positioning information, and the destination to obtain the initial navigation information of the simulation vehicle. The conversion module is used to convert the initial navigation information and the initial positioning information into target navigation information and target positioning information that can be recognized by the target NOA controller, and send the target navigation information and the target positioning information to the target NOA controller; The testing module is used to simulate the driving scenario of the simulated vehicle based on the initial positioning information, and output the simulated driving scenario to the target NOA controller, so that the target NOA controller uses the target navigation information, the target positioning information and the driving scenario to control the simulated vehicle to the destination; the control information of the target NOA controller to control the operation of the simulated vehicle and the operation information of the simulated vehicle are used as reference data for testing the target NOA controller.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the simulation test method as described in any one of claims 1 to 8.

Images