An intelligent automobile simulation test scene library generation method and test system and method

Abstract

This invention provides a method for generating a simulation test scenario library for intelligent vehicles, as well as a simulation test system and method. The method for generating the simulation test scenario library includes: acquiring real-scene images; annotating the real-scene images to generate labeled scene images; and generating a simulation test scenario library using the scene images. Constructing simulation test scenarios using scene images generated from real-scene images greatly improves the realism of simulating real traffic environments, thereby ensuring the efficiency, reliability, and scenario coverage of the tests.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle technology, and more specifically to a method for generating an intelligent vehicle simulation test scenario library, as well as a simulation test system and method. Background Technology

[0002] Intelligent vehicles, also known as intelligent connected vehicles, achieve intelligent control through intelligent control modules such as Advanced Driving Assistance Systems (ADAS) or Automated Driving (AD) systems, in order to assist drivers in driving safely or partially or even completely replace drivers in controlling the vehicle.

[0003] During the development phase of intelligent vehicles, comprehensive and extensive testing is required to ensure their safety. Existing testing technologies mainly fall into two categories: real-vehicle testing on real roads using actual vehicles and virtual simulation testing based on digital modeling.

[0004] While real-vehicle testing yields highly reliable results, it is inefficient and costly, requiring significant manpower and resources, and struggles to cover diverse weather conditions, lighting, traffic situations, and road alignments. Currently, virtual simulation testing is increasingly being applied in the intelligent vehicle field. Virtual simulation testing utilizes 3D engine-based digital modeling technology to simulate various virtual traffic scenarios for testing. These virtual traffic scenarios can be presented as video animations. The video animations include built-in tags containing information such as coordinates, geometric dimensions, and textures of objects like people, vehicles, and roads within the animation. Summary of the Invention

[0005] Virtual traffic scenarios simulated using digital modeling technology differ significantly in realism from actual traffic scenarios, making it difficult to accurately represent the myriad variations of real-world traffic conditions. Therefore, the reliability of test results is low, and coverage is incomplete.

[0006] The present invention addresses the aforementioned problems. It provides a method for generating a simulation test scenario library for intelligent vehicles, as well as a simulation test system and method.

[0007] According to one aspect of the present invention, a method for generating a simulation test scenario library for intelligent vehicles is provided, comprising:

[0008] Acquire real-world images;

[0009] The real-world images are labeled to generate tagged scene images;

[0010] The simulation test scenario library is generated using the scene images.

[0011] For example, acquiring the real-scene image includes:

[0012] The real-scene images are captured using an in-vehicle terminal.

[0013] For example, acquiring the real-scene image includes:

[0014] Images acquired by the vehicle terminal are selected from those within a preset time range that include specific moments in the vehicle's state, in order to determine the real-world image, wherein the specific moments in the vehicle's state are determined based on the sensing parameters of the vehicle's sensors.

[0015] For example, acquiring the real-scene image includes:

[0016] Images collected by the vehicle terminal are selected from those within a preset time range that includes the vehicle's warning or alarm time, in order to determine the real-scene image.

[0017] For example, the vehicle-mounted terminal includes a monocular camera.

[0018] The process of acquiring the real-scene image using the vehicle-mounted terminal includes: acquiring the real-scene image through the monocular camera.

[0019] For example, the vehicle-mounted terminal includes a depth camera and / or LiDAR.

[0020] The acquisition of the real-scene images using the vehicle-mounted terminal includes: acquiring the real-scene images through the depth camera and / or lidar.

[0021] For example, the step of annotating the real-world image to generate a labeled scene image includes:

[0022] The real-scene images are classified using a classifier;

[0023] The real-world image is labeled based on the classification result to generate the scene image, wherein the scene image has a label corresponding to the classification result.

[0024] For example, the label may include one or more of the following: traffic participant label, environmental characteristic label, traffic condition label, and driver label.

[0025] Exemplarily, the method further includes:

[0026] Receive adjustment information for the labels of the scene image, and correct the labels according to the adjustment information.

[0027] For example, generating the simulation test scene library using the scene image includes:

[0028] The simulation test scene library is generated based on the vehicle forward dimension according to the labels of the scene images; and / or

[0029] The simulation test scenario library is generated based on the labels of the scene images, using the in-vehicle dimension.

[0030] For example, acquiring the real-world image includes: acquiring the real-world image based on a standard test site.

[0031] According to another aspect of the present invention, an intelligent vehicle simulation testing system is also provided, comprising a test host and a display device interconnected, wherein:

[0032] The test host is used to play scene images in the simulation test scene library generated by the above method, receive detection information output by the intelligent vehicle based on the currently played scene image, and determine the test result based on the detection information and the currently played scene image.

[0033] The display device is used to display the scene image played by the test host.

[0034] For example, the test host is also used for:

[0035] Select the corresponding scene image from the simulation test scene library according to the test cases and play it.

[0036] For example, the detection information includes one or more of the following: traffic participant information, environmental characteristics, traffic conditions, and driver information.

[0037] For example, the test host determines the test result based on the detection information and the currently playing scene image in the following way:

[0038] The detection information and the labels of the currently playing scene image are compared to determine the recognition accuracy, rejection rate, false recognition rate and / or detection speed of the intelligent vehicle.

[0039] Based on the driver information in the currently playing scene image, determine whether there are factors affecting safe driving, and based on the factors affecting safe driving, determine the accuracy, precision, and false alarm rate of the driver monitoring alarm of the intelligent vehicle; and / or

[0040] Based on the depth information of the currently playing scene image, determine the distance and relative speed information between the traffic participant and the intelligent vehicle, and based on the distance and relative speed information, determine the accuracy, precision, and false alarm rate of the intelligent vehicle's forward collision warning; and / or

[0041] The lane departure rate of the intelligent vehicle is determined based on the position information of the lane lines in the currently playing scene image, and the correctness, accuracy, and false alarm rate of the lane departure warning of the intelligent vehicle are determined based on the lane departure rate.

[0042] For example, the system also includes a memory for storing the simulation test scenario library.

[0043] According to another aspect of the present invention, a method for simulating and testing an intelligent vehicle is also provided, comprising:

[0044] Play and display scene images from the simulation test scene library generated using the above method;

[0045] Receive detection information output by the intelligent vehicle based on the currently playing scene image during testing; and

[0046] The test result is determined based on the detection information and the currently playing scene image.

[0047] The intelligent vehicle simulation test scenario library generation method and simulation test system and method according to embodiments of the present invention utilize scene images generated based on real-scene images to construct simulation test scenarios, which greatly improves the realism of the simulation of real traffic environments, thereby ensuring the efficiency, reliability and scene coverage of the test, and thus greatly promotes the development and research of intelligent vehicles.

[0048] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0049] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0050] Figure 1 A schematic flowchart of a method for generating a smart car simulation test scenario library according to an embodiment of the present invention is shown;

[0051] Figure 2 A schematic flowchart illustrating the annotation of a real-world image to generate a labeled scene image is shown according to an embodiment of the present invention.

[0052] Figure 3A schematic block diagram of an intelligent vehicle simulation test system and an intelligent vehicle under test, according to an embodiment of the present invention, is shown; and

[0053] Figure 4 A schematic flowchart of a smart car simulation testing method according to an embodiment of the present invention is shown. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0055] To conduct extensive testing of intelligent vehicles with limited resources, simulation technology is typically used to simulate real-world traffic environments. As mentioned earlier, existing virtual simulation testing generally utilizes 3D engine-based digital modeling technology to simulate various virtual traffic scenarios for testing intelligent vehicles. This virtual simulation technology has the following drawbacks:

[0056] 1. Traffic participants are model-based, such as cars, buses, trucks, special vehicles, motorcycles, bicycles, electric bicycles, pedestrians, animals, and other various models. However, the types of models are limited and cannot cover all traffic participants. For example, there may not be a suitable model for electric bicycles used for food delivery.

[0057] 2. The simulation of other objects in the traffic environment lacks realism. For example, trees have different shapes, textures, and details, but in existing virtual simulations, they are represented by a single shape. Also, because trees have different shapes, they might be recognized as vehicles by intelligent cars under different lighting conditions, but virtual simulations cannot represent such complex scenarios. Furthermore, while road alignments are finite, real-world scenarios may involve a variety of shapes, and lane markings may have varying degrees of wear and tear, stains, etc., all of which cannot be realistically represented in existing virtual simulations.

[0058] 3. The simulation of environmental factors such as weather and lighting is not realistic enough. The real world has a wide variety of weather and lighting conditions, and existing virtual simulations struggle to accurately simulate and cover all these situations.

[0059] 4. The simulation accuracy of traffic conditions is not high. Virtual simulation models traffic conditions based on traffic condition models. However, in real driving environments, different drivers have different driving habits, different driving behaviors such as cutting in, braking, and accelerating under different conditions, as well as the impact of different traffic flows, which makes the traffic condition models in virtual simulations differ from real traffic conditions.

[0060] 5. Special operating conditions cannot be realized in virtual simulation. For example, special scenarios such as an object falling from a truck and the following vehicle needing to swerve to avoid it cannot be fully simulated and covered by virtual simulation technology, and these scenarios often need to be tested.

[0061] To address the aforementioned problems, this invention proposes a method for generating a simulation test scene library based on real-world images. The following will refer to... Figure 1 A method for generating a smart car simulation test scenario library according to an embodiment of the present invention is described. Figure 1 A schematic flowchart of a method 100 for generating a smart car simulation test scenario library according to an embodiment of the present invention is shown.

[0062] like Figure 1 As shown, method 100 includes steps S110, S120 and S130.

[0063] Step S110: Obtain the real-scene image.

[0064] Real-world images can be any suitable image containing real traffic scenarios. For example, images obtained by capturing or scanning vehicles in motion using cameras, radar, or other devices. Real-world images can be static images or dynamic videos. They can be acquired using any existing or future-developed technology. For example, real-world images can be collected in real-time by using camera sensors to follow cars, by using drones to fly along lanes and capture aerial images, or by acquiring real-world images from high-resolution satellite imagery, etc. Real-world images include real traffic participants, such as cars, buses, trucks, special vehicles, motorcycles, bicycles, electric bicycles, pedestrians, and animals. Real-world images also include other objects in the traffic environment, such as roadside trees, green belts, and various roadblocks. Through a large amount of real-world images, such as tens of millions of hours of images accumulated under various weather conditions, road conditions, traffic conditions, and driving states, thorough simulation tests of real traffic scenarios can be conducted.

[0065] Real-world images can also include images of drivers during the driving process, so as to take into account driver factors when conducting simulation tests on corresponding real traffic scenarios.

[0066] Step S120: Label the real-world image obtained in step S110 to generate a labeled scene image.

[0067] Each real-scene image possesses traffic-related features, such as weather, lighting, road type, obstacle type, and the number, type, and status of traffic participants. Different features may have different impacts on vehicle movement. Therefore, the real-scene images acquired in step S110 need to be labeled to indicate which features the image possesses. Various features of the real-scene image can be represented using tags, thereby generating labeled scene images. For example, a scene image might have the following tags: "Sunny day," "Moderate lighting," "General city road," "Lane lines unclear," "7 pedestrians," "3 cars," and "Driver using a mobile phone."

[0068] Step S130: Use the scene images generated in step S120 to generate a simulation test scene library.

[0069] The scene images generated in step S120 have labels that reflect traffic-related features. These labels allow for the classification and management of scene images as needed, thereby generating a corresponding simulation test scene library. For example, using the labels "rainy day" and "highway" can generate a simulation test scene library that includes scene features of driving on a highway in rainy weather. Similarly, using the labels "snowy day" and "collision" can generate a simulation test scene library that includes scene features of collisions occurring in snowy weather.

[0070] According to the above embodiments of the present invention, the simulation test scenario is constructed by using scene images generated based on real-scene images, which greatly improves the realism of the simulation of the real traffic environment, significantly improves the test efficiency and reduces the test cost compared with real vehicle testing, and significantly improves the reliability and scene coverage of the test compared with virtual simulation testing.

[0071] For example, the above-mentioned real-scene images can be captured using an in-vehicle terminal.

[0072] The in-vehicle terminal can be, for example, an ADAS intelligent in-vehicle terminal or a dashcam, or other in-vehicle device with image acquisition capabilities. The in-vehicle terminal can acquire real-world images of the vehicle's forward direction during driving. These images exhibit a scene change rate consistent with the vehicle's speed, thus effectively simulating real-world dynamic traffic scenarios, such as simulation test scenarios for different vehicle speeds. The in-vehicle terminal may also include an in-vehicle camera to simultaneously acquire real-world images of the in-vehicle environment, including the driver's physical characteristics and state. It is understood that simulation tests without considering driver factors can directly utilize simulation test scenarios constructed based on real-world images of the vehicle's forward direction. Simulation tests requiring consideration of driver factors can utilize driver-related information from in-vehicle real-world images to test simulation test scenarios constructed based on simultaneously acquired real-world images of the vehicle's forward direction. It is understood that the real-world images mentioned below can be solely real-world images of the vehicle's forward direction, or they can be a combination of simultaneously acquired real-world images of the vehicle's forward direction and in-vehicle real-world images.

[0073] For example, images within a preset time range including specific moments of vehicle status are selected from the images acquired by the vehicle terminal to determine the real-world image, wherein the specific moments of vehicle status are determined based on the sensing parameters of the vehicle sensors.

[0074] Automotive sensors, such as accelerometers, can detect rapid acceleration, deceleration, and sharp turns. A specific moment in the vehicle's state is determined based on the sensor's sensing parameters; it is the moment when these parameters fall within a specific range. In one embodiment, a threshold value can be preset. For example, a first acceleration threshold can be set for rapid acceleration, and the moment when the vehicle's acceleration is in the same direction as the velocity and exceeds the first acceleration threshold can be considered the specific moment in the vehicle's state. Similarly, a second acceleration threshold can be set for rapid deceleration, and the moment when the vehicle's acceleration is in the opposite direction to the velocity and exceeds the second acceleration threshold can be considered the specific moment in the vehicle's state. Likewise, turning speed and turning angle thresholds can be set for sharp turns, and the moment when the vehicle's turning speed exceeds the turning speed threshold or the turning angle exceeds the turning angle threshold can be considered the specific moment in the vehicle's state. At the specific moment in the vehicle's state, a real-world image confirmation operation is triggered.

[0075] For example, a preset duration range can be set from M seconds before the specific state of the vehicle to N seconds after the specific state of the vehicle. Images within this preset duration range are selected from the images acquired by the vehicle-mounted terminal as real-scene images. Here, M and N are independently set duration parameters, which can be the same or different. For example, both M and N can be set to 10. Alternatively, M can be set to 20 and N to 10. For example, for images acquired by the vehicle-mounted terminal, only the real-scene images selected from the image cache can be saved. This enables online processing based on the specific state of the vehicle, allowing real-scene images to be acquired in real time. This allows for the construction of realistic simulation test scenarios for different driving behaviors, while effectively utilizing image data storage space. Optionally, the acquired real-scene images can be transmitted to the cloud or a remote server in real time, reducing the burden on local storage and facilitating timely subsequent processing of the real-scene images, thus improving the efficiency of constructing simulation test scenarios.

[0076] For example, images within a preset time range including the warning time or alarm time of the vehicle are selected from the images collected by the vehicle terminal to determine the real-scene image.

[0077] While driving, vehicles can issue warnings or alerts for various dangerous situations. This means issuing an early warning for potential hazards. It also means issuing an alert when a dangerous situation has already occurred or is about to occur. For example, if a collision is imminent due to excessive close proximity to the vehicle in front, a Forward Collision Warning (FCW) is issued. Similarly, if a vehicle is not staying within its lane and its deviation could cause a collision with a vehicle in an adjacent lane, a Lane Departure Warning (LDW) is issued. Finally, if the driver is in a dangerous driving state, such as using a mobile phone, smoking, yawning, or closing their eyes, a Driver State Monitoring (DSM) warning is issued.

[0078] When a vehicle issues a warning or alarm, a real-scene image confirmation operation is triggered. For example, a preset duration range can be set from K seconds before the warning or alarm to L seconds after the warning or alarm. Images within this preset duration range are selected from the images acquired via the vehicle-mounted terminal as real-scene images. K and L are independently set duration parameters; they can be the same or different. For example, both K and L can be set to 10. Alternatively, K can be set to 20 and L to 10. For example, only the real-scene images selected from the image cache can be saved from the images acquired by the vehicle-mounted terminal. This enables online processing based on the vehicle's warning or alarm time, allowing real-scene image acquisition in real time. This allows for the construction of realistic simulation test scenarios for dangerous situations, while effectively utilizing image data storage space. Optionally, the acquired real-scene images can be transmitted to the cloud or a remote server in real time, reducing the burden on local storage and facilitating timely subsequent processing of the real-scene images, thus improving the efficiency of constructing simulation test scenarios.

[0079] For example, images within a preset time range including the vehicle's warning or alarm time can be selected from the real-world images acquired based on the vehicle's specific state at a given moment to further determine the real-world image. Optionally, images within the intersection of the preset time range including the vehicle's specific state and the preset time range including the vehicle's warning or alarm time can be selected as the real-world image. Optionally, if the vehicle issues a warning or alarm within the preset time range including the vehicle's specific state, the real-world image within that preset time range can be directly determined as the further determined real-world image. Thus, online processing based on the vehicle's specific state and the vehicle's warning or alarm time can be achieved, enabling real-world image acquisition in real time. This allows for the construction of realistic simulation test scenarios for dangerous driving behaviors, while effectively utilizing image data storage space. Optionally, the acquired real-world images can be transmitted to the cloud or remote server in real time, reducing the burden on local storage and facilitating timely subsequent processing of the real-world images, thereby improving the efficiency of constructing simulation test scenarios.

[0080] For example, the vehicle-mounted terminal includes a monocular camera, and the acquisition of real-scene images using the vehicle-mounted terminal includes: acquiring real-scene images through the monocular camera.

[0081] A monocular camera consists of only one lens. Since much image processing algorithm research is based on monocular cameras, their algorithms are more mature than those of other camera types. Appropriate algorithms can be used to process real-world images captured by monocular cameras, improving the efficiency of constructing simulation test scenarios and enhancing the quality of scene images.

[0082] For example, the above-mentioned vehicle-mounted terminal includes a depth camera and / or LiDAR, and the above-mentioned acquisition of the real-scene image using the vehicle-mounted terminal includes: acquiring the real-scene image through the depth camera and / or LiDAR.

[0083] A depth camera is a camera capable of acquiring scene depth information. For example, it can be a binocular camera based on binocular vision technology, a depth camera based on structured light technology, or a depth camera based on Time-of-Flight (TOF) technology, etc. LiDAR can acquire point cloud data that includes depth information of the surrounding scene, which is also referred to as a real-scene image in this application. Real-scene images acquired by depth cameras or LiDAR include scene depth information, such as the distance between the vehicle and the vehicle in front, and the distance between the vehicle and other traffic participants or buildings. Based on distance changes in consecutive video frames, information such as the relative speed between the vehicle and the vehicle in front can also be obtained. Using the aforementioned distance and / or relative speed information, simulation test scenarios supporting intelligent vehicle alarm testing can be constructed.

[0084] Figure 2 A schematic flowchart illustrating step S120 of the present invention, which involves labeling a real-world image to generate a labeled scene image, is shown. Figure 2 As shown, step S120 above may include the following sub-steps:

[0085] Step S121: Classify the real-world image using a classifier.

[0086] By using computers to analyze images and categorize them into several classes to replace human visual interpretation, a classifier can be constructed to classify images. Any existing or future-developed technology can be used to construct the classifier, such as deep learning-based classifiers. The classifier distinguishes between different categories of targets based on the different features reflected in the real-world image, thus obtaining the classification result for that real-world image. For example, real-world images can be classified according to weather characteristics into "sunny," "rainy," "foggy," "snowy," and "cloudy." Similarly, real-world images can be classified according to road characteristics into "general urban roads," "urban expressways," "highways," and "rural roads." Thus, real-world images can be classified based on different traffic influencing factors, obtaining a corresponding classification result for each real-world image.

[0087] Step S122: Label the real-world image based on the classification result obtained in step S121 to generate a scene image, wherein the scene image has a label corresponding to the classification result.

[0088] Based on various traffic influencing factors, the classification results of the real-scene images obtained in step S121 are used across various classification dimensions. The corresponding classification results are then labeled onto the real-scene images to generate scene images. The labels of the scene images are the classification results of the corresponding real-scene images. For example, a real-scene image might have the following classification results after being classified in step S121:

[0089] Weather type: Rainy;

[0090] Lighting type: Dim lighting;

[0091] Road type: General urban road;

[0092] Obstacle types: cars + pedestrians + bicycles;

[0093] Traffic conditions: Minor conflicts exist, but are easily avoided;

[0094] Driver status: Using a mobile phone.

[0095] Label the real-world image with the above classification information to generate a scene image. The scene image has the following labels: "Weather type: rainy day", "Lighting type: dim light", "Road type: general urban road", "Obstacle type: car + pedestrian + bicycle", "Traffic condition: minor conflict, easily avoidable" and "Driver status: using mobile phone".

[0096] Thus, tagged scene images are obtained, with the label content representing the characteristics of the scene images among various traffic influencing factors, thereby enabling the construction of targeted simulation test scenarios.

[0097] In one embodiment, the labels of the above-mentioned scene images include one or more of the following: traffic participant labels, environmental feature labels, traffic condition labels, and driver labels.

[0098] Traffic participant tags can include a type tag for the traffic participant. The type of traffic participant can be, for example, a car, bus, truck, special vehicle, motorcycle, bicycle, electric bicycle, pedestrian, animal, etc. Traffic participant tags can also include a status tag for the traffic participant. The status of a traffic participant can be, for example, its position information in the scene, which can be represented by the coordinates of the four corner points of the bounding box containing the traffic participant. The status of a traffic participant can also be, for example, its relative speed information with respect to the vehicle itself. For example, a traffic participant in a scene image might be labeled: "Type: Car", "Position: Bounding Box{A,B,C,D}", "Relative Speed: 20 km / h". Here, A, B, C, and D represent the coordinates of the four corner points of the bounding box of the traffic participant.

[0099] Environmental features can include characteristics such as weather, lighting, roads, lane markings, and obstacles. For example, the environmental feature labels for a scene image might be: "Weather: Sunny", "Lighting: Strong Light", "Road Type: Highway", "Lane Markings: Emergency Lane", and "Obstacle Type: Warning Triangle".

[0100] Traffic conditions, also known as traffic flow, describe the overall dynamic state of a traffic scenario. Intelligent vehicles can take corresponding control measures based on dynamic traffic conditions. Traffic condition labels can be, for example, "Traffic Condition: Minor conflict, easily avoidable," "Traffic Condition: Significant conflict, requires certain measures to avoid," "Traffic Condition: Conflict is obvious, requires timely measures to avoid," and "Traffic Condition: Collision." More traffic condition classifications can be found in the 37 hazardous conditions classifications published by the National Highway Traffic Safety Administration (NHTSA) in the United States.

[0101] Driver tags can include physical characteristics such as age, gender, height, presence of headwear, and facial lighting. Significant differences exist between male and female driving habits, as well as between younger drivers under 50 and older drivers over 50. A driver's height affects their field of vision, thus influencing driving judgment. Wearing hats, glasses, or sunglasses also impacts vision. Different lighting conditions on a driver's face, such as complete exposure, shadows, glare, or dim lighting, affect attention and eye focus, thus impacting effective vision and even driving maneuvers. Driver tags can also include status indicators. These include states such as closed eyes, smoking, using a mobile phone, and yawning; these states of inattention seriously affect safe driving. In-vehicle cameras, such as driver behavior recognition cameras, can capture real-time images of the driver inside the vehicle. For example, the driver's physical characteristics in a scene image are labeled as: "Gender: Female", "Age: Under 50 years old", "Accessory type: Wearing glasses", and "Facial lighting: Partially shadowed"; the driver's status label in the scene image is: "Yawning".

[0102] By using the various types of labels mentioned above, we can reflect the various traffic factors in the scene image, thereby enabling us to construct corresponding simulation test scenarios for various traffic conditions.

[0103] In one embodiment, adjustment information for the label of a scene image can be received, and the label of the scene image can be corrected according to the adjustment information.

[0104] Classifying real-world images using the aforementioned classifier may yield inaccurate or imprecise results, leading to mismatches between scene image labels and actual scene content. To address this, scene testing techniques or manual verification can be used to provide adjustment information for scene image labels. The labels are then modified accordingly based on this information. This ensures that the scene image labels more accurately describe the actual scene content, enabling the construction of accurate simulation test scenarios.

[0105] By using the labels of the various scene images mentioned above, simulation test scenarios can be categorized from various dimensions, generating simulation test scenario libraries for each dimension. For example, they can be categorized according to dimensions such as weather type, road type, and traffic conditions.

[0106] For example, a simulation test scenario library can be generated based on the vehicle forward dimension according to the labels of the scene images. Table 1 shows some of the labels based on the vehicle forward dimension, where only the type of traffic participant is shown, not the state of the traffic participant.

[0107] Table 1 Vehicle Forward Dimension Labels

[0108]

[0109] For example, a simulation test scenario library can be generated based on the labels of scene images according to the in-vehicle dimension. Table 2 shows some of the labels based on the in-vehicle dimension.

[0110] Table 2 Vehicle Interior Dimension Labels

[0111]

[0112] The vehicle-forward dimension includes traffic-related features outside the vehicle in the scene image, representing objective factors influencing traffic. The in-vehicle dimension includes driver-related features, representing subjective factors influencing traffic. A simulation test scenario library can be constructed based solely on the vehicle-forward dimension to test objective factors affecting intelligent vehicles, or solely on the in-vehicle dimension to test subjective factors, or a combination of both can be used to construct a comprehensive simulation test scenario library for testing intelligent vehicles. This results in a simulation test scenario library capable of comprehensively testing intelligent vehicles, improving test coverage.

[0113] For example, real-world images can be obtained based on a standard test site.

[0114] A standard test track is a venue for real-world testing of intelligent vehicles, such as a closed-course autonomous driving test base certified by the transportation department. Standard test tracks, by arranging markings, signs, markers, movable traffic lights, target vehicles, and target pedestrians, can construct various road traffic scenarios and forward accident warning scenarios of different forms and scales to meet the needs of autonomous driving testing. To ensure that simulation testing achieves the same testing effect as real-world testing on actual roads, this invention employs a calibration method combined with track testing. Traffic conditions and test scenarios that can be relatively well reproduced are selected within the standard test track. Onboard terminals, such as ADAS devices, dashcams, and in-vehicle cameras, record real-world images during the test process. Differential GPS and other technologies are used to mark the absolute and relative positions of the vehicle, vehicles ahead, pedestrians, bicycles, and lane lines. After the test, scene images are generated based on the recorded real-world images, and the labels of the scene images can be corrected using the aforementioned annotation information. Thus, by obtaining scene images based on the standard test track, simulation test scenarios can be constructed using the standard test track, thereby achieving test results close to those of real-world testing through simulation testing. For example, in one embodiment, the consistency rate between the test results output by the simulation test and the test results of the actual vehicle test reaches over 99%.

[0115] According to another aspect of the present invention, an intelligent vehicle simulation testing system is provided. Figure 3 A schematic block diagram of an intelligent vehicle simulation testing system 300 according to an embodiment of the present invention is shown. Figure 3 As shown, system 300 includes a test host 310 and a display device 320 that are interconnected.

[0116] The test host 310 is used to play scene images from the simulation test scene library generated using the above method, receive detection information output by the intelligent vehicle based on the currently played scene image, and determine the test result based on the detection information and the currently played scene image. The display device 320 is used to display the scene images played by the test host 310.

[0117] A smart car can be understood as a regular car equipped with a smart control module. The smart control module enables intelligent control to assist the driver in safe driving or partially or even completely replace the driver in controlling the vehicle. Examples of smart control modules include ADAS (Advanced Driver Assistance Systems) devices or AD (Autonomous Driving) devices. In simulation testing, testing the smart control module can replace testing the smart car itself. In the following text, "smart car 330" can refer to either the entire vehicle system or the smart control module.

[0118] The test host 310 reads scene images from the simulation test scene library generated using the above method and plays them. The display device 320 displays the scene images played by the test host 310. The display device 320 can be, for example, a projector or a large-size display, ensuring that the size of the vehicles in the displayed scene images matches the size of real vehicles, thereby improving the realism of the simulation test. The vision system of the intelligent vehicle 330 aligns with the display device 320, performs tests based on the currently played scene images, and outputs detection information to the test host 310. The detection information is based on various output information from the intelligent vehicle 330, such as the identification of relevant traffic factors in the scene images. For example, the identification of traffic participants, weather, road types, etc., in the scene images. For example, the identification of behaviors affecting traffic safety, such as whether the driver is smoking or using a mobile phone. The detection information may also include the intelligent vehicle's reaction based on the currently played scene images, such as issuing warnings or alarms, changing lanes to avoid obstacles, or emergency braking. The test host 310 determines the test results based on the detection information received from the intelligent vehicle 330 and the currently played scene images. For example, if the label of a traffic participant in a scene image is "car," and the detection information identifies the traffic participant as "car," then the test result for that traffic participant is determined to be correct. For instance, if the collision time (TTC) obtained by the intelligent vehicle 330 based on the position and speed information of the vehicle in front identified in the scene image is less than a preset collision time threshold, it determines that there is a risk of collision between the vehicle and the vehicle in front and issues a forward collision warning. If the error between the TTC obtained by the test host 310 based on the position and speed information of the vehicle in front in the scene image and the TTC output by the intelligent vehicle 330 is within the threshold range, then the forward collision warning is determined to be a correct test result; otherwise, it is an incorrect warning.

[0119] According to the simulation testing system of the present invention, the simulation test scenario is constructed by using scene images generated based on real scene images for simulation testing, which greatly improves the realism of the simulation test, significantly improves the testing efficiency and reduces the testing cost compared with real vehicle testing, and significantly improves the reliability and scene coverage of the test compared with virtual simulation testing.

[0120] For example, system 300 also includes a memory (not shown) connected to test host 310 for storing a simulation test scenario library generated using the method described above. Test host 310 can read scenario images from the memory for simulation testing. The memory storing the simulation test scenario library is independent of test host 310 and can be connected to multiple test hosts simultaneously, thereby supporting simultaneous simulation testing of multiple intelligent vehicles and improving testing efficiency.

[0121] Optionally, the simulation test scenario library generated using the above method can be stored in the cloud or on a remote data server. The test host 310 can read scenario images from the cloud or remote data server for simulation testing. Multiple test hosts 310 can simultaneously read scenario images from the cloud or remote data server for simulation testing, thereby supporting multiple intelligent vehicles to conduct simulation testing simultaneously and improving testing efficiency.

[0122] Optionally, the simulation test scene library generated using the above method can be stored in the built-in memory of the test host 310. The test host 310 reads scene images from its built-in memory for simulation testing. The built-in memory can provide high data bandwidth, thereby supporting high-resolution scene image transmission, ensuring that simulation testing can be performed based on high-quality scene images, thus improving the effectiveness of simulation testing.

[0123] For example, the test host 310 is also used to select and play corresponding scene images from the simulation test scene library generated using the above method according to the test cases.

[0124] To comprehensively test intelligent vehicles, test cases covering various functional and / or performance characteristics need to be constructed. The test host 310 selects and plays corresponding scene images from a simulation test scene library generated using the above method, based on each constructed test case. For example, a test case might be: driving on a highway curve in rainy weather. The test host 310 selects and plays scene images including "rainy weather," "highway," and "curve" based on the test case. This automates intelligent vehicle simulation testing and improves its efficiency.

[0125] For example, the detection information output by the intelligent vehicle 330 includes one or more of the following: traffic participant information, environmental features, traffic conditions, and driver information. Traffic participant information may include the type of traffic participant. The type of traffic participant may be, for example, a car, bus, truck, special vehicle, motorcycle, bicycle, electric bicycle, pedestrian, animal, etc. Traffic participant information may also include the state of the traffic participant. The state of the traffic participant may be, for example, the location information of the traffic participant in the scene, which can be represented using the coordinates of the four corner points of the bounding box containing the traffic participant. The state of the traffic participant may be, for example, the relative speed information between the traffic participant and the vehicle itself. Environmental features may include features such as weather, lighting, road conditions, lane markings, and obstacles. Traffic conditions, also known as traffic flow, are a description of the overall dynamic situation of the traffic scene. The intelligent vehicle 330 can take corresponding control measures based on the dynamic traffic conditions. Driver information may include the driver's physical characteristics, such as age, gender, height, whether there are head ornaments, facial lighting conditions, etc. Driver information can also include the driver's state information, such as: eyes closed, smoking, using a mobile phone, yawning, etc. These states of inattention have a serious impact on safe driving. The intelligent vehicle 330 performs intelligent control based on various detection information obtained from scene images; therefore, the correct operation of each detection function reflects whether the intelligent vehicle can achieve correct intelligent control. Thus, various detection information can be used to determine the test results of the intelligent vehicle, making the test results clear and unambiguous.

[0126] For example, the test host 310 can determine the test result based on the detection information output by the intelligent vehicle 330 and the currently playing scene image through the following four steps.

[0127] Step 1: The test host 310 compares the detection information output by the intelligent vehicle 330 with the labels of the currently playing scene image to determine one or more of the following indicators of the intelligent vehicle 330: recognition accuracy, rejection rate, false recognition rate and / or detection speed.

[0128] For each test case, the test host 310 compares the detection information output by the intelligent vehicle 330 with the labels of the currently playing scene image to obtain the test result for that test case. Based on the executed test cases, all obtained test results can be statistically analyzed to determine the intelligent vehicle 330's recognition accuracy, rejection rate, false recognition rate, and / or detection speed. For example, the weather type recognition accuracy is 99%, the rejection rate is 0%, the false recognition rate is 1%, and the detection speed is 30 frames per second (FPS). For example, the traffic participant recognition accuracy is 95%, the rejection rate is 2%, the false recognition rate is 3%, the detection speed is 50 FPS, and the bounding box coordinate accuracy of the traffic participant is 95%. Based on the time delay from the playback of scene images to the output of corresponding detection information by the intelligent vehicle 330, the test host 310 can also output the overall latency result of the intelligent vehicle 330. For example, for a scene image played at 50 FPS, the overall latency of the intelligent vehicle 330 is 10 frames, meaning that the intelligent vehicle 330 outputs detection information after a delay of 10 frames from the playback of the scene image. Thus, the quantitative detection accuracy and detection speed of the intelligent vehicle's perception layer can be obtained.

[0129] Step two, the test host 310 determines whether there are factors affecting safe driving based on the driver information in the currently playing scene image, and determines the correctness, accuracy, and false alarm rate of the driver monitoring alarm of the intelligent vehicle based on the factors affecting safe driving.

[0130] The factors affecting safe driving mentioned above include the driver's condition and physical characteristics. Driver condition monitoring alarms include alarms for fatigued driving, smoking (dangerous driving), and using a mobile phone while driving. Driver physical characteristic monitoring alarms include alarms indicating the need to adjust the seat height and alarms indicating excessive facial glare.

[0131] For each test case, the intelligent vehicle 330 can detect driver information, such as whether the driver is in a state of inattention (eyes closed, smoking, using a mobile phone, yawning, etc.). These states of inattention have a serious impact on safe driving and fall under the aforementioned factors affecting safe driving. If the intelligent vehicle 330 detects that the driver has been continuously closing their eyes for more than a first fatigue driving time threshold (e.g., 10 seconds) or if the driver has yawned a preset number of times (e.g., 3 times) within a second fatigue driving time threshold (e.g., 2 minutes), the intelligent vehicle 330 will issue a fatigue driving alarm. The aforementioned first fatigue driving time threshold, second fatigue driving time threshold, and preset values ​​can use system default values ​​or be preset by the user. If the intelligent vehicle 330 detects the driver smoking, it will issue a smoking hazard driving alarm. If the intelligent vehicle 330 detects the driver using a mobile phone, it will issue a mobile phone hazard driving alarm. The test host 310 determines whether there are factors affecting safe driving based on the driver's state in the currently playing scene image. If there are factors affecting safe driving and they are consistent with the content of the driver monitoring alarm issued by the intelligent vehicle 330, then the current driver monitoring alarm is determined to be correct; otherwise, it is an incorrect alarm.

[0132] For each test case, the intelligent vehicle 330 can detect the driver's physical characteristics, such as height and facial lighting. If it detects that the driver is too short and has not adjusted the seat to a suitable height, resulting in insufficient visibility for safe driving, the intelligent vehicle 330 will issue an alarm indicating that the seat height needs to be adjusted. If it detects that the driver's facial lighting is too strong, affecting the driver's attention and eye focus, the intelligent vehicle 330 will issue an alarm indicating excessive facial lighting, and optionally, prompt the driver to use the sun visor. The test host 310 determines whether there are factors affecting safe driving based on the driver's physical characteristics in the currently playing scene image. If there are factors affecting safe driving and they are consistent with the content of the driver monitoring alarm issued by the intelligent vehicle 330, then the current driver monitoring alarm is determined to be correct; otherwise, it is an incorrect alarm.

[0133] For example, if a smart car 330 is supposed to issue 100 fatigue driving warnings, it actually issues 93 forward collision warnings. Of these, 87 warnings were issued when fatigue driving should have been issued, and 6 warnings were issued when it shouldn't have been issued. Therefore, the accuracy rate of the smart car 330's fatigue driving warning system is 87% (87 ÷ 100), the accuracy rate is 93.55% (87 ÷ 93), and the false alarm rate is 6.45% (6 ÷ 93). This provides a quantitative test result for the fatigue driving warning logic of the smart car. Similarly, test results can be obtained for driver monitoring warnings such as smoking warnings, mobile phone use warnings, warnings requiring seat height adjustment, and excessive facial illuminance warnings.

[0134] Step 3: The test host 310 determines the distance and relative speed information between the traffic participants and the intelligent vehicle 330 based on the depth information of the currently playing scene image, and determines the accuracy, precision and false alarm rate of the intelligent vehicle 330's forward collision warning based on the distance and relative speed information.

[0135] For each test case, the intelligent vehicle 330 determines in real time whether there is a vehicle ahead and whether the vehicle ahead is within its lane based on the currently playing scene image. It obtains the detailed coordinates of the bounding box of the vehicle ahead within its lane and calculates the Time-to-Cross (TTC) based on the vehicle's current speed. When the TTC is less than a preset collision time threshold, it determines that there is a risk of collision between the vehicle and the vehicle ahead and issues a forward collision warning. If the error between the TTC obtained by the test host 310 based on the position and speed information of the vehicle ahead in the currently playing scene image and the TTC output by the intelligent vehicle 330 is within the threshold range, the forward collision warning is determined to be a correct test result; otherwise, it is an incorrect warning. Based on the executed test cases, all obtained test results can be statistically analyzed to determine the correctness, accuracy, and false alarm rate of the intelligent vehicle 330's forward collision warnings. For example, if the intelligent vehicle 330 should issue 100 forward collision warnings but actually issues 97, 89 warnings were issued when they should have been issued and 8 warnings were issued when they should not have been issued. Therefore, the accuracy rate of the forward collision warning of this intelligent vehicle 330 is 89%.

[0136] The accuracy rate is 91.75% (89 ÷ 97), and the false alarm rate is 8.25% (8 ÷ 97). Therefore, quantitative test results of the forward collision warning logic of the intelligent vehicle's decision-making layer can be obtained. Similarly, forward collision warning systems can be tested to obtain quantitative test results of the forward collision warning logic of the intelligent vehicle's decision-making layer.

[0137] Step four: The test host 310 determines the lane deviation degree of the intelligent vehicle 330 based on the lane line position information of the currently playing scene image, and determines the correctness, accuracy and false alarm rate of the lane deviation warning of the intelligent vehicle 330 based on the lane deviation degree.

[0138] For each test case, the intelligent vehicle 330 identifies lane lines in real time based on the currently playing scene image and obtains the lane line position information. Based on the lane line position information, it determines the degree of lane deviation and whether the lane deviation exceeds a preset deviation threshold. If it does, a lane departure warning is issued. If the test host 310 determines, based on the lane line position information in the currently playing scene image and the intelligent vehicle 330's position information, that the intelligent vehicle 330's lane deviation exceeds the preset threshold, then the lane departure warning is considered a correct test result; otherwise, it is considered an incorrect warning. Based on the executed test cases, all obtained test results can be statistically analyzed to determine the correctness, accuracy, and false alarm rate of the intelligent vehicle 330's lane departure warnings. For example, if the intelligent vehicle 330 should issue 100 lane departure warnings but actually issues 95 forward collision warnings, then 90 lane departure warnings were issued when they should have been issued, and 5 lane departure warnings were issued when they shouldn't have been issued. Therefore, the lane departure warning accuracy rate of the intelligent vehicle 330 is 90% (90 ÷ 100), the precision rate is 94.74% (90 ÷ 95), and the false alarm rate is 5.26% (5 ÷ 95). This provides a quantitative test result for the lane departure warning logic of the intelligent vehicle's decision-making layer. Similarly, lane departure alarms can be tested to obtain quantitative test results for the lane departure alarm logic of the tested intelligent vehicle's decision-making layer.

[0139] It is understood that steps one, two, three, and four above are not related in terms of execution order. The test host 310 can arbitrarily select one, two, three, or four steps from steps one, two, three, or four to determine the test result.

[0140] According to another aspect of the present invention, a method for simulating and testing intelligent vehicles is also provided. Figure 4 A schematic flowchart of an intelligent vehicle simulation testing method 400 according to an embodiment of the present invention is shown. Figure 4 As shown, method 400 includes steps S410, S420 and S430.

[0141] Step S410: Play and display the scene images in the simulation test scene library generated using the above method.

[0142] Step S420: Receive detection information output by the intelligent vehicle based on the currently playing scene image.

[0143] Step S430: Determine the test result based on the detection information and the label of the currently playing scene image.

[0144] Based on the above description of the intelligent vehicle simulation test system, those skilled in the art can understand the detailed execution steps of method 400. For the sake of brevity, further details will not be repeated here.

[0145] The intelligent vehicle simulation testing method of the present invention utilizes scene images generated based on real-world images to construct simulation test scenarios for testing, which greatly improves the realism of the simulation of real traffic environments, thereby ensuring the efficiency, reliability, and scene coverage of the test, and thus can greatly promote the development and research of intelligent vehicles.

[0146] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0147] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0149] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0150] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0151] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0152] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0153] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the image recognition apparatus according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0154] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0155] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating a simulation test scenario library for intelligent vehicles, comprising: Acquiring real-scene images, the method for acquiring real-scene images includes using an in-vehicle terminal to collect the real-scene images, the in-vehicle terminal including an in-vehicle camera, the in-vehicle camera being used to acquire the real-scene images including the driver's physical characteristics and state, the real-scene images collected by the in-vehicle terminal including images of the car within a preset time range at a specific state time and images of the car within a preset time range at a warning time and / or an alarm time, wherein the specific state time is determined according to the sensing parameters of the car's sensors, and the warning time and / or alarm time is determined according to the driver's driving behavior or driving state; The real-scene images are classified using a classifier; The real-world image is labeled based on the classification results to generate the scene image, wherein the scene image has a label corresponding to the classification results. The label includes a traffic participant label, an environmental feature label, a traffic condition label, and a driver label. The traffic participant label includes a type label of the traffic participant. The environmental features include features of weather, lighting, road, lane lines, and / or obstacles. The driver label includes a driver's physical characteristics label, which includes age, gender, height, whether there are accessories on the head, and / or facial lighting conditions. The simulation test scenario library is generated using the scene images.

2. The method according to claim 1, wherein, The vehicle-mounted terminal includes a monocular camera. The process of acquiring the real-scene image using the vehicle-mounted terminal includes: acquiring the real-scene image through the monocular camera.

3. The method according to claim 1, wherein, The vehicle-mounted terminal includes a depth camera and / or LiDAR. The acquisition of the real-scene images using the vehicle-mounted terminal includes: acquiring the real-scene images through the depth camera and / or lidar.

4. The method according to claim 1, wherein, The method further includes: Receive adjustment information for the labels of the scene image, and correct the labels according to the adjustment information.

5. The method according to claim 1, wherein generating the simulation test scene library using the scene image comprises: The simulation test scenario library is generated based on the vehicle forward dimension according to the labels of the scene images; And / or generate the simulation test scene library based on the in-vehicle dimension according to the labels of the scene images.

6. The method according to claim 1, wherein, The acquisition of real-scene images includes: acquiring the real-scene images based on a standard test site.

7. An intelligent vehicle simulation testing system, comprising a test host and a display device interconnected, wherein: The test host is used to play scene images in the simulation test scene library generated by the method according to any one of claims 1 to 6, receive detection information output by the intelligent vehicle based on the currently played scene image, and determine the test result according to the detection information and the currently played scene image. The display device is used to display the scene image played by the test host.

8. The system according to claim 7, wherein, The test host is also used for: Select the corresponding scene image from the simulation test scene library according to the test cases and play it.

9. The system according to any one of claims 7 or 8, wherein, The detection information includes one or more of the following: information on traffic participants, environmental characteristics, traffic conditions, and driver information.

10. The system according to any one of claims 7 or 8, wherein, The test host determines the test result based on the detection information and the currently playing scene image in the following way: The detection information and the labels of the currently playing scene image are compared to determine the intelligent vehicle's recognition accuracy, rejection rate, false recognition rate, and / or detection speed; and / or Based on the driver information in the currently playing scene image, determine whether there are factors affecting safe driving, and based on the factors affecting safe driving, determine the accuracy, precision, and false alarm rate of the driver monitoring alarm of the intelligent vehicle; and / or Based on the depth information of the currently playing scene image, determine the distance and relative speed information between the traffic participant and the intelligent vehicle, and based on the distance and relative speed information, determine the accuracy, precision, and false alarm rate of the intelligent vehicle's forward collision warning; and / or The lane departure rate of the intelligent vehicle is determined based on the position information of the lane lines in the currently playing scene image, and the correctness, accuracy, and false alarm rate of the lane departure warning of the intelligent vehicle are determined based on the lane departure rate.

11. The system according to any one of claims 7 or 8, wherein, The system also includes a memory for storing the simulation test scenario library.

12. A method for simulating and testing intelligent vehicles, comprising: Play and display scene images from a simulation test scene library generated using the method described in any one of claims 1 to 6; Receive detection information output by the intelligent vehicle based on the currently playing scene image; as well as The test result is determined based on the detection information and the currently playing scene image.

Images