An intelligent driving multi-scene visual simulation test system and test method

Abstract

The application belongs to the technical field of automobiles, and particularly relates to an intelligent driving multi-scene visual simulation test system and a test method. The system comprises a scene information acquisition module, a data transmission module, a scene segmentation module, a multi-scene simulation module and a result analysis module. The scene information acquisition module is connected with the scene segmentation module through the data transmission module. The scene segmentation module is connected with the multi-scene simulation module through the data transmission module. The multi-scene simulation module is connected with the result analysis module through the data transmission module. The application can simulate real multi-scene information by collecting a small amount of actual road scene data, and the system saves a large amount of work in collecting a multi-scene data set in intelligent driving and increases the authenticity of the simulated road scene.

Description

Technical Field

[0001] This invention belongs to the field of automotive technology, specifically a multi-scenario visual simulation testing system and method for intelligent driving. Background Technology

[0002] In the field of intelligent driving algorithm evaluation, intelligent driving traffic scenarios involve a wide variety of targets, a large amount of background information, and diverse changes in lighting and weather. This results in driving simulation systems facing challenges such as a huge workload in data collection, long processing times, and low scene coverage. Furthermore, they can only conduct individual experiments based on existing data collection scenarios. Therefore, how to more conveniently build intelligent driving scenario simulation systems to evaluate related algorithms, reduce workload, and improve evaluation efficiency and scene coverage has become an urgent problem to be solved. Summary of the Invention

[0003] This invention provides a multi-scenario visual simulation testing system and method for intelligent driving. It only requires the collection of a small amount of actual road scene data to simulate real multi-scenario information, including different time periods each day, weather, vehicle and pedestrian congestion, etc. This invention saves the huge workload of collecting diverse scenario datasets in intelligent driving, while increasing the realism of simulated road scenarios. It solves the problems of huge workload, long time consumption, and low scene coverage of existing driving simulation systems.

[0004] The technical solution of this invention is described below in conjunction with the accompanying drawings:

[0005] In a first aspect, embodiments of the present invention provide an intelligent driving multi-scenario visual simulation testing system, comprising:

[0006] The scene information acquisition module is used to collect information about actual road scenes;

[0007] The data transmission module is used for information transfer between different modules;

[0008] The scene segmentation module is used to segment and statistically analyze different target categories and probability distributions in actual road scenes, extract visual pixel information of different target categories in different scenes, and transmit it to the multi-scene simulation module.

[0009] The multi-scenario simulation module is used to randomly combine segmented scenarios to simulate different road traffic scenarios;

[0010] The results analysis module is used to evaluate algorithms related to intelligent driving.

[0011] The scene information acquisition module is connected to the scene segmentation module through the data transmission module; the scene segmentation module is connected to the multi-scene simulation module through the data transmission module; and the multi-scene simulation module is connected to the result analysis module through the data transmission module.

[0012] Furthermore, the scene information acquisition module is a vehicle-mounted camera; the vehicle-mounted camera is used to acquire scenes; the vehicle-mounted camera is set at a position that can capture the road scene in front of the vehicle with the largest possible range; the scenes acquired by the vehicle-mounted camera include different weather conditions, different time periods, and different structured or unstructured roads; the acquired scenes cover multiple scenes; the road scenes include target categories in all roads.

[0013] Furthermore, the data transmission module is a wireless network communication device.

[0014] Secondly, embodiments of the present invention provide a method for multi-scenario visual simulation testing of intelligent driving, implemented through a multi-scenario visual simulation testing system for intelligent driving, comprising the following steps:

[0015] Step 1: Collect scene information through the scene information acquisition module, and then transmit the collected scene information to the scene segmentation module through the data transmission module;

[0016] Step 2: The scene information is segmented and statistically analyzed by the scene segmentation module to extract visual pixel information of different target categories in different scenes, and then transmitted to the multi-scene simulation module through the data transmission module.

[0017] Step 3: The visual pixel information obtained by the scene segmentation module is randomly combined and embedded through the multi-scene simulation module to simulate different road traffic scenarios, and the different road traffic scenarios are passed to the result analysis module.

[0018] Step 4: Evaluate the performance of intelligent driving-related algorithms under different road traffic scenarios through the results analysis module.

[0019] Furthermore, the specific method for step two is as follows:

[0020] 21) Using semantic segmentation methods, vehicles, pedestrians, and roads are used as the primary target categories, while other categories are used as background categories;

[0021] 22) Based on vehicles, pedestrians and roads as the main categories, the frequency of each category in all scenarios is statistically analyzed to obtain its probability distribution f, which represents its frequency of occurrence in road scenarios.

[0022] 23) Select a road scene information of a certain period of time on a day with sufficient light, and use the static vehicle to take a certain section of road scene information as the reference scene information γ(t0). Use the same road scene information of other time periods and other weather conditions as the identifier extraction scene. Calculate the difference of image pixel value of a single video frame between the identifier extraction scene and the reference scene in different time periods t, and between the identifier extraction scene in different weather conditions and the reference scene. Obtain the pixel value difference represented by traffic environment such as different time periods and weather conditions as γ(t)-γ(t0) and k(t)-γ(t0), respectively. Arrange the information according to the time sequence of video frames to represent the pixel sequence of different time periods and weather changes.

[0023] Furthermore, the specific method for step three is as follows:

[0024] The multi-scene simulation module randomly combines and embeds the time period, weather, and road target categories obtained by the scene segmentation module to simulate different road traffic scenarios. The multi-scene segmentation module uses computer-generated, randomly changing curves as road boundary lines, which also serve as demarcation lines between the module and other categories of information. All embedded target categories and background information will not exceed the road boundary lines and interfere with the road information. Specifically:

[0025] 31) Based on the perspective of the road scene captured by the vehicle-mounted camera, establish road boundary lines and their extension directions in a blank image background that conform to the width and angle of a real road scene. The lower left corner of the image is taken as the origin o of the coordinate system, the upward direction is the y-axis direction, and the rightward direction is the x-axis direction. (x, y) represents a point on the road boundary line. To simulate the changes in the real road boundary line, the specific changes in the road boundary lines on both sides are represented by two function curves:

[0026] Left boundary line: y = ψ(x) * ω(t)

[0027] Right boundary line: y = ψ(x + a(x))ω(t)

[0028] Where ψ(x) represents the curvature change of the lane line; a(x) represents the change in road width; t represents the time series; ω(t) represents the rate of change of the function over time; and the set scenario is used as the baseline scenario.

[0029] 32) Based on the defined lane lines, the background target categories segmented by the scene segmentation module are randomly embedded into both sides of the road boundary lines according to probability f and time t. Vehicles and pedestrians, as the main categories, are randomly embedded into the road scene according to probability f and time t. Then, the simulated change rate v0 of the background in the video sequence changes with the simulated detected vehicle speed v1 as follows:

[0030] v0=δ(f,t)v1

[0031] The changes in simulated road vehicle speed v2 and simulated pedestrian speed v3 with respect to simulated detected vehicle speed v1, time t, and the horizontal axis x of the image are expressed as follows:

[0032] v2=λ(v1,t,x0,y), v3=φ(v1,t,x), [x <x0<x+a]

[0033] All vehicles and pedestrians are given the attribute of slowing down or avoiding obstacles when they encounter them, where λ and φ are curves related to the speed of vehicles and pedestrians, and the threshold is the same as the actual speed of vehicles and pedestrians.

[0034] 33) To simulate weather changes, time pixel difference γ(t)-γ(t0) and weather pixel difference k(t)-γ(t0) are embedded in the baseline scene in 31) to represent the weather conditions at different time periods.

[0035] Furthermore, the results analysis module evaluates the effectiveness, robustness, and generalization ability of intelligent driving-related algorithms.

[0036] The beneficial effects of this invention are as follows:

[0037] This invention requires only a small amount of actual road scene data collection to simulate real multi-scene information, including different time periods each day, weather, vehicle and pedestrian congestion, etc. The system saves a huge amount of work in collecting diverse scene datasets in intelligent driving, while increasing the realism of simulated road scenes. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram of the process of a multi-scenario visual simulation testing system for intelligent driving;

[0040] Figure 2 A schematic diagram of road scene modeling in the multi-scenario simulation module of an intelligent driving multi-scenario visual simulation test system;

[0041] Figure 3 This is a schematic diagram illustrating the fusion of background and road information in the multi-scenario simulation module of an intelligent driving multi-scenario visual simulation test system.

[0042] Figure 4This is a schematic diagram illustrating the integration of different weather conditions into a scene within the multi-scene simulation module of an intelligent driving multi-scene visual simulation testing system. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] See Figure 1 This embodiment provides an intelligent driving multi-scenario visual simulation testing system, including:

[0046] The scene information acquisition module is used to collect information about actual road scenes;

[0047] The scene information acquisition module is a vehicle-mounted camera; the vehicle-mounted camera is used to acquire scene data; the vehicle-mounted camera is positioned to capture the maximum range of road scene in front of the vehicle; the scenes acquired by the vehicle-mounted camera include different weather conditions, different time periods, and different structured or unstructured roads; the acquired scenes cover multiple scenarios without requiring extensive collection of data from the same scenario; the acquired road scene information should, as far as possible, include all target categories on the roads, including but not limited to vehicles, pedestrians, and background information.

[0048] The data transmission module is used for information transfer between different modules;

[0049] The data transmission module is a wireless network communication device.

[0050] The scene segmentation module is used to segment and statistically analyze different target categories and probability distributions in actual road scenes, extract visual pixel information of different target categories in different scenes, and transmit it to the multi-scene simulation module.

[0051] The multi-scene simulation module is used to randomly combine and embed time periods, weather, road target categories, etc., obtained by the scene segmentation module to simulate different road traffic scenarios. This multi-scene segmentation module does not utilize real-world road information, but rather uses computer-generated, randomly changing curves as road boundary lines, which also serve as demarcation lines between the road and other categories of information. All embedded target categories and background information will not cross the road boundary lines and interfere with road information.

[0052] The results analysis module is used to evaluate algorithms related to intelligent driving.

[0053] The scene information acquisition module is connected to the scene segmentation module through the data transmission module; the scene segmentation module is connected to the multi-scene simulation module through the data transmission module; and the multi-scene simulation module is connected to the result analysis module through the data transmission module.

[0054] The system provided by this invention has a certain update capability. The threshold set in the system can be continuously updated through data collection and learning training in actual autonomous driving; the intelligent algorithm and training method integrated in the control module can be updated by input.

[0055] Example 2

[0056] This embodiment provides a method for multi-scenario visual simulation testing of intelligent driving, implemented through an intelligent driving multi-scenario visual simulation testing system, including the following steps:

[0057] Step 1: Collect scene information through the scene information acquisition module, and then transmit the collected scene information to the scene segmentation module through the data transmission module;

[0058] Step 2: The scene information is segmented and statistically analyzed by the scene segmentation module to extract visual pixel information of different target categories in different scenes, and then transmitted to the multi-scene simulation module through the data transmission module.

[0059] The specific method is as follows:

[0060] 21) Using semantic segmentation methods, vehicles, pedestrians, and roads are used as the primary target categories, while other categories are used as background categories;

[0061] 22) Based on vehicles, pedestrians and roads as the main categories, the frequency of each category in all scenarios is statistically analyzed to obtain its probability distribution f, which represents its frequency of occurrence in road scenarios.

[0062] 23) Select a road scene information of a certain period of time on a day with sufficient light, and use the static road scene information of a vehicle as the reference scene information γ(t0). Use the same road scene information of other time periods and other weather conditions as the identifier extraction scene. Calculate the difference in image pixel value of a single video frame between the identifier extraction scene and the reference scene at different time periods t, and between the identifier extraction scene and the reference scene at different weather conditions. Obtain the pixel value differences represented by traffic environments such as different time periods and weather conditions as γ(t)-γ(t0) and κ(t)-γ(t0), respectively. Arrange the information according to the time sequence of video frames to represent the pixel sequence of different time periods and weather changes.

[0063] Step 3: The visual pixel information obtained by the scene segmentation module is randomly combined and embedded through the multi-scene simulation module to simulate different road traffic scenarios, and the different road traffic scenarios are passed to the result analysis module.

[0064] The specific method is as follows:

[0065] The multi-scene simulation module randomly combines and embeds the time period, weather, and road target categories obtained by the scene segmentation module to simulate different road traffic scenarios. The multi-scene segmentation module uses computer-generated, randomly changing curves as road boundary lines, which also serve as demarcation lines between the module and other categories of information. All embedded target categories and background information will not exceed the road boundary lines and interfere with the road information. Specifically:

[0066] 31) See Figure 2 Based on the perspective of road scenes captured by the vehicle-mounted camera, road boundary lines and their extension directions are established in a blank image background, conforming to the width and angle of a real road scene. The lower left corner of the image is taken as the origin o of the coordinate system, the upward direction is the y-axis, and the rightward direction is the x-axis. (x, y) represents a point on the road boundary line. To simulate the changes in real road boundary lines, the specific changes in the road boundary lines on both sides are represented by two function curves:

[0067] Left boundary line: y = ψ(x) * ω(t)

[0068] Right boundary line: y = ψ(x + a(x))ω(t)

[0069] Where ψ(x) represents the curvature change of the lane line; a(x) represents the change in road width; t represents the time series; ω(t) represents the rate of change of the function over time; and the set scenario is used as the baseline scenario.

[0070] 32) See Figure 3 Based on the defined lane lines, the background target categories segmented by the scene segmentation module are randomly embedded onto both sides of the road boundary lines according to probability f and time t. Vehicles and pedestrians, as the main categories, are randomly embedded into the road scene according to probability f and time t. Then, the simulated change rate v0 of the background in the video sequence changes with the simulated detected vehicle speed v1 as follows:

[0071] v0=δ(f,t)v1

[0072] The changes in simulated road vehicle speed v2 and simulated pedestrian speed v3 with respect to simulated detected vehicle speed v1, time t, and the horizontal axis x of the image are expressed as follows:

[0073] v2=λ(v1,t,x0,y), v3=φ(v1,t,x), [x <x0<x+a]

[0074] All vehicles and pedestrians are given the attribute of slowing down or avoiding obstacles when they encounter them, where λ and φ are curves related to the speed of vehicles and pedestrians, and the threshold is the same as the actual speed of vehicles and pedestrians.

[0075] 33) See Figure 4 To simulate weather changes, time pixel difference γ(t)-γ(t0) and weather pixel difference κ(t)-γ(t0) are embedded in the baseline scene in step 31) to represent the weather conditions at different time periods.

[0076] Step 4: Evaluate the performance of intelligent driving-related algorithms under different road traffic scenarios through the results analysis module; assess the effectiveness, robustness, and generalization ability of intelligent driving-related algorithms.

[0077] Different algorithms have specific evaluation criteria and methods.

[0078] This invention provides a randomly combinable autonomous driving evaluation simulation scenario that can simulate as many real-world scenarios as possible without requiring multiple collections of scenario information.

[0079] In summary, this invention requires only a small amount of actual road scene data collection to simulate real-world multi-scenario information, including different time periods each day, weather conditions, and vehicle and pedestrian congestion. This system saves a huge amount of work in collecting diverse scenario datasets for intelligent driving, while increasing the realism of simulated road scenarios.

[0080] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

[0081] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.

Claims

1. A method for intelligent driving multi-scene visual simulation test, realized by an intelligent driving multi-scene visual simulation test system, characterized in that, Comprise the following steps: Step one, through the scene information acquisition module acquires scene information, and the collected scene information module is transmitted to the scene segmentation module through the data transmission module; Step two, through the scene segmentation module, the visual pixel information of different scene different target categories is extracted, and the data transmission module is transmitted to the multi-scene simulation module; Step three, through the multi-scene simulation module, the visual pixel information obtained by the scene segmentation module is randomly combined and embedded, different road traffic scenes are simulated, and different road traffic scenes are transmitted to the result analysis module; Step four, through the result analysis module, the performance of intelligent driving related algorithm in different road traffic scenes is evaluated; The specific method of step three is as follows: The time period, weather, road target category obtained by the scene segmentation module are randomly combined and embedded by the multi-scene simulation module, and different road traffic scenes are simulated; The multi-scene simulation module uses the randomly changing curve generated by the computer as the road boundary line, which is also used as the boundary line of other categories of information. All embedded target categories and background information will not exceed the road boundary line and interfere with the road information. Specifically: 31) According to the perspective of the vehicle-mounted camera shooting the road scene, the road boundary line and the extension direction conforming to the real road scene width and angle are established in the blank picture background. The top vertex of the left corner of the image is taken as the coordinate system origin O, the upward direction is taken as the image y axis direction, and the right direction is taken as the x axis direction. (x, y) represents the point on the road boundary line. In order to simulate the real change of road boundary line, the specific change mode of the road boundary line on both sides is replaced by two function curves: Left boundary line: y = ψ (x) * ω (t) Right boundary line: y = ψ (x + a (x)) ω (t) Wherein, ψ (x) represents the curvature change of lane line; a (x) represents the change value of road width; t represents time sequence; ω (t) represents the change rate of function change with time; The set scene is taken as the reference scene; 32) According to the defined lane line, the background target category segmented by the scene segmentation module is randomly embedded into the road boundary line on both sides according to the probability f and time t. The vehicle and pedestrian as the main category are randomly embedded into the road scene according to the probability f and time t. The simulation change speed v0 of the background in the video sequence is represented as: v0 = δ (f, t) v1 The simulation road vehicle speed v2 and the simulation pedestrian speed change v3 are represented as: v2 = λ (v1, t, x0, y), v3 = φ (v1, t, x), [x < x0 < x + a] All vehicles and pedestrians are endowed with the properties of deceleration or obstacle avoidance when encountering obstacles, wherein λ and φ are curves related to vehicle and pedestrian speed, and the threshold is the same as the actual vehicle and pedestrian speed; 33) In order to simulate the weather change condition, the time pixel difference γ(t)-γ(t0) and the weather pixel difference κ(t)-γ(t0) are embedded in the reference scene in step 31) respectively to represent the weather condition in different time period. 2.The intelligent driving multi-scene visual simulation test method according to claim 1, characterized in that, The specific method of step two is as follows: 21) Using semantic segmentation method, taking vehicle, pedestrian and road as main target categories, and other categories as background categories; 22) Based on taking vehicle, pedestrian and road as main categories, counting the number of all categories appearing in all scenes to obtain the probability distribution f to represent the frequency of appearing in road scene; 23) Selecting a certain time period of a day with sufficient light, a certain period of road scene information of static vehicle as reference scene information γ(t0), and the same road scene information of other time period and other weather as identification extraction scene, calculating the single video frame image pixel value difference between the identification extraction scene and the reference scene in different time period t, and obtaining the pixel value difference represented by different time period and weather as γ(t)-γ(t0) and κ(t)-γ(t0), and arranging the information according to the video frame time sequence to represent the pixel sequence of different time period and weather change.

3. An intelligent driving multi-scene visual simulation test system for implementing the intelligent driving multi-scene visual simulation test method of claim 1, characterized in that, It includes: Scene information acquisition module, used for acquiring information of actual road scene; Data transmission module, used for information transmission between different modules; Scene segmentation module, used for segmenting and counting different target categories and probability distribution in actual road scene, extracting visual pixel information of different target categories in different scenes, and transmitting to multi-scene simulation module; Multi-scene simulation module, used for randomly combining segmented scenes to simulate different road traffic scenes; Result analysis module, used for evaluating intelligent driving related algorithm; The scene information acquisition module is connected with the scene segmentation module through the data transmission module; the scene segmentation module is connected with the multi-scene simulation module through the data transmission module; the multi-scene simulation module is connected with the result analysis module through the data transmission module.

4. The intelligent driving multi-scene visual simulation test system according to claim 3, characterized in that, The scene information acquisition module is a vehicle-mounted camera; the vehicle-mounted camera is used for acquiring scene; the vehicle-mounted camera is arranged at a position capable of shooting the maximum range of road scene in front of the vehicle; the scene acquired by the vehicle-mounted camera includes different weather, different time period, different structured road or unstructured road; the acquired scene covers multiple scenes; the road scene includes target categories in all roads.

5. The intelligent driving multi-scene visual simulation test system according to claim 3, characterized in that, The data transmission module is a wireless network communication device.

6. The intelligent driving multi-scene visual simulation test method according to claim 3, characterized in that, The result analysis module evaluates the effectiveness, robustness and generalization ability of intelligent driving related algorithm.

Images