Unmanned vehicle adaptive test system based on digital twinning and reinforcement learning

Abstract

The invention provides an unmanned vehicle adaptive test system based on digital twinning and reinforcement learning, and the system comprises a digital twinning modeling module which is used for constructing an unmanned vehicle digital twinning model and a digital twinning test scene, and the digital twinning model is used for mapping the state of a virtual unmanned vehicle in real time, and carrying out the collision and comfort detection of the virtual unmanned vehicle, the digital twin test scene is used for simulating a physical test environment; the reinforcement learning test module is used for performing path planning test on the virtual unmanned vehicle by utilizing a reinforcement learning algorithm to generate a test case; and the test optimization module is used for comparing the deviation between the digital twin test scene and the physical real test environment in the test case and the reaction deviation between the virtual unmanned vehicle and the entity unmanned vehicle, and optimizing the test case. The physical test environment and the unmanned vehicle state are mapped in real time, the driving path of the unmanned vehicle is dynamically adjusted, the optimal test case is generated, the test strategy is continuously optimized, and the unmanned vehicle test comprehensiveness, efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned vehicle testing technology, and in particular to an adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning. Background Technology

[0002] Traditional autonomous vehicle testing methods mainly include real-vehicle road testing and field testing. Real-vehicle road testing can truly reflect the performance of autonomous vehicles in real-world environments, but it suffers from problems such as long testing cycles, high costs, limited scenario coverage, and a lack of testing for complex and dangerous scenarios. Field testing cannot dynamically adjust test conditions, incurs high testing costs, and makes it difficult to fully verify the adaptive capabilities of autonomous vehicles.

[0003] In existing technologies, digital twins and reinforcement learning are used for autonomous vehicle simulation testing. However, these technologies cannot dynamically adjust the test cases for autonomous vehicle simulation to achieve the best state, and they cannot solve the consistency problem between virtual testing and physical testing. Summary of the Invention

[0004] Therefore, the purpose of this invention is to provide an adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning, so as to solve or at least partially solve the above-mentioned problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides an adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning, the system comprising: Digital twin modeling module: used to construct digital twin models and digital twin test scenarios for autonomous vehicles. The digital twin model is used to map the virtual autonomous vehicle state in real time. The virtual autonomous vehicle state includes driving parameters and control logic. It also performs longitudinal driving collision detection, lane change collision detection, and comfort detection on the virtual autonomous vehicle in the digital twin test scenario. The comfort detection is used to calculate the passenger comfort of the virtual autonomous vehicle during driving. The digital twin test scenario is used to simulate the road conditions, traffic participants, and weather lighting of the physical test environment. Reinforcement learning testing module: Used to perform path planning tests on virtual driverless vehicles using reinforcement learning algorithms in different digital twin testing scenarios, generate test cases and dynamically adjust them; Test optimization module: This module receives test cases generated by the reinforcement learning test module, compares the deviations between the digital twin test scenarios and the physical real test environment in the test cases, and compares the reaction deviations between the virtual driverless car in the digital twin model and the physical driverless car. Based on the comparison results, iteratively optimizes the test cases.

[0006] Furthermore, the construction of the digital twin test scenario includes: A dynamic traffic scene database is constructed using geographic information systems and 3D rendering technology. The dynamic traffic scene database includes road conditions, traffic participants, and weather and lighting scene parameters, and the scene parameters can be customized. Virtual coordinate mapping is performed on the traffic participants, and the virtual coordinate mapping enables data exchange and information synchronization between the physical unmanned vehicles and the virtual unmanned vehicles.

[0007] Furthermore, the virtual coordinate mapping specifically includes the following steps: S11. Obtain the coordinate information of the physical unmanned vehicle, and perform virtual coordinate mapping on the current coordinates of the virtual unmanned vehicle in the digital twin test scenario based on the coordinate information; S12. Perform translation, rotation, and scaling on the current coordinates of the virtual unmanned vehicle to obtain the coordinate translation mapping matrix, rotation mapping matrix, and scaling mapping matrix. The translation mapping involves moving the virtual unmanned vehicle along the current coordinates in a collision-free state. axis, axis, Moving along the axis, the rotation mapping is the virtual unmanned vehicle rotating around a coordinate axis at a preset angle; S13. Perform matrix linear transformation on the current coordinates of the virtual unmanned vehicle based on the translation mapping matrix, rotation mapping matrix and scaling mapping matrix to obtain the virtual coordinate matrix. Use the virtual coordinate matrix to convert the coordinate information of the physical unmanned vehicle into the coordinates in the digital twin test scenario.

[0008] Furthermore, the digital twin test scenario also includes a virtual camera, which is used to switch perspectives between different virtual driverless vehicles, specifically including the following steps: S21. Create an empty object in the digital twin test scenario, use the empty object as the target point of the virtual camera, and set multiple target points around the virtual unmanned vehicle. S22. Obtain the coordinate information of the current virtual camera's target point and the coordinate information of the virtual unmanned vehicle's target point. Calculate the vector difference required to transform the virtual camera's viewpoint to the virtual unmanned vehicle's target point. Perform rotation and translation mapping on the virtual camera's coordinate information based on the vector difference to achieve viewpoint transformation for different virtual unmanned vehicles.

[0009] Furthermore, the digital twin model performs longitudinal collision detection for the autonomous vehicle, specifically including the following steps: S31. Assuming that the autonomous vehicles performing collision detection are autonomous vehicle A and autonomous vehicle B, obtain the total distance from the center point of autonomous vehicle A to the center point of autonomous vehicle B. S32. Obtain the first distance from the center point of autonomous vehicle A to one end of autonomous vehicle A adjacent to autonomous vehicle B, and the second distance from the center point of autonomous vehicle B to one end of autonomous vehicle B adjacent to autonomous vehicle A. If the sum of the first distance and the second distance is less than the total distance, autonomous vehicle A and autonomous vehicle B do not collide; otherwise, autonomous vehicle A and autonomous vehicle B collide. S33. Based on the longitudinal collision detection results, correct the virtual driverless vehicle state to avoid collisions.

[0010] Furthermore, the digital twin model performs lane change collision detection on the virtual driverless vehicle, specifically including: Assuming the lane-changing autonomous vehicle is The unmanned vehicle behind the target lane change lane is The target lane ahead of the unmanned vehicle is Set a lane-changing distance threshold; if the driverless car... With driverless cars If the real-time distance is less than the lane change distance threshold, the lane change condition is not met; otherwise, the lane change condition is met. When the lane change condition is met, it is represented as follows:

[0011] in, For lane changing driverless car The forward travel distance during a lane change. The driverless car behind the target lane changed lanes In driverless cars The forward travel distance during a lane change. For lane changing driverless car The length of the car body, For lane changing driverless car The width of the vehicle body, For lane changing driverless car The angle formed between the vehicle and the lane line during the lane change process for and The relative distance from the start to the end of the lane change; after the lane change ends, if there is no driver or vehicle... The driving speed is less than that of the driverless car The driving speed is at risk of collision. The virtual driverless car's state is corrected based on the lane change collision detection results to avoid lane change collisions.

[0012] Furthermore, the reinforcement learning testing module is used to perform the following operations: S41. Obtain the starting point, ending point, environmental state, and driving action parameters of the virtual unmanned vehicle. The environmental state is the location of the virtual traffic participant. Construct a mapping table between the environmental state, driving action parameters, and future rewards. The mapping table is used to store the expected rewards of the virtual unmanned vehicle. The expected rewards include the rewards for the virtual unmanned vehicle to take different driving actions under different environmental states. S42. If the virtual driverless car successfully reaches the destination by selecting the driving action with the highest expected reward, it receives a reward feedback; if it collides during driving, it receives a penalty feedback and records the collision sample. S43. Based on the update rule of the model-free temporal difference learning algorithm, an experience cache reuse mechanism is introduced to update the expected reward and optimize the driving action; S44. If the destination is not reached, return to the starting point and replan the route; otherwise, record the length of the traveled route. S45. Based on the update of the expected return, determine whether the mapping table has converged. If it has converged, the virtual driverless car selects the optimal driving action and determines the final driving path according to the mapping table. Otherwise, it searches for a new path until the convergence condition is met.

[0013] Furthermore, the test optimization module specifically includes the following steps: S51. By comparing the deviations between the digital twin test scenarios in the test cases and the physical real test environment, a scenario deviation matrix is ​​generated. S52. Compare the virtual driverless car state mapped in real time by the digital twin model with the actual state of the physical driverless car to generate a driverless car deviation matrix. S53. Perform structural correlation analysis based on the scene deviation matrix and the unmanned vehicle deviation matrix. Based on the correlation analysis results, the parameters of the digital twin model and the digital twin test scenario are corrected, and the test cases are iteratively optimized based on the corrected parameters.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes an adaptive testing system for autonomous vehicles based on digital twins and reinforcement learning. The system utilizes a digital twin modeling module to achieve real-time mapping of the virtual autonomous vehicle's state, collision detection simulation, and simulation of the physical testing environment. A reinforcement learning testing module performs path planning testing on the virtual autonomous vehicle, adjusting its driving path in real time. A test optimization module refines the digital twin model and the digital twin testing scenario. This invention can map the physical testing environment and the autonomous vehicle's state in real time, dynamically adjust the vehicle's driving path to generate optimal test cases, and continuously optimize testing strategies. This significantly improves the comprehensiveness, efficiency, and accuracy of autonomous vehicle testing, providing strong support for verifying the safety performance of autonomous vehicles. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the structure of an adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning, provided in an embodiment of the present invention. Detailed Implementation

[0017] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0018] Reference Figure 1 This embodiment provides an adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning. The system includes: Digital twin modeling module: Used to construct a digital twin model of the autonomous vehicle and a digital twin test scenario. The digital twin model is used to map the virtual autonomous vehicle's state in real time. The virtual autonomous vehicle's state includes driving parameters and control logic. It also performs longitudinal collision detection, lane change collision detection, and comfort detection on the virtual autonomous vehicle in the digital twin test scenario. The comfort detection is used to calculate passenger comfort during the virtual autonomous vehicle's operation. The digital twin test scenario is used to simulate road conditions, traffic participants, and weather lighting in the physical test environment, specifically including: The construction of the digital twin test scenario includes: A dynamic traffic scene library is constructed using geographic information systems and 3D rendering technology. The dynamic traffic scene library includes road conditions, traffic participants, and weather and lighting scene parameters. The road conditions include traffic light conditions, road curvature, etc. The dynamic traffic scene library has a variety of preset typical scene templates and supports users to customize and adjust scene parameters, such as road curvature, traffic flow, and weather conditions. At the same time, it can dynamically generate sudden scenes, such as pedestrians crossing the road, to improve the diversity and complexity of test scenes. Virtual coordinate mapping is performed on the traffic participants. This virtual coordinate mapping enables data exchange and information synchronization between the physical autonomous vehicles and the virtual autonomous vehicles among the traffic participants. The virtual coordinate mapping specifically includes the following steps: S11. Obtain the coordinate information of the physical unmanned vehicle, and perform virtual coordinate mapping on the current coordinates of the virtual unmanned vehicle in the digital twin test scenario based on the coordinate information; S12. Perform translation, rotation, and scaling on the current coordinates of the virtual unmanned vehicle to obtain the coordinate translation mapping matrix, rotation mapping matrix, and scaling mapping matrix. The translation mapping involves moving the virtual unmanned vehicle along the current coordinates in a collision-free state. axis, axis, Moving a certain distance along the axis, the rotation mapping is the virtual unmanned vehicle rotating around a coordinate axis at a certain angle; S13. Perform matrix linear transformation on the current coordinates of the virtual unmanned vehicle based on the translation mapping matrix, rotation mapping matrix and scaling mapping matrix to obtain the virtual coordinate matrix. Use the virtual coordinate matrix to convert the coordinate information of the physical unmanned vehicle into the coordinates in the digital twin test scenario.

[0019] Traditional coordinate mapping relies on fixed algorithms and manual parameter input, which can easily lead to mapping deviations due to changes in the state of the physical object. Furthermore, it cannot cope with the real-time movement of the physical object or changes in the scene. In contrast, digital twin test scene simulation based on translation mapping, rotation mapping, and scaling mapping can control the deviation between the virtual unmanned vehicle and the physical unmanned vehicle in terms of spatial position and attitude, ensuring the consistency of virtual and real mapping, and achieving more accurate digital twin test scene simulation, allowing the virtual scene to accurately replicate the state of the physical entity.

[0020] The digital twin test scenario also includes a virtual camera, which is used to switch the perspectives of different virtual driverless vehicles. Specifically, this includes the following steps: S21. Create an empty object in the digital twin test scenario, use the empty object as the target point of the virtual camera, and set multiple target points around the virtual unmanned vehicle. S22. Obtain the coordinate information of the current virtual camera's target point and the coordinate information of the virtual unmanned vehicle's target point. Calculate the vector difference required to switch the virtual camera's viewpoint to the virtual unmanned vehicle's target point. Perform rotation and translation mapping on the virtual camera's coordinate information based on the vector difference, allowing users to flexibly switch to the target vehicle's viewpoint, thereby providing comprehensive support for operational details such as road condition judgment and multi-vehicle collaborative exercises in simulation tests.

[0021] The digital twin model performs longitudinal collision detection on the virtual driverless vehicle, specifically including the following steps: S31. Assuming that the autonomous vehicles performing collision detection are autonomous vehicle A and autonomous vehicle B, obtain the total distance from the center point of autonomous vehicle A to the center point of autonomous vehicle B. S32. Obtain the first distance from the center point of autonomous vehicle A to one end of autonomous vehicle A adjacent to autonomous vehicle B, and the second distance from the center point of autonomous vehicle B to one end of autonomous vehicle B adjacent to autonomous vehicle A. If the sum of the first distance and the second distance is less than the total distance, autonomous vehicle A and autonomous vehicle B do not collide; otherwise, autonomous vehicle A and autonomous vehicle B collide. S33. Based on the longitudinal collision detection results, correct the virtual driverless vehicle state to avoid collisions, monitor the distance and relative speed between driverless vehicles in real time, provide early warning of rear-end collision risks, and provide reliable testing support for the safety and stability of physical driverless vehicles on roads.

[0022] The digital twin model performs lane change collision detection on the virtual driverless car, specifically including: Assuming the lane-changing autonomous vehicle is The unmanned vehicle behind the target lane change lane is The target lane ahead of the unmanned vehicle is Set a lane-changing distance threshold; if the driverless car... With driverless cars If the real-time distance is less than the lane change distance threshold, the lane change condition is not met; otherwise, the lane change condition is met. When the lane change condition is met, it is represented as follows:

[0023] in, For lane changing driverless car The forward travel distance during a lane change. The driverless car behind the target lane changed lanes In driverless cars The forward travel distance during a lane change. For lane changing driverless car The length of the car body, For lane changing driverless car The width of the vehicle body, For lane changing driverless car The angle formed between the vehicle and the lane line during the lane change process for and The relative distance from the start to the end of the lane change; after the lane change ends, if there is no driver or vehicle... The driving speed is less than that of the driverless car The driving speed is such that there is a risk of collision. The virtual driverless car state is corrected based on the lane change collision detection results to avoid lane change collisions. This simulates the collision phenomenon and enables the driverless car to make corresponding collision judgment decisions.

[0024] The longitudinal collision detection and lane change collision detection are also applicable to collisions between autonomous vehicles and other traffic participants.

[0025] The digital twin model is also used to detect the comfort level of the virtual driverless car, specifically including: With traffic signal light adaptation (e.g., green light start response time ≤ 1.5s), starting speed (e.g., initial acceleration ≤ 0.8m / s²), longitudinal acceleration (e.g., -2~1.5m / s²), and lateral acceleration (e.g., ≤ 1.2m / s²) as the core, a complete detection system is formed by preset safety thresholds for each indicator. Based on the safety thresholds of each indicator, the comfort weight of each indicator is analyzed, and passenger comfort reward and penalty items are introduced. Passenger comfort limit values ​​are preset, and passenger comfort values ​​are calculated based on the safety thresholds and comfort weights of each indicator, as shown below:

[0026] in, For passenger comfort, To adapt indicator weights for traffic lights, Safety thresholds for traffic signal light adaptation indicators. As the weight of the starting speed indicator, The starting speed indicator is a safe threshold. Weights for longitudinal and lateral acceleration indicators. The longitudinal and lateral acceleration indicators are safety thresholds. If the passenger comfort value is greater than or equal to the passenger comfort limit value, the passenger comfort is deemed to meet the standard, and the reward function is triggered. Conversely, if the passenger comfort is deemed to be unsatisfactory, the reward function is triggered. The indicators mentioned are not unique and can be set independently according to passenger comfort requirements.

[0027] The reinforcement learning testing module, based on the digital twin modeling module, uses reinforcement learning algorithms to test the path planning of autonomous vehicles in different digital twin testing scenarios, generates test cases, and dynamically adjusts them. It performs the following operations: S41. Obtain the starting point, ending point, environmental state, and driving action parameters of the unmanned vehicle. The environmental state refers to the location of other traffic participants in the digital twin test scenario. The driving action parameters refer to the acceleration, deceleration, lane changing, and braking actions of the virtual unmanned vehicle. Construct a mapping table between the environmental state, driving action parameters, and future rewards. The mapping table is used to store the expected rewards of the virtual unmanned vehicle. The expected rewards are the expected rewards of the virtual unmanned vehicle taking different driving actions under different environmental states. S42. The virtual unmanned vehicle observes the environmental state and real-time coordinate information, selects the driving action with the highest expected reward, and hopes to obtain the maximum immediate reward. If the virtual unmanned vehicle successfully reaches the destination through the selected driving action, it receives reward feedback. If the virtual unmanned vehicle collides during the driving process, it receives penalty feedback and records the collision sample. The weight of the recorded collision sample is increased to remind the unmanned vehicle to avoid the same mistake. If the virtual unmanned vehicle neither reaches the target point nor collides, the feedback value is between reward and penalty. S43. Based on the update rules of the model-free temporal difference learning algorithm, an experience cache reuse mechanism is introduced to update the expected reward of the virtual unmanned vehicle and optimize the driving action of the unmanned vehicle. The experience cache reuse mechanism is used to store the current environmental state, movement action, and reward feedback of the unmanned vehicle. The update rules of the model-free temporal difference learning algorithm can integrate immediate rewards and future rewards, balance short-term profits and long-term optimization, allow the virtual unmanned vehicle to gradually learn the optimal decision-making strategy, and update the expected reward according to the driving process of the virtual unmanned vehicle without waiting for the end of each test round, quickly iterate and correct value judgments, and quickly converge the cycle. S44. If the virtual driverless car does not reach the destination, it returns to the starting point and replans the route; if it reaches the destination, it records the length of the route traveled by the driverless car, which helps to evaluate the effectiveness and performance of the route planning. S45. Based on the update of the expected return of the autonomous vehicle in step S43, determine whether the mapping table constructed in step S41 has converged. If it has converged, the virtual autonomous vehicle selects the optimal driving action and determines the final driving path according to the mapping table. If it has not converged, the virtual autonomous vehicle restarts the path search until the convergence condition is met. The reinforcement learning testing module enables the virtual autonomous vehicle to improve the safety and efficiency of the virtual autonomous vehicle's driving path through reinforcement learning in different digital twin testing scenarios. It can dynamically adapt to changes in scenarios and avoid the limitations of fixed use cases.

[0028] Based on the determined final driving path, the safe distance between the virtual driverless car and other traffic participants in the corresponding environmental state is obtained, as well as the numerical range of speed changes in the optimal driving action. Based on the safe distance and numerical range, the safe distance limit value and the speed limit value under different environmental states of the virtual driverless car are determined. The safe distance limit value and the speed limit value are combined with passenger comfort detection to correct and constrain the state of the virtual driverless car, avoid generating extreme working conditions that exceed the passenger comfort limit value, and optimize the state of the virtual driverless car within the safety constraints.

[0029] The test optimization module receives test cases generated by the reinforcement learning test module, compares the deviations between the digital twin test scenarios and the physical real test environment in the test cases, and compares the reaction deviations between the virtual autonomous vehicle in the digital twin model and the physical autonomous vehicle. Based on the comparison results, iteratively optimizes the test cases, specifically including the following steps: S51. By comparing the deviations between the digital twin test scenarios in the test cases and the physical real test environment, a scenario deviation matrix is ​​generated. S52. Compare the virtual driverless car state mapped in real time by the digital twin model with the actual state of the physical driverless car to generate a driverless car deviation matrix. S53. Structural correlation analysis based on scene deviation matrix and autonomous vehicle deviation matrix, specifically including: Unify the mapping rules between the scene deviation matrix and the autonomous vehicle deviation matrix to ensure that the correspondence between the scene and the autonomous vehicle state deviation is consistent; Based on the correspondence, the correlation between the scene deviation matrix and the autonomous vehicle deviation matrix is ​​calculated, and the associated scene-autonomous vehicle state deviation combinations are identified. Based on the associated scenario-autonomous vehicle state deviation combination, analyze the sources of deviation between matrices, such as the deviation of scenario parameters leading to autonomous vehicle state deviation; Based on the sources of deviation between matrices, output the correlation analysis results.

[0030] Based on the correlation analysis results, the parameters of the digital twin model and the digital twin test scenario are corrected. The test cases are then iteratively optimized based on the corrected parameters to continuously improve the test results. The test optimization module can accurately quantify the dual deviations between the digital twin test scenario and the physical real test environment, as well as between the virtual and physical unmanned vehicles. Based on the comparison results, the test cases are iterated to narrow the consistency gap between the virtual and real scenarios, improve the model's mapping accuracy to the physical unmanned vehicle, and make the test cases more in line with the real operating logic. Ultimately, this provides a more realistic test basis for optimizing the test cases of the virtual unmanned vehicle, enhancing its reliability and adaptability for actual road driving.

[0031] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive testing system for unmanned vehicles based on digital twins and reinforcement learning, characterized in that, The system includes: Digital twin modeling module: used to construct digital twin models and digital twin test scenarios for autonomous vehicles. The digital twin model is used to map the virtual autonomous vehicle state in real time. The virtual autonomous vehicle state includes driving parameters and control logic. It also performs longitudinal driving collision detection, lane change collision detection, and comfort detection on the virtual autonomous vehicle in the digital twin test scenario. The comfort detection is used to calculate the passenger comfort of the virtual autonomous vehicle during driving. The digital twin test scenario is used to simulate the road conditions, traffic participants, and weather lighting of the physical test environment. Reinforcement learning testing module: Used to perform path planning tests on virtual driverless vehicles using reinforcement learning algorithms in different digital twin testing scenarios, generate test cases and dynamically adjust them; Test optimization module: This module receives test cases generated by the reinforcement learning test module, compares the deviations between the digital twin test scenarios and the physical real test environment in the test cases, and compares the reaction deviations between the virtual driverless car in the digital twin model and the physical driverless car. Based on the comparison results, iteratively optimizes the test cases.

2. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 1, characterized in that, The construction of the digital twin test scenario includes: A dynamic traffic scene database is constructed using geographic information systems and 3D rendering technology. The dynamic traffic scene database includes road conditions, traffic participants, and weather and lighting scene parameters, and the scene parameters can be customized. Virtual coordinate mapping is performed on the traffic participants, and the virtual coordinate mapping enables data exchange and information synchronization between the physical unmanned vehicles and the virtual unmanned vehicles.

3. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 2, characterized in that, The virtual coordinate mapping specifically includes the following steps: S11. Obtain the coordinate information of the physical unmanned vehicle, and perform virtual coordinate mapping on the current coordinates of the virtual unmanned vehicle in the digital twin test scenario based on the coordinate information; S12. Perform translation, rotation, and scaling on the current coordinates of the virtual unmanned vehicle to obtain the coordinate translation mapping matrix, rotation mapping matrix, and scaling mapping matrix. The translation mapping involves moving the virtual unmanned vehicle along the current coordinates in a collision-free state. axis, axis, Moving along the axis, the rotation mapping is the virtual unmanned vehicle rotating around a coordinate axis at a preset angle; S13. Perform matrix linear transformation on the current coordinates of the virtual unmanned vehicle based on the translation mapping matrix, rotation mapping matrix and scaling mapping matrix to obtain the virtual coordinate matrix. Use the virtual coordinate matrix to convert the coordinate information of the physical unmanned vehicle into the coordinates in the digital twin test scenario.

4. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 3, characterized in that, The digital twin test scenario also includes a virtual camera, which is used to switch the perspectives of different virtual driverless vehicles. Specifically, this includes the following steps: S21. Create an empty object in the digital twin test scenario, use the empty object as the target point of the virtual camera, and set multiple target points around the virtual unmanned vehicle. S22. Obtain the coordinate information of the current virtual camera's target point and the coordinate information of the virtual unmanned vehicle's target point. Calculate the vector difference required to transform the virtual camera's viewpoint to the virtual unmanned vehicle's target point. Perform rotation and translation mapping on the virtual camera's coordinate information based on the vector difference to achieve viewpoint transformation for different virtual unmanned vehicles.

5. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 1, characterized in that, The virtual driverless vehicle performs longitudinal collision detection in a digital twin testing scenario, specifically including the following steps: S31. Assuming that the autonomous vehicles performing collision detection are autonomous vehicle A and autonomous vehicle B, obtain the total distance from the center point of autonomous vehicle A to the center point of autonomous vehicle B. S32. Obtain the first distance from the center point of autonomous vehicle A to one end of autonomous vehicle A adjacent to autonomous vehicle B, and the second distance from the center point of autonomous vehicle B to one end of autonomous vehicle B adjacent to autonomous vehicle A. If the sum of the first distance and the second distance is less than the total distance, autonomous vehicle A and autonomous vehicle B do not collide; otherwise, autonomous vehicle A and autonomous vehicle B collide. S33. Based on the longitudinal collision detection results, correct the virtual driverless vehicle state to avoid collisions.

6. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 1, characterized in that, The digital twin model performs lane change collision detection on the virtual driverless car, specifically including: Assuming the lane-changing autonomous vehicle is The unmanned vehicle behind the target lane change lane is The target lane ahead is an unmanned vehicle. Set a lane-changing distance threshold; if the driverless car... With driverless cars If the real-time distance is less than the lane change distance threshold, the lane change condition is not met; otherwise, the lane change condition is met. When the lane change condition is met, it is represented as follows: in, For lane changing driverless car The forward travel distance during a lane change. The driverless car behind the target lane changed lanes In driverless cars The forward travel distance during a lane change. For lane changing driverless car The length of the car body, For lane changing driverless car The width of the vehicle body, For lane changing driverless car The angle formed between the vehicle and the lane line during the lane change process for and The relative distance from the start to the end of the lane change; after the lane change ends, if there is no driver or vehicle... The driving speed is less than that of the driverless car The driving speed is such that there is a risk of collision. The virtual driverless car's state is corrected based on the lane change collision detection results to avoid lane change collisions.

7. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 1, characterized in that, The reinforcement learning testing module is used to perform the following operations: S41. Obtain the starting point, ending point, environmental state, and driving action parameters of the virtual unmanned vehicle. The environmental state is the location of the virtual traffic participant. Construct a mapping table between the environmental state, driving action parameters, and future rewards. The mapping table is used to store the expected rewards of the virtual unmanned vehicle. The expected rewards include the rewards for the virtual unmanned vehicle to take different driving actions under different environmental states. S42. If the virtual driverless car successfully reaches the destination by selecting the driving action with the highest expected reward, it receives a reward feedback; if it collides during driving, it receives a penalty feedback and records the collision sample. S43. Based on the update rule of the model-free temporal difference learning algorithm, an experience cache reuse mechanism is introduced to update the expected reward and optimize the driving action; S44. If the destination is not reached, return to the starting point and replan the route; otherwise, record the length of the route. S45. Based on the update of the expected return, determine whether the mapping table has converged. If it has converged, the virtual driverless car selects the optimal driving action and determines the final driving path according to the mapping table. Otherwise, it searches for a new path until the convergence condition is met.

8. The autonomous vehicle adaptive testing system based on digital twin and reinforcement learning according to claim 1, characterized in that, The test optimization module specifically Includes the following steps: S51. By comparing the deviations between the digital twin test scenarios in the test cases and the physical real test environment, a scenario deviation matrix is ​​generated. S52. Compare the virtual driverless car state mapped in real time by the digital twin model with the actual state of the physical driverless car to generate a driverless car deviation matrix. S53. Perform structural correlation analysis based on the scene deviation matrix and the unmanned vehicle deviation matrix. Based on the correlation analysis results, the parameters of the digital twin model and the digital twin test scenario are corrected, and the test cases are iteratively optimized based on the corrected parameters.

Images