A virtual-real fusion test system comprising a virtual vehicle, a real vehicle and a digital twin vehicle thereof

Abstract

The application provides a virtual-real fusion test system comprising virtual vehicles, real vehicles and digital twin vehicles, and relates to the field of virtual-real fusion testing.The system comprises a virtual simulation environment, a virtual-real sensor fusion module and a registration module.The virtual-real sensor fusion module is used to obtain virtual environment information perceived by the digital twin vehicles and physical environment information perceived by the real vehicles.The virtual simulation environment comprises the digital twin vehicles, the digital twin vehicles are in communication connection with the registration module, the registration module can obtain state data of the real vehicles and state data of the digital twin vehicles, and generate correction control instructions.The virtual simulation environment further comprises a plurality of virtual vehicles, and each virtual vehicle is in communication connection with a preset virtual vehicle autonomous decision module.The application can realize the function of joint testing of multiple vehicles, and when virtual obstacles are set in the virtual environment, each virtual vehicle can simultaneously obtain virtual obstacle information, thereby ensuring the effect of joint testing.

Description

Technical Field

[0001] This invention relates to the field of virtual-real fusion testing, and in particular to a virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle. Background Technology

[0002] In the development of special tracked vehicles, joint obstacle avoidance testing of multiple special tracked vehicles is involved to test the performance of obstacle avoidance actions when multiple vehicles work together. Unmanned special vehicles are tracked special vehicles that operate on non-standard road surfaces. Existing testing methods are not adapted to this type of special vehicle. In actual testing, in order to simplify the testing process, virtual obstacles are usually used as test obstacles. That is, virtual obstacle information is sent to each special tracked vehicle under test so that each special tracked vehicle can perceive the existence of the virtual obstacle and take corresponding avoidance actions. However, due to the different locations of special vehicles and the different time taken for the virtual obstacle information to be transmitted to each special tracked vehicle, each special tracked vehicle cannot receive the virtual obstacle information at the same time, resulting in poor joint obstacle avoidance test results of multiple special tracked vehicles. Summary of the Invention

[0003] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0004] According to a first aspect of this application, a virtual-real fusion testing system is provided, comprising a virtual vehicle, a real vehicle, and their digital twins, the system comprising:

[0005] The system includes a virtual simulation environment, a virtual-real sensor fusion module, and a registration module. The virtual simulation environment contains a digital twin vehicle and a virtual vehicle. The digital twin vehicle is simultaneously connected to the registration module and the virtual-real sensor fusion module, while the preset real vehicle is also simultaneously connected to the registration module and the virtual-real sensor fusion module. Each digital twin vehicle uniquely corresponds to one real vehicle.

[0006] The virtual-real sensor fusion module is used to acquire virtual environment information perceived by the digital twin vehicle and physical environment information perceived by the real vehicle, and to fuse the virtual environment information perceived by the digital twin vehicle and the physical environment information perceived by the real vehicle to obtain fused data, and send the fused data to the real vehicle; the real vehicle generates real vehicle control commands based on the fused data and real vehicle status data, and sends the real vehicle control commands to the registration module.

[0007] The virtual simulation environment includes a digital twin vehicle, which is communicatively connected to a registration module. The registration module can acquire the status data of the real vehicle and the digital twin vehicle, and correct the control commands of the real vehicle based on the status data of the real vehicle and the digital twin vehicle, generate corrected control commands, and send the corrected control commands to the digital twin vehicle, so that the digital twin vehicle keeps its state synchronized with the real vehicle according to the corrected control commands.

[0008] The virtual simulation environment also includes several virtual vehicles, each of which is connected to a preset virtual vehicle autonomous decision-making module. Each virtual vehicle can sense virtual environment information and send the virtual environment information to the corresponding virtual vehicle autonomous decision-making module. The virtual vehicle autonomous decision-making module generates virtual vehicle control commands based on the virtual environment information and sends the virtual vehicle control commands to the corresponding virtual vehicle, so that the virtual vehicle drives according to the virtual vehicle control commands.

[0009] The present invention has at least the following beneficial effects:

[0010] The present invention provides a virtual-real fusion testing system comprising a virtual vehicle, a real vehicle, and their digital twin vehicles. The system includes a virtual simulation environment, a virtual-real sensor fusion module, and a registration module. The virtual-real sensor fusion module acquires virtual environment information perceived by the digital twin vehicle and physical environment information perceived by the real vehicle, fuses these two information to obtain fused data, and sends the fused data to the real vehicle. The real vehicle generates control commands based on the fused data and its status data, and sends these commands to the registration module. The virtual simulation environment includes several virtual vehicles. Each virtual vehicle communication connection has a preset virtual vehicle autonomous decision-making module; each virtual vehicle can sense virtual environment information and send the virtual environment information to the corresponding virtual vehicle autonomous decision-making module. The virtual vehicle autonomous decision-making module generates virtual vehicle control commands based on the virtual environment information and sends the virtual vehicle control commands to the corresponding virtual vehicle, so that the virtual vehicle drives according to the virtual vehicle control commands; each autonomous decision-making module represents a real vehicle, thereby realizing the function of joint testing of multiple vehicles. When virtual obstacles are set in the virtual environment, each virtual vehicle can simultaneously obtain virtual obstacle information, thereby ensuring the effectiveness of joint testing.

[0011] Furthermore, the virtual simulation environment includes a digital twin vehicle, which is communicatively connected to a registration module. The registration module can acquire real vehicle status data and digital twin vehicle status data, and correct the real vehicle control commands based on the real vehicle status data and digital twin vehicle status data, generating corrected control commands, and sending the corrected control commands to the digital twin vehicle, so that the digital twin vehicle keeps its state synchronized with the real vehicle according to the corrected control commands. This allows the actual vehicle in motion to be synchronized in the virtual environment, forming a virtual-real fusion testing system of the real vehicle, digital twin vehicle, and virtual vehicle. This reduces the investment of actual vehicles while ensuring data synchronization, making the test results more accurate. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A framework diagram of a virtual-real fusion testing system including a virtual vehicle, a real vehicle, and its digital twin vehicle, provided for an embodiment of the present invention;

[0014] Figure 2 This is a framework diagram of another virtual-real fusion testing system including a virtual vehicle, a real vehicle, and its digital twin vehicle, provided for an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0017] Traditional virtual simulation experimental environments primarily utilize structured roads, which cannot meet the unique requirements of specialized unmanned vehicles. Therefore, this patent aims to address the challenges of virtual testing for specialized unmanned vehicles, providing a more accurate and reliable parallel testing system for the design, development, and testing of such vehicles.

[0018] The following will refer to Figure 1 The diagram shown illustrates a virtual-real fusion testing system that includes a virtual vehicle, a physical vehicle, and its digital twin. This diagram introduces a virtual-real fusion testing system that includes a virtual vehicle, a physical vehicle, and its digital twin.

[0019] The virtual-real fusion testing system, which includes virtual vehicles, physical vehicles, and their digital twins, comprises:

[0020] The system includes a virtual simulation environment, a virtual-real sensor fusion module, and a registration module. The virtual simulation environment contains a digital twin vehicle and a virtual vehicle. The digital twin vehicle is simultaneously connected to the registration module and the virtual-real sensor fusion module, while the preset real vehicle is also simultaneously connected to the registration module and the virtual-real sensor fusion module. There is a one-to-one correspondence between a digital twin vehicle and a real vehicle.

[0021] In this embodiment, the virtual simulation environment is a virtual platform, and the communication method is wireless communication; the virtual simulation environment is equipped with a digital twin vehicle and several virtual vehicles, the digital twin vehicle including:

[0022] The system comprises a digital twin vehicle status acquisition module, a digital twin vehicle virtual sensor module, and a digital twin vehicle controller module; wherein, the digital twin vehicle status acquisition module is used to acquire status data of the digital twin vehicle and send the status data of the digital twin vehicle to the registration module.

[0023] In this embodiment, the digital twin vehicle status data includes the digital twin vehicle's position data, attitude data, and speed data; the digital twin vehicle's attitude data includes the digital twin vehicle's curvature; the digital twin vehicle's position data is the digital twin vehicle's position coordinates in the virtual environment, and the digital twin vehicle's curvature is the curvature corresponding to the digital twin vehicle's driving path.

[0024] The digital twin vehicle virtual sensor module is used to sense virtual environment information and send the virtual environment information to the virtual-real sensor fusion module.

[0025] Furthermore, the virtual environment information includes virtual obstacle information, and each virtual vehicle and digital twin vehicle can simultaneously acquire virtual obstacle information.

[0026] In this embodiment, the test personnel can set one or more virtual obstacles at preset locations in the virtual environment. The virtual obstacle information includes the location information and size information of the virtual obstacle. The virtual obstacle information can be sent to each virtual vehicle and the digital twin vehicle at the same time, so that each virtual vehicle and the digital twin vehicle can simultaneously perceive the existence of the virtual obstacle.

[0027] It should be noted that both the digital twin car and the virtual vehicle are modeled one-to-one with the real vehicle, and have the functions of simulating LiDAR sensors, simulating RGB camera sensors, and simulating distance sensors.

[0028] The virtual controller module is used to receive correction control commands sent by the registration module and control the digital twin vehicle to drive according to the correction control commands.

[0029] In this embodiment, the driving state of the digital twin vehicle is ideally synchronized with that of the real vehicle, for example, their speeds and curvatures are the same. However, due to numerous uncertain influencing factors in the physical environment, such as wind resistance and suddenly appearing moving objects, the digital twin vehicle, although driving according to the control commands of the real vehicle, still cannot maintain state synchronization with the real vehicle. Therefore, in this application, a registration module is set up. After receiving the control command of the real vehicle, it acquires the current state data of the digital twin vehicle and the current state data of the real vehicle. Based on the current state data of the digital twin vehicle and the real vehicle, it corrects the control command of the real vehicle, generates a corrected control command, and uses the corrected control command to control the digital twin vehicle, thereby keeping the state of the digital twin vehicle synchronized with that of the real vehicle.

[0030] The virtual-real sensor fusion module is used to acquire virtual environment information perceived by the digital twin vehicle and physical environment information perceived by the real vehicle, and to fuse the virtual environment information perceived by the digital twin vehicle and the physical environment information perceived by the real vehicle to obtain fused data, and send the fused data to the real vehicle; the real vehicle generates real vehicle control commands based on the fused data and real vehicle status data, and sends the real vehicle control commands to the registration module.

[0031] In this embodiment, since the state of the digital twin vehicle differs from that of the real vehicle, there is a difference between the virtual environment information perceived by the digital twin vehicle and the physical environment information perceived by the real vehicle. The virtual environment information is absolutely accurate, while the physical environment information is affected by the sensors of the real vehicle and may have errors. Therefore, in this application, the virtual environment information and the physical environment information are fused by the virtual-real sensor fusion module to obtain fused data, and the fused data is sent to the registration module.

[0032] It should be noted that those skilled in the art can use existing data fusion methods to fuse virtual environment information and physical environment information according to actual needs. For example, they can overlay virtual images in the virtual environment information and actual images in the physical environment information to obtain a fused image; or they can add the coordinates in the virtual environment and the coordinates in the physical environment and calculate the average value.

[0033] The virtual simulation environment includes a digital twin vehicle, which is communicatively connected to a registration module. The registration module can acquire real vehicle status data and digital twin vehicle status data, and correct the real vehicle control commands based on the real vehicle status data and digital twin vehicle status data, generate corrected control commands, and send the corrected control commands to the digital twin vehicle, so that the digital twin vehicle keeps its state synchronized with the real vehicle according to the corrected control commands.

[0034] In this embodiment, the entire testing system uses only one actual unmanned vehicle, i.e., a real vehicle; the real vehicle includes:

[0035] The system includes a real vehicle status data acquisition module, a real vehicle perception module, a real vehicle autonomous decision-making module, and a real vehicle control module. The real vehicle status data acquisition module is used to collect the status data of the real vehicle and send the real vehicle status data to the registration module.

[0036] In this embodiment, the vehicle status data includes the vehicle's position data, attitude data, and speed data; the vehicle's attitude data includes the vehicle's tilt angle and front orientation; the vehicle's position data is the vehicle's position coordinates in the physical environment, and the vehicle's curvature is the tilt angle and front orientation corresponding to the vehicle's driving path.

[0037] The vehicle perception module is used to perceive physical environment information and send the physical environment information to the vehicle autonomous decision-making module; for example, the perception module includes distance sensors, image sensors, radar sensors, etc.; it can perceive various types of information about the physical environment.

[0038] The real vehicle autonomous decision-making module is used to generate real vehicle control commands based on the fused data and the real vehicle's status data, and send the real vehicle control commands to the real vehicle control module and the registration module. The real vehicle control commands include the desired speed and desired curvature of the real vehicle, where the desired speed corresponds to the throttle opening or electric throttle opening of the real vehicle, and the desired curvature corresponds to the steering wheel angle of the real vehicle. The above control information is sent to the real vehicle control module, and the real vehicle control module controls the real vehicle to drive according to the above control information.

[0039] The virtual simulation environment also includes several virtual vehicles, each of which is connected to a preset virtual vehicle autonomous decision-making module. Each virtual vehicle can sense virtual environment information and send the virtual environment information to the corresponding virtual vehicle autonomous decision-making module. The virtual vehicle autonomous decision-making module generates virtual vehicle control commands based on the virtual environment information and sends the virtual vehicle control commands to the corresponding virtual vehicle, so that the virtual vehicle drives according to the virtual vehicle control commands.

[0040] In this embodiment, several virtual vehicles are set up in the virtual simulation environment, and each virtual vehicle includes:

[0041] The system includes a virtual sensor module, a hardware / software-in-the-loop interface, and a virtual vehicle controller module. The virtual sensor module is used to sense virtual environment information and send the virtual environment information to the virtual vehicle autonomous decision-making module through the hardware / software-in-the-loop interface.

[0042] The software / hardware-in-the-loop interface is used to receive virtual vehicle control commands generated by the virtual vehicle autonomous decision-making module and send the virtual vehicle control commands to the virtual vehicle control module;

[0043] The virtual vehicle controller module is used to control the virtual vehicle's movement according to the virtual vehicle control commands.

[0044] In this embodiment, the virtual vehicle autonomous decision-making module can be understood as the control module of the real vehicle. Each virtual vehicle is connected to the control module of a real vehicle. The virtual vehicle autonomous decision-making module can generate corresponding virtual vehicle control commands based on the virtual environment information perceived by the virtual sensor module, and then send the virtual vehicle control commands to the corresponding virtual vehicle. The virtual vehicle then drives according to the virtual vehicle control commands.

[0045] In specific multi-vehicle joint testing, even if there is only one real vehicle in the entire test scenario, it is still possible to achieve joint testing of multiple vehicles. After setting virtual obstacles, each virtual vehicle and digital twin vehicle can simultaneously obtain virtual obstacle information, making the test results more accurate.

[0046] In specific implementation, you can refer to, for example Figure 2 The diagram shows a virtual-real fusion testing system that includes a virtual vehicle, a physical vehicle, and its digital twin. The detailed processing flow is shown in the diagram and will not be elaborated here.

[0047] This embodiment of the virtual-real fusion test system includes a virtual vehicle, a real vehicle, and their digital twin vehicles. It comprises a virtual simulation environment, a virtual-real sensor fusion module, and a registration module. The virtual-real sensor fusion module acquires virtual environment information perceived by the digital twin vehicle and physical environment information perceived by the real vehicle, fuses these two information to obtain fused data, and sends the fused data to the real vehicle. The real vehicle generates control commands based on the fused data and its status data, and sends these commands to the registration module. The virtual simulation environment includes several virtual vehicles. Each virtual vehicle has a pre-set autonomous decision-making module for communication. Each virtual vehicle can sense virtual environment information and send it to the corresponding autonomous decision-making module. The autonomous decision-making module generates virtual vehicle control commands based on the virtual environment information and sends them to the corresponding virtual vehicle, enabling the virtual vehicle to drive according to the commands. Each autonomous decision-making module represents a real vehicle, thus enabling joint testing of multiple vehicles. When virtual obstacles are set in the virtual environment, each virtual vehicle can simultaneously obtain virtual obstacle information, thereby ensuring the effectiveness of joint testing.

[0048] Furthermore, the virtual simulation environment includes a digital twin vehicle, which is communicatively connected to a registration module. The registration module can acquire real vehicle status data and digital twin vehicle status data, and correct the real vehicle control commands based on the real vehicle status data and digital twin vehicle status data, generating corrected control commands, and sending the corrected control commands to the digital twin vehicle, so that the digital twin vehicle keeps its state synchronized with the real vehicle according to the corrected control commands. This allows the actual vehicle in motion to be synchronized in the virtual environment, forming a virtual-real fusion testing system of the real vehicle, digital twin vehicle, and virtual vehicle. This reduces the investment of actual vehicles while ensuring data synchronization, making the test results more accurate.

[0049] The virtual-real fusion testing system of the present invention, which includes a virtual vehicle, a real vehicle, and their digital twin vehicle, also has the following advantages:

[0050] 1. Collaborative Testing: Supports multi-mode simulation and encourages collaborative work between virtual vehicles, physical vehicles, and digital twins to achieve more comprehensive vehicle performance evaluation.

[0051] 2. Reduce costs: Digital twin technology reduces the cost of actual vehicle testing and simulation, including physical testing, fuel, labor, and equipment costs.

[0052] 3. Hardware and software collaboration: Virtual simulation facilitates collaborative testing between vehicle hardware and software to ensure smooth cooperation and reduce integration costs.

[0053] 4. Environmental diversity: Supports a variety of simulation environments, including various road and climate conditions, to reduce the cost of testing diversity while maintaining high accuracy.

[0054] 5. Safety Verification: Through virtual simulation testing, the vehicle's response under dangerous conditions is verified to ensure the vehicle's safety and reliability, while reducing the risks in actual road testing.

[0055] 6. Data Analysis and Optimization: Provide data analysis tools to gain insights into vehicle performance from simulation tests and use these insights to improve design and performance, reducing development costs.

[0056] In addition, this highly collaborative and cost-effective simulation testing system supports vehicle simulation testing in multiple modes. By leveraging digital twin technology, it reduces testing costs, improves efficiency, and promotes the development of autonomous driving and vehicle autonomy technologies to create a more economical, safer, and more sustainable transportation environment.

[0057] In this embodiment, digital twin technology is used to link a virtual vehicle with a real vehicle, creating an integrated virtual-real test environment. This parallel testing system and method can simulate the behavior and performance of special-purpose unmanned vehicles in real-world working scenarios, providing a realistic operational experience and accurate data feedback. Through parallel testing with the real vehicle, the simulation accuracy and reliability of the virtual vehicle can be verified, further optimizing the design and performance of the special-purpose unmanned vehicle.

[0058] Furthermore, the system in this embodiment makes the design, development, and testing of special unmanned vehicles more efficient, accurate, and reliable; by associating the digital twin virtual vehicle with the corresponding real vehicle, an integrated virtual and real test environment is achieved.

[0059] In an exemplary embodiment, based on the virtual-real fusion testing system including a virtual vehicle, a real vehicle, and its digital twin vehicle as described in the above embodiments, this embodiment provides a registration method to support parallel testing of a real vehicle and its digital twin vehicle. The method is applied to the registration module of a digital twin vehicle testing system, which also includes a digital twin vehicle located in a virtual simulation environment. The registration module is communicatively connected to both the digital twin vehicle and the real vehicle. One digital twin vehicle corresponds one-to-one with one real vehicle. It should be noted that in this embodiment, there can be one or more real vehicles, with each real vehicle corresponding one-to-one with a digital twin vehicle. The digital twin vehicle and the real vehicle described in this application have a one-to-one correspondence, which will not be elaborated further below.

[0060] The actual vehicle includes a vehicle perception module, a vehicle control module, and a vehicle autonomous decision-making module; the vehicle perception module and the vehicle autonomous decision-making module are communicatively connected, and the vehicle autonomous decision-making module and the vehicle control module are communicatively connected; the vehicle perception module is used to collect the vehicle's status data and physical environment information, and the vehicle autonomous decision-making module generates vehicle control commands based on the vehicle's status data and physical environment information, and sends the vehicle control commands to the registration module.

[0061] The digital twin vehicle includes a digital twin vehicle status acquisition module and a digital twin vehicle control module. The digital twin vehicle status acquisition module is used to collect the status data of the digital twin vehicle and send the status data to the registration module.

[0062] In this embodiment, the actual vehicle is a tracked special vehicle, and the environment in which the actual vehicle travels is an unstructured road surface wilderness environment. The actual vehicle has a perception module that can perceive the surrounding environmental information and the actual vehicle status data, and the actual vehicle autonomous decision-making module can autonomously make decisions and generate actual vehicle control commands based on the environmental information and the actual vehicle status data. Among them, the actual vehicle control commands include the desired speed and desired curvature for controlling the actual vehicle, and the curvature can be understood as the curvature of the actual vehicle's travel path. At the same time, the actual vehicle also has an actual vehicle control module for controlling the actual vehicle's travel according to the actual vehicle control commands.

[0063] It should be noted that the digital twin vehicle does not have an autonomous decision-making module. The digital twin vehicle drives according to the correction control commands sent by the registration module. The registration module can acquire the real vehicle control commands sent by the real vehicle, the status data of the digital twin vehicle, and the status data of the real vehicle in real time. It can also correct the real vehicle control commands based on the status data of the digital twin vehicle and the real vehicle, and generate correction control commands for controlling the digital twin vehicle. It can be understood that the digital twin vehicle is synchronized with its corresponding real vehicle.

[0064] The registration method for supporting parallel testing of a real vehicle and its digital twin includes the following steps:

[0065] S100, in response to receiving the real vehicle control command ZL, the current state data MA_now of the digital twin vehicle and the current state data MB_now of the real vehicle are acquired. In this embodiment, since the motion model of the digital twin vehicle in the virtual environment cannot completely fit the motion environment of the real vehicle in the physical environment, after a period of accumulation, the state of the digital twin vehicle may differ from the state of the real vehicle; therefore, after receiving the real vehicle control command, the registration module acquires the current state data MA_now of the digital twin vehicle and the current state data MB_now of the real vehicle for subsequent judgment.

[0066] S200, based on MA_now and MB_now, determine the difference in current state data between the digital twin vehicle and the real vehicle, ΔMC_now=|MA_now-MB_now|.

[0067] In this embodiment, MA_now includes the current position data, attitude data, and speed data of the digital twin vehicle; MB_now includes the current position data, attitude data, and speed data of the real vehicle; the current attitude data of the digital twin vehicle includes the current tilt angle and front orientation of the digital twin vehicle; and the current attitude data of the real vehicle includes the current tilt angle and front orientation of the real vehicle.

[0068] Furthermore, MA_now = (WZ1, QL1, SD1), MB_now = (WZ2, QL2, SD2); where WZ1 is the current position data of the digital twin vehicle, QL1 is the current attitude data of the digital twin vehicle, SD1 is the current speed data of the digital twin vehicle; WZ2 is the current position data of the real vehicle, QL2 is the current attitude data of the real vehicle, and SD2 is the current speed data of the real vehicle.

[0069] Furthermore, we can obtain ΔMC_now=(ΔWZ_now,ΔQL_now,ΔSD_now); where ΔWZ_now is the difference in current position data between the digital twin vehicle and the real vehicle, ΔQL_now is the difference in current attitude data between the digital twin vehicle and the real vehicle, and ΔSD_now is the difference in current speed data between the digital twin vehicle and the real vehicle; ΔWZ_now=WZ1-WZ2, ΔQL_now=QL1-QL2, ΔSD_now=SD1-SD2.

[0070] Furthermore, after step S200 and before step S300, the method further includes:

[0071] S210, if ΔMC_now>BH, proceed to step S300; otherwise, send ZL to the digital twin vehicle control module; where BH is a preset state data difference threshold.

[0072] In some cases, ΔMC_now≤BH, it is assumed that the states of the digital twin vehicle and the real vehicle are synchronized. In this case, there is no need to modify the control commands of the real vehicle, and ZL can be sent directly to the control module of the digital twin vehicle. This can reduce the frequency of modifying the control commands of the real vehicle, reduce computing power consumption, and improve operating efficiency.

[0073] S300, determine the instruction correction parameters corresponding to ΔMC_now based on ΔMC_now and the preset instruction correction mapping table; wherein, the instruction correction mapping table includes several rows and several columns, each row corresponds to a state data difference range, and each column corresponds to the instruction correction parameters corresponding to the state data difference range.

[0074] Further, BH = (BH1, BH2, BH3); where BH1 is a preset position data difference threshold, BH2 is a preset attitude data difference threshold, and BH3 is a preset velocity data difference threshold; step S210 includes the following steps:

[0075] S211, if ΔWZ_now > BH1, ΔQL_now > BH2, or ΔSD_now > BH3, then obtain the state data difference range of each row in the first column of the instruction correction mapping table to obtain the state data difference range list HE = (HE1, HE2, ..., HE...). a HE b ), a = 1, 2, ..., b; where, HE a The range of state data differences in row a of the first column of the instruction correction mapping table, where b is the number of rows in the instruction correction mapping table; HE a =(HE a，1 HE a，2 HE a，3 HE a，1 HE represents the range of positional data differences within the range of state data differences in row a. a，2 HE represents the range of attitude data differences within the range of state data differences in row a. a，3 The range of speed data differences within the range of state data differences in row a.

[0076] In this embodiment, a pre-defined command correction mapping table is provided. This command correction mapping table includes several rows, each row corresponding to a state data difference range. Each state data difference range includes a position data difference range, an attitude data difference range, and a velocity data difference range; for example, HE a =(HE a，1 HE a，2 HE a，3 HE a，1 = [1, 2], meaning the location data difference ranges from 1 to 2 meters; HE a，2 = [0.1, 0.2], meaning the attitude data difference ranges from 0.1 to 0.2 degrees; HE a，3 = [0.5, 1], meaning the speed data difference range is 0.5m / s-1m / s; each line corresponds to an instruction correction parameter.

[0077] S212, traverse HE, if ΔWZ_now∈HE a，1 ,ΔQL_now∈HE a，2 And ΔSD_now∈HE a，3 Then the instruction correction parameter corresponding to row a in the instruction correction mapping table will be determined as the instruction correction parameter corresponding to ΔMC_now.

[0078] Based on ΔMC_now, we can determine which row in the instruction correction mapping table corresponds to ΔMC_now, and then determine the instruction correction parameter corresponding to that row as the instruction correction parameter corresponding to ΔMC_now; the instruction correction parameter includes positive and negative values.

[0079] S400 generates the correction control instruction XZL based on the instruction correction parameters and ZL corresponding to ΔMC_now.

[0080] Furthermore, step S400 includes the following steps:

[0081] S410, obtain the velocity correction parameter VA1 and curvature correction parameter VA2 from the instruction correction parameters corresponding to ΔMC_now.

[0082] S420, based on VA1, VA2 and ZL, determine XZL = (VA'1 + VA1, VA'2 + VA2); where VA'1 is the expected speed corresponding to the actual vehicle included in ZL, and VA'2 is the expected curvature corresponding to the actual vehicle included in ZL.

[0083] In this embodiment, the desired speed of the actual vehicle corresponds to the throttle opening or electric switch opening of the actual vehicle. Similarly, the desired speed contained in the modified control command can be converted into the throttle opening or electric switch opening of the digital twin vehicle. Those skilled in the art can use existing speed and throttle opening or electric switch opening conversion methods according to actual needs, which will not be elaborated here.

[0084] The S500 sends the XZL to the digital twin vehicle control module to control the driving of the digital twin vehicle.

[0085] The registration method for supporting parallel testing of a real vehicle and its digital twin in this embodiment involves acquiring the current state data MA_now of the digital twin and the current state data MB_now of the real vehicle after receiving the control command from the real vehicle. Based on MA_now and MB_now, the difference in current state data between the digital twin and the real vehicle, ΔMC_now, is determined. According to ΔMC_now and a preset command correction mapping table, the command correction parameter corresponding to ΔMC_now is determined. Based on the command correction parameter corresponding to ΔMC_now and ZL, a correction control command XZL is generated and sent to the digital twin control module to control the digital twin's movement. Thus, for each received real vehicle control command, the control command is corrected based on the state data of the digital twin and the real vehicle to obtain a correction control command that reduces the state difference between the digital twin and its corresponding real vehicle, thereby reducing the state difference between the digital twin and the real vehicle.

[0086] In one embodiment, a registration system is also provided to support parallel testing of a real vehicle and its digital twin vehicle. The system includes a registration module and a digital twin vehicle; wherein the registration module is communicatively connected to both the digital twin vehicle and the real vehicle.

[0087] The actual vehicle includes: an actual vehicle perception module, an actual vehicle control module, and an actual vehicle autonomous decision-making module; the actual vehicle perception module and the actual vehicle autonomous decision-making module are communicatively connected, and the actual vehicle autonomous decision-making module and the actual vehicle control module are communicatively connected; the actual vehicle perception module is used to collect the actual vehicle's status data and physical environment information, and the actual vehicle autonomous decision-making module generates actual vehicle control commands based on the actual vehicle's status data and physical environment information, and sends the actual vehicle control commands to the registration module.

[0088] The digital twin vehicle includes a digital twin vehicle status acquisition module and a digital twin vehicle control module. The digital twin vehicle status acquisition module is used to collect the status data of the digital twin vehicle and send the status data of the digital twin vehicle to the registration module.

[0089] The registration module is used to perform the following steps:

[0090] S100, in response to receiving the real vehicle control command ZL, acquires the current status data MA_now of the digital twin vehicle and the current status data MB_now of the real vehicle.

[0091] S200, based on MA_now and MB_now, determine the difference in current state data between the digital twin vehicle and the real vehicle, ΔMC_now=|MA_now-MB_now|.

[0092] S300, determine the instruction correction parameters corresponding to ΔMC_now based on ΔMC_now and the preset instruction correction mapping table; wherein, the instruction correction mapping table includes several rows and several columns, each row corresponds to a state data difference range, and each column corresponds to the instruction correction parameters corresponding to the state data difference range.

[0093] S400 generates the correction control instruction XZL based on the instruction correction parameters and ZL corresponding to ΔMC_now.

[0094] The S500 sends the XZL to the digital twin vehicle control module to control the driving of the digital twin vehicle.

[0095] The registration method for supporting parallel testing of a real vehicle and its digital twin in this embodiment involves acquiring the current state data MA_now of the digital twin and the current state data MB_now of the real vehicle after receiving the control command from the real vehicle. Based on MA_now and MB_now, the difference in current state data between the digital twin and the real vehicle, ΔMC_now, is determined. According to ΔMC_now and a preset command correction mapping table, the command correction parameters corresponding to ΔMC_now are determined. Based on the command correction parameters corresponding to ΔMC_now and ZL, a correction control command XZL is generated and sent to the digital twin control module to control the digital twin's movement. Thus, for each received real vehicle control command, the control command is corrected based on the state data of both the digital twin and the real vehicle to obtain a correction control command that better matches the current state of the digital twin, thereby reducing the state difference between the digital twin and the real vehicle.

[0096] In this embodiment, digital twin technology is used to achieve synchronous operation of the real vehicle and the virtual system digital twin vehicle, thereby evaluating the performance of the real vehicle and development testing in a virtual testing and simulation environment, reducing the need for physical environment testing and improving safety and efficiency.

[0097] In an exemplary embodiment, to reduce hardware investment and testing complexity during real-vehicle testing, this embodiment provides a parallel testing method supporting real vehicles, digital twin vehicles, and virtual vehicles. The method is applied to a parallel testing system supporting real vehicles, digital twin vehicles, and virtual vehicles. The system includes a virtual simulation environment, which includes a digital twin vehicle and several virtual vehicles. The digital twin vehicle has a communication connection with its corresponding real vehicle, and the digital twin vehicle drives according to the real vehicle control commands issued by its corresponding real vehicle.

[0098] Each virtual vehicle has a pre-set autonomous decision-making hardware communication connection; each virtual vehicle has a perception module that can perceive virtual environment information and send the virtual environment information to the corresponding autonomous decision-making hardware. The autonomous decision-making hardware generates virtual vehicle control commands based on the virtual environment information and sends the virtual vehicle control commands to the corresponding virtual vehicle, so that the virtual vehicle drives according to the virtual vehicle control commands; the digital twin vehicle and each virtual vehicle can simultaneously perceive virtual obstacles in the virtual environment.

[0099] In this embodiment, the digital twin vehicle and all virtual vehicles can simultaneously acquire obstacle information in the virtual environment. At the same time, the digital twin vehicle drives according to the control commands of the real vehicle. When the real vehicle senses a real obstacle in the physical environment, it will perform an obstacle avoidance maneuver, and the digital twin vehicle will follow the real vehicle and perform the same obstacle avoidance maneuver. It can be understood that the driving trajectory of the real vehicle and the driving trajectory of the digital twin vehicle are the same.

[0100] The method includes the following steps:

[0101] A100 places the real vehicle in a physical environment and the digital twin vehicle and each virtual vehicle in a virtual environment; wherein the virtual environment is simulated based on the physical environment; the starting point of the real vehicle in the physical environment is the same as the starting point of its corresponding digital twin vehicle in the virtual environment.

[0102] A200 controls the real vehicle to autonomously execute preset test content in a physical environment, and controls the virtual vehicle to autonomously execute the same preset test content as the real vehicle in a virtual environment; the digital twin vehicle conducts parallel tests synchronously with the real vehicle in the virtual test environment, and the real vehicle can acquire the virtual environment information perceived by the digital twin vehicle.

[0103] Furthermore, the preset test content includes the following steps:

[0104] A210 sets the starting and ending points of the actual vehicle's journey in the physical environment, as well as the corresponding starting and ending points in the virtual environment.

[0105] In this embodiment, a start point and an end point can be set in the test area corresponding to the actual vehicle.

[0106] The A220 places the actual vehicle at the starting point in the physical environment, enabling the vehicle to autonomously drive from that starting point to the destination.

[0107] For the actual vehicle, it can autonomously perceive real environmental information, and then receive corresponding vehicle control commands based on the environmental information to drive autonomously.

[0108] A230 places any virtual car at the starting point in the virtual environment, allowing the virtual car to autonomously drive from the starting point to the destination.

[0109] For virtual vehicles, their communication connection includes autonomous decision-making hardware, which is equivalent to the autonomous decision-making module of a real vehicle. It can generate virtual vehicle control commands based on the virtual environment information perceived by the virtual vehicle, indicating that the virtual vehicle can drive autonomously in the virtual environment. It should be noted that when the virtual vehicle perceives virtual obstacles, it can also autonomously perform obstacle avoidance actions.

[0110] A240 identifies abnormal areas in the virtual environment based on the driving trajectory of the digital twin vehicle and the virtual vehicle, and optimizes the data in the abnormal areas.

[0111] Specifically, step A400 includes the following steps:

[0112] A241, obtain the driving trajectory SE of the digital twin vehicle in the virtual environment, and the driving trajectory FT of the virtual vehicle in the virtual environment; wherein, the starting point of SE is the same as the starting point of FT, and the ending point of SE is the same as the ending point of FT; the digital twin vehicle drives according to the control command ZL of the corresponding real vehicle, and the virtual vehicle drives autonomously according to the perceived virtual environment information.

[0113] In this embodiment, the virtual environment information is obtained based on the physical environment. It can be understood that the digital twin vehicle and the corresponding real vehicle maintain a twin relationship in parallel testing. Ideally, the driving state of the digital twin vehicle should be synchronized with that of the real vehicle. A starting point and an ending point are set in the physical environment. The real vehicle can autonomously drive from the starting point to the ending point, and the digital twin vehicle will also follow the real vehicle from the starting point in the virtual environment to the ending point. The starting point in the virtual environment corresponds to the starting point in the physical environment, and the ending point in the virtual environment also corresponds to the ending point in the physical environment.

[0114] Furthermore, the virtual vehicle communication connection includes autonomous decision-making hardware, which is used to generate virtual vehicle control commands based on the virtual environment information perceived by the corresponding virtual vehicle, and send the virtual vehicle control commands to the corresponding virtual vehicle; that is, the virtual vehicle can perceive virtual environment information and drive autonomously based on the virtual environment information.

[0115] Furthermore, after step A241 and before step A242, the method further includes the following steps:

[0116] A411, obtain the similarity XSD between SE and FT.

[0117] It should be noted that those skilled in the art can use existing graphic similarity determination methods to obtain the similarity XSD between SE and FT according to actual needs, which will not be elaborated here.

[0118] A412, if XSD < DR, proceed to step S200; otherwise, determine that SE and FT are the same; DR is the preset first similarity threshold.

[0119] In this embodiment, the SE and FT are first determined as a whole. If the SE and FT are the same, it means that the physical environment has not changed. Therefore, there is no need to perform subsequent steps, thereby saving computing power and improving operating efficiency.

[0120] A242, divide SE from the starting point to the ending point into several sub-trajectories to obtain a list of digital twin car sub-trajectories SA = (SA1, SA2, ..., SA2). c SA d), c = 1, 2, ..., d; and divide the FT from the starting point to the ending point into several sub-trajectories to obtain the virtual car sub-trajectory list FA = (FA1, FA2, ..., FA... c , ...,FA d ); where SA c Let d be the c-th sub-trajectory obtained by dividing SE, and d be the number of sub-trajectories obtained by dividing SE; FA c SA is the c-th sub-trajectory obtained by dividing the FT; c Corresponding driving area and FA c The corresponding driving areas are the same.

[0121] In this embodiment, there are several uncertain factors in the physical environment that may cause it to change. The virtual environment information is obtained by simulating the physical environment and will not change unless it is updated.

[0122] Furthermore, step A242 includes the following steps:

[0123] A421, obtain the rectangle XH corresponding to the start and end points of SE; where the start and end points of SE are a pair of diagonal vertices of XH.

[0124] A422, divide XH into d equal sub-rectangles to obtain a sub-rectangle list ZXH = (ZXH1, ZXH2, ..., ZXH...). c ..., ZXH d ); among them, ZXH c Let XH be the c-th sub-rectangle obtained by dividing XH equally.

[0125] A423, SE in ZXH c Part of the driving trajectory was determined to be SA c And FT in ZXH c Part of the driving trajectory was determined to be FA c To obtain SA and FA.

[0126] In this embodiment, the change in physical environment will not be a change in the environment of the entire driving area, but may be a change in a certain area; for example, a new obstacle is set in a certain area, and the information of the obstacle is not simulated in the virtual environment; through the above steps, SE and FT are divided into several sub-trajectories, and each sub-trajectory can be analyzed one by one.

[0127] A243, based on SE and FT, obtain the similarity between the trajectory of each digital twin car and the corresponding virtual car trajectory to obtain a similarity list η = (η1, η2, ..., η3). c , ..., η d ); where η c for SAc with FA c The similarity between them.

[0128] It should be noted that those skilled in the art can use existing graphic similarity determination methods to determine the similarity between the trajectory of each digital twin car and the corresponding virtual car trajectory, as needed, which will not be elaborated here.

[0129] A244, iterate through η, if η c <HL, then η c The target similarity is determined to obtain the target similarity list η' = (η'1, η'2, ..., η'). p ,…,η' q ), p = 1, 2, ..., q; where η' p Let q be the p-th target similarity determined from η, q be the number of target similarities determined from η, and HL be the preset first similarity threshold.

[0130] If η c <HL indicates SA c with FA c There are significant differences, η c The similarity is determined as the target for further judgment.

[0131] Furthermore, DR > HL; for example, DR = 0.95, HL = 0.9; because for the entire large area, small environmental changes in the physical environment need to be reflected by a higher threshold; while for the area corresponding to a smaller sub-trajectory, even small changes in the physical environment will have a significant impact on the similarity. Therefore, setting DR > HL can improve the accuracy of the judgment.

[0132] A245, based on η', obtain the virtual car trajectory corresponding to each target similarity to obtain the target virtual car trajectory list FN = (FN1, FN2, ..., FN...). p , ..., FN q ); where FN p For η' p The corresponding virtual vehicle trajectory.

[0133] As another implementation method, the digital twin vehicle trajectory corresponding to each target similarity can also be obtained based on η', which can also achieve the same effect.

[0134] A246, traverse FN, merge adjacent driving regions corresponding to the driving regions of each virtual car trajectory in FN, and identify them as abnormal regions.

[0135] Furthermore, step A246 includes the following steps:

[0136] A461, obtain the driving area corresponding to each virtual car sub-trajectory in FN to obtain the target virtual car driving sub-region list XNC = (XNC1, XNC2, ..., XNC...). p ..., XNC q ); among which, XNC p For FN p The corresponding driving sub-region.

[0137] A462 merges adjacent driving sub-regions in XNC into a single merged region, resulting in several merged regions.

[0138] A463 identifies each merged region as an abnormal region.

[0139] For example, determine whether XNC1 and XNC2 are adjacent. If they are adjacent, merge XNC1 and XNC2 into the first merged region. Then determine whether XNC2 and XNC3 are adjacent. If they are adjacent, merge the first merged region with XNC3 into the second merged region. If XNC1 and XNC2 are not adjacent, then XNC1 is merged as a separate region. By merging them one by one in this way, several abnormal regions can be obtained.

[0140] It is understandable that the environmental information corresponding to each abnormal area is different from the environmental information of the corresponding physical area, and the physical environmental information has changed.

[0141] A247 updates the data for each abnormal region based on the physical region corresponding to that region in the physical environment.

[0142] Furthermore, step S247 includes the following steps:

[0143] A471, obtain the data of the physical region corresponding to each abnormal region in the physical environment to obtain the physical region data list SJQ = (SJQ1, SJQ2, ..., SJQ...). g , ..., SJQ h ); among them, SJQ g represents the physical region corresponding to the g-th abnormal region in the physical environment, and h represents the number of abnormal regions.

[0144] A472, replace the virtual environment information corresponding to the g-th abnormal region with SJQ. g .

[0145] In this embodiment, the data for the physical area includes obstacle information, road surface information, etc.

[0146] This embodiment supports a parallel testing method for real vehicles, digital twin vehicles, and virtual vehicles. It obtains the driving trajectory SE of the digital twin vehicle in a virtual environment and the driving trajectory FT of the virtual vehicle in the same virtual environment. SE and FT are divided into several sub-trajectories, and the similarity between each sub-trajectory corresponding to SE and the corresponding sub-trajectory in FT is obtained. Virtual vehicle sub-trajectories with similarity scores below a preset first similarity threshold are identified as target virtual vehicle sub-trajectories. Then, adjacent driving regions within the driving region corresponding to each virtual vehicle sub-trajectory are merged and identified as abnormal regions. Based on the actual region corresponding to each abnormal region in the physical environment, data updates are performed on each abnormal region. Thus, when the environment in some areas of the physical environment changes, the corresponding regions in the virtual environment can be updated, ensuring that the states of the digital twin vehicle and the real vehicle remain synchronized.

[0147] In an exemplary embodiment, when obstacles exist in the virtual environment, in order for the digital twin vehicle to maintain state synchronization with the real vehicle when the obstacle is detected, this embodiment provides an enhanced registration method for digital twin vehicle state data. The method is applied to the registration module of a digital twin vehicle test system. The system also includes a virtual simulation environment, which is communicatively connected to the registration module. The registration module is communicatively connected to a preset real vehicle. The virtual simulation environment includes a digital twin vehicle, a digital twin vehicle state acquisition module, and a digital twin vehicle control module. The digital twin vehicle state acquisition module is used to acquire the state data of the digital twin vehicle and send the state data of the digital twin vehicle to the registration module.

[0148] The actual vehicle includes an actual vehicle perception module and an actual vehicle autonomous decision-making module; the actual vehicle perception module and the actual vehicle autonomous decision-making module are communicatively connected; the actual vehicle perception module is used to perceive physical environment information and send the physical environment information to the registration module; the actual vehicle autonomous decision-making module is used to generate actual vehicle control commands based on the physical environment information and the actual vehicle's status data, so as to control the actual vehicle's driving through the actual vehicle control commands.

[0149] In this embodiment, the actual vehicle is a tracked special unmanned vehicle, and the environment in which the actual vehicle travels is an unstructured outdoor environment. The actual vehicle has an environmental information perception module, which includes sensors such as RGB cameras and LiDAR. The perception module can perceive the environmental information around the actual vehicle. The actual vehicle autonomous decision-making module can autonomously generate actual vehicle control commands based on the environmental information and the actual vehicle's state data. Among them, the actual vehicle control commands include the desired speed and desired curvature for controlling the actual vehicle. The curvature can be understood as the curvature of the actual vehicle's travel path. At the same time, the actual vehicle also has an actual vehicle control module for controlling the actual vehicle's travel according to the actual vehicle control commands.

[0150] The registration module is used to generate correction control commands based on the real vehicle status data and the twin vehicle status data, and send the twin vehicle control commands or the real vehicle control commands to the digital twin vehicle control module so as to control the digital twin vehicle to drive.

[0151] In this embodiment, it should be noted that the digital twin vehicle does not have an autonomous decision-making module. The digital twin vehicle drives according to the correction control commands sent by the registration module. The registration module can acquire the real vehicle control commands sent by the real vehicle, the status data of the digital twin vehicle, and the status data of the real vehicle in real time. It can also correct the real vehicle control commands based on the status data of the digital twin vehicle and the status data of the real vehicle to generate correction control commands for controlling the digital twin vehicle. It can be understood that the digital twin vehicle and the corresponding real vehicle are synchronized.

[0152] The method includes the following steps:

[0153] B100, obtain the distance LA between the digital twin vehicle and the target obstacle; wherein, the target obstacle is the virtual obstacle in the virtual environment of the virtual simulation environment that is located in front of the digital twin vehicle and is the closest to it; the real vehicle obtains information about the target obstacle in real time.

[0154] In this embodiment, the digital twin vehicle is located in a virtual simulation environment. The virtual simulation environment is a virtual platform that allows virtual obstacles to be set within the virtual environment. Simultaneously, information about these virtual obstacles, such as their location and size, can be sent to the autonomous decision-making module of the real vehicle, enabling the real vehicle to perceive the presence of obstacles ahead. It is understood that in the actual environment, there are no obstacles in front of the real vehicle; the information is merely based on virtual obstacles from the virtual environment. In the virtual environment, the positions of the digital twin vehicle and the virtual obstacles can be obtained in real time; therefore, the distance LA between the digital twin vehicle and the target obstacle can be determined.

[0155] B200, if LA≤QR+ΔL, then the current first data processing frequency FQ of the registration module is adjusted to the preset second data processing frequency F_max; where F_max>FQ; QR is the maximum detection distance of the distance sensor of the actual vehicle, and ΔL is the distance threshold, so that the registration module generates correction control commands at the frequency of F_max.

[0156] In this embodiment, a distance sensor is installed on the actual vehicle to detect the distance between the actual vehicle and obstacles. This distance sensor has a maximum detection range, for example, a maximum detection range of 100, which can detect obstacles within 100 meters. After the actual vehicle detects an obstacle, the actual vehicle's autonomous decision-making module will autonomously take obstacle avoidance actions. Ideally, the digital twin vehicle should also take obstacle avoidance actions synchronously with the actual vehicle. However, due to the many uncertainties in the physical environment, such as wind resistance and sudden changes in ground friction, the state of the digital twin vehicle may differ from that of the actual vehicle. Therefore, when LA≤QR+ΔL, the current first data processing frequency FQ of the registration module is adjusted to the preset second data processing frequency F_max. That is, the frequency at which the registration module generates correction control commands is increased, so that the digital twin vehicle receives more correction control commands within the distance ΔL. Thus, when the distance between the digital twin vehicle and the target obstacle is QR, the state of the actual vehicle remains synchronized. At this time, the actual vehicle also senses the target obstacle, and both can take avoidance actions at the same time.

[0157] Furthermore, F_max is determined through the following steps:

[0158] B210, acquire the maximum data acquisition frequency F1 of the real vehicle perception module, the maximum frequency F2 of the real vehicle autonomous decision-making module generating real vehicle control commands, the maximum communication frequency F3 between the real vehicle autonomous decision-making module and the registration module, and the maximum calculation frequency F4 of the registration module.

[0159] B211, based on F1, F2 and F3, determine the intermediate frequency FU = MIN(F1, F2, F3); where MIN() is a preset function for finding the minimum value.

[0160] B212, if F4 > FU, then determine F_max = FU; otherwise, determine F_max = F4.

[0161] In this embodiment, if F4 > FU, then F_max = FU is determined; otherwise, F_max = F4 is determined; thus, F_max can be determined; thereby, it is possible to avoid the data transmission failure caused by setting F_max too large or the synchronization effect being poor when F_max is set too small.

[0162] Furthermore, the state data of the digital twin vehicle includes the current speed V1 of the digital twin vehicle, and the state data of the real vehicle includes the current speed V2 of the real vehicle and the distance LB between the real vehicle and the virtual point, where the virtual point is the point in the real scene corresponding to the virtual obstacle; ΔL is determined through the following steps:

[0163] B220, based on V1 and V2, determine the speed difference ΔV = |V1 - V2| between the digital twin vehicle and the real vehicle.

[0164] B221, based on LA and LB, determine the distance difference LC between the digital twin vehicle and the real vehicle = |LA-LB|.

[0165] B222, based on ΔV, determine the speed weight α = |ΔV - ΔV'| / ΔV', and based on LC, determine the distance weight β = |LC - LC'| / LC'; where ΔV' is the preset standard speed difference and LC' is the preset standard distance difference.

[0166] B223, based on α, β and ΔL', determine ΔL = (1 + α + β) × ΔL'; where ΔL' is the preset standard distance threshold; ΔL' is obtained based on ΔV', LC' and F_max.

[0167] In this embodiment, the virtual environment in which the digital twin vehicle is located is simulated based on the physical environment. The state data of the digital twin vehicle includes the current speed V1 of the digital twin vehicle, and the state data of the real vehicle includes the current speed V2 of the real vehicle and the distance LB between the real vehicle and the virtual point. Therefore, ΔV and LC can be obtained, and the speed weight α and distance weight β can be determined. It should be noted that ΔV' is a preset standard speed difference, and LC' is a preset standard distance difference. When the data processing frequency of the registration module is F_max, ΔL' can be determined through multiple trials. Thus, ΔL = (1 + α + β) × ΔL' is obtained. That is, the larger the speed difference between the digital twin vehicle and the real vehicle and / or the larger the distance difference between the digital twin vehicle and the real vehicle, the larger ΔL is, and the more corrections are required.

[0168] B300, if LA≤QR, then adjust the current second data processing frequency F_max of the registration module to the first data processing frequency FQ, so that the registration module generates correction control commands at the frequency of FQ.

[0169] When LA≤QR, the current second data processing frequency F_max of the registration module is adjusted to the first data processing frequency FQ. Since the state between the digital twin vehicle and the real vehicle has been synchronized before the distance to the target obstacle is QR, reducing the data processing frequency of the registration module can reduce the overall computing power consumption of the system and improve the operating efficiency.

[0170] Furthermore, the registration module is connected to a preset real vehicle via wireless communication.

[0171] Furthermore, the actual vehicle status data includes the vehicle's position data, attitude data, and speed data; the attitude data includes the vehicle's tilt angle and frontal orientation.

[0172] Furthermore, the vehicle control commands include the desired speed and the desired curvature.

[0173] Furthermore, the corrective control commands include the speed and curvature corresponding to the digital twin vehicle.

[0174] The enhanced registration method for digital twin vehicle state data in this embodiment is used in the registration module of the digital twin vehicle test system. The registration module acquires the distance LA between the digital twin vehicle and the target obstacle in real time. When LA≤QR+ΔL, it indicates that the distance between the digital twin vehicle and the target obstacle is small, and the real vehicle can also acquire the information of the target obstacle in real time. The digital twin vehicle maintains a relatively synchronized state with the real vehicle. Therefore, the real vehicle can also perceive that the distance to the target obstacle is close at this time. When the distance between the real vehicle and the virtual obstacle corresponding to the target obstacle information is less than QR, the real vehicle will autonomously take evasive action. During this period, it is necessary to maintain... The digital twin vehicle and the real vehicle are kept in sync. Therefore, in this invention, if LA≤QR+ΔL, the current first data processing frequency FQ of the registration module is adjusted to the preset second data processing frequency F_max. That is, the frequency at which the registration module generates correction control commands is increased, so that the registration module can send more correction control commands within a distance of ΔL, thereby increasing the number of state corrections of the digital twin vehicle. This ensures that the state of the digital twin vehicle and the real vehicle is synchronized before the distance to the target obstacle is QR, thereby ensuring that the digital twin vehicle and the real vehicle can make evasive actions at the same time and improve the accuracy of the test data.

[0175] Furthermore, when LA≤QR, the current second data processing frequency F_max of the registration module is adjusted to the first data processing frequency FQ. Since the state between the digital twin vehicle and the real vehicle has been synchronized before the distance to the target obstacle is QR, reducing the data processing frequency of the registration module can reduce the overall computing power consumption of the system and improve the operating efficiency.

[0176] In one exemplary embodiment, the real vehicle generates control commands based on several sensed state data, which requires a long time period. If the digital twin vehicle relies solely on the control commands generated by the real vehicle to control its driving, the driving state of the digital twin vehicle differs significantly from that of the real vehicle due to the low correction frequency of the digital twin vehicle. Therefore, this embodiment provides a digital twin vehicle registration method based on control command correction, which includes the following steps:

[0177] C100 acquires the control command ZL1 of the real vehicle, the current real vehicle status data WQ1, and the digital twin vehicle status data WQ2; where WQ1 includes several different types of driving parameters corresponding to the real vehicle, and WQ2 includes several different types of driving parameters corresponding to the digital twin vehicle; T1 > T2, where T1 is the generation period of ZL1, and T2 is the sampling period of the real vehicle status data and the digital twin vehicle status data; ZL1 is obtained based on WQ1; one real vehicle corresponds to one digital twin vehicle.

[0178] In this embodiment, the actual vehicle control commands are generated based on environmental information and actual vehicle status data, which takes a certain amount of time to generate. However, the actual vehicle status data and digital twin vehicle status data are directly collected without complex data processing. Therefore, the sampling period of the actual vehicle status data and digital twin vehicle status data is shorter than the generation period of the actual vehicle control commands. For example, T2 = 100ms, T1 = 10ms, where T1 is the sampling period of the actual vehicle status data and digital twin vehicle status data, and T2 is the generation period of the actual vehicle control commands.

[0179] WQ1 includes the position, attitude, and speed data of the actual vehicle; the attitude data of the actual vehicle includes the corresponding tilt angle and frontal orientation; WQ2 includes the position, attitude, and speed data of the digital twin vehicle; the attitude data of the digital twin vehicle includes the corresponding tilt angle and frontal orientation.

[0180] C200 modifies ZL1 based on WQ1 and WQ2 to obtain the control command ZL'1 for the digital twin vehicle; ZL'1 is used to control the digital twin vehicle to maintain the state synchronization with the real vehicle corresponding to ZL'1.

[0181] In this embodiment, ZL1 includes the desired speed and desired curvature of the real vehicle, that is, the speed and curvature that the real vehicle hopes to achieve in the next moment. However, there is a difference between the current state data of the digital twin vehicle and the current state data of the real vehicle. Therefore, in order to make the digital twin vehicle and the real vehicle synchronize their states in the next moment, ZL1 needs to be corrected. For example, if the current speed of the real vehicle is 10 km / h and the current speed of the digital twin vehicle is 8 km / h, then the control commands corresponding to the desired speed in ZL1, such as the throttle or electric throttle opening, need to be adjusted so that the digital twin vehicle can be controlled with a larger acceleration.

[0182] C300, retrieve each synchronization time point within the target period T1_now to obtain a list of synchronization time points TH = (TH1, TH2, ..., TH...). i , ...,TH n ), i = 1, 2, ..., n; where TH iLet be the i-th synchronization time point within T1_now, and n be the number of synchronization time points within T1_now; the synchronization time point is the end time point of each sampling period of the real vehicle status data and the twin vehicle status data; the target time period is the time interval between the time point corresponding to ZL'1 and the time point corresponding to SL'1, and SL'1 is the next adjacent digital twin vehicle control command of ZL'1.

[0183] In this embodiment, since T1 < T2, T1 can contain several T2s. By setting the end time of each T2 as the synchronization time point, a synchronization time point list TH can be obtained.

[0184] C400 determines the difference ΔZL1 between the current actual state data of the digital twin vehicle and the actual vehicle based on the current actual state data of the real vehicle and the digital twin vehicle.

[0185] In this embodiment, the current state data of the real vehicle may include the current position, speed, and attitude of the real vehicle, and the current state data of the digital twin vehicle may include the current position, speed, and attitude of the digital twin vehicle. Specifically, ΔZL1 can be determined through the following steps:

[0186] C410 acquires the current actual state data LQ of the real vehicle and the current actual state data LW of the digital twin vehicle; the actual state data includes speed parameters.

[0187] C420, based on LQ and LW, determine ΔZL1=|LQ-LW|.

[0188] C500, based on TH, divide ΔZL1 into n sub-expected state data differences to obtain the sub-expected state data difference list ZH = (ΔZL1 / n) corresponding to TH. 1，1 ΔZL 1，2 , …, ΔZL 1，i , …, ΔZL 1，n ); where ΔZL 1，i For TH i The corresponding sub-expected state data difference.

[0189] Furthermore, ΔZL 1，i This can be determined through the following steps:

[0190] C510, based on n and ΔZL1, determine the average parameter difference CA = ΔZL1 / n.

[0191] C520, determine ΔZL based on CA and ΔZL1. 1，i =ΔZL1-n×CA.

[0192] C600, obtain the target value N=1.

[0193] C700, using ZL' N Control the digital twin vehicle; and according to QU N and QR N , determined in TH N The difference in actual state data between the digital twin vehicle and the real vehicle is ΔWE N Among them, QU N For TH N Corresponding real-vehicle status data, QR N For TH N The corresponding digital twin vehicle status data.

[0194] In this embodiment, the state data difference can be a vector containing multiple state data differences.

[0195] C800, if |ΔZL 1，N -ΔWE N |>GH, then according to ΔWE N For ZL' N Make corrections; otherwise, do not correct ZL' N Make corrections; to obtain TH N The corresponding correction control command ZL' N+1 GH is the preset threshold for the difference in state data.

[0196] GH can also be a vector containing thresholds for the differences between multiple state data, thus making ΔWE N It can be compared with GH; specifically, ΔWE can be used for comparison. N Each state data in the GH is compared with the corresponding state data difference threshold in the GH.

[0197] C900, using ZL' N+1 Control the digital twin vehicle; if N < n, then obtain N = N + 1 and proceed to step C700.

[0198] In this embodiment, through the above steps, after the digital twin vehicle receives the correction control command, the current actual state data difference ΔZL1 between the digital twin vehicle and the real vehicle is evenly divided into each synchronization time point, so that the state data difference between the digital twin vehicle and the real vehicle gradually decreases, and when the next correction control command is received, the synchronization state is achieved.

[0199] In this embodiment, the digital twin vehicle registration method based on control command correction uses a control command generation cycle of the real vehicle that is longer than the sampling cycle of the real vehicle state data and the digital twin vehicle state data. Within the time interval between acquiring the current real vehicle control command and the next real vehicle control command, several synchronization time points are set. The difference ΔZL1 between the current actual state data of the digital twin vehicle and the real vehicle is divided into each synchronization time point according to the number of synchronization time points. Upon reaching each synchronization time point, the current real vehicle control command is corrected once based on the corresponding digital twin vehicle state data and real vehicle state data. This allows for multiple corrections of the current digital twin vehicle control command within the time interval between the current and next real vehicle control command, increasing the correction frequency of the digital twin vehicle's control command and ensuring that the driving state of the digital twin vehicle remains synchronized with the driving state of the real vehicle.

[0200] In one exemplary embodiment, the registration module uses virtual perception data perceived by the digital twin vehicle and physical environment information perceived by the real vehicle to register the control commands of the real vehicle, generating corrective control commands, and the digital twin vehicle drives according to the corrective control commands. However, the registration module may fail in some cases, and once registration fails, it will cause a significant difference in the state between the digital twin vehicle and the real vehicle. Based on this, this embodiment provides a method for parallel testing of a real vehicle and its digital twin vehicle that supports registration failure. The method for parallel testing of a real vehicle and its digital twin vehicle that supports registration failure includes the following steps:

[0201] D100, obtains the distance LA between the digital twin vehicle and the target obstacle; where the target obstacle is the virtual obstacle in the virtual environment that is closest to the digital twin vehicle in front of it; the digital twin vehicle corresponds to a real vehicle, and the real vehicle can generate real vehicle control commands based on the perceived environmental information, and the digital twin vehicle drives according to the real vehicle control commands; one digital twin vehicle corresponds to one real vehicle.

[0202] In this embodiment, the digital twin vehicle is located in a virtual simulation environment. The virtual simulation environment is a software platform that can set virtual obstacles in the virtual environment in which the digital twin vehicle is located. At the same time, it can send information about the virtual obstacles, such as the information of the virtual obstacles perceived by the sensor simulation module of the digital twin vehicle through the simulation sensors, and then the information is processed by the fusion module and sent to the real vehicle. It can be understood that there are no obstacles in front of the real vehicle in the physical environment; only the virtual obstacle information of the virtual environment is used. In the virtual environment, the position of the digital twin vehicle and the position of the virtual obstacles can be obtained in real time. Therefore, the distance LA between the digital twin vehicle and the target obstacle can be determined.

[0203] D200, if QR < LA ≤ QR + ΔL, then the state data of the digital twin vehicle is acquired sequentially at a first preset frequency to obtain a digital twin vehicle state data list MA = (MA1, MA2, ..., MA...). j MA k j = 1, 2, ..., k; and sequentially acquire the vehicle's status data at a first preset frequency to obtain a vehicle status data list MB = (MB1, MB2, ..., MBk). j ... MB k ); where MA j To obtain the j-th state data of the digital twin vehicle, MB j ΔL represents the j-th state data of the actual vehicle; k represents the number of state data of the acquired digital twin vehicle; QR represents the maximum detection distance of the distance sensor of the actual vehicle; and ΔL represents the distance threshold.

[0204] In this embodiment, a distance sensor is installed on the actual vehicle to detect the distance between the actual vehicle and obstacles. This distance sensor has a maximum detection range, for example, a maximum detection range of 100, which can detect obstacles within 100 meters. After the actual vehicle detects an obstacle, the actual vehicle's autonomous decision-making module will autonomously take obstacle avoidance actions. Ideally, the digital twin vehicle should also take obstacle avoidance actions synchronously with the actual vehicle. However, due to the many uncertainties in the physical environment, such as wind resistance and sudden changes in ground friction, the state of the digital twin vehicle may differ from that of the actual vehicle. Therefore, when QR < LA ≤ QR + ΔL, the state data of the digital twin vehicle and the state data of the actual vehicle are acquired sequentially at a first preset frequency. The first preset frequency can be F_max, which is the maximum data processing frequency that the system can provide.

[0205] D300, based on MA and MB, determines the state data difference between the digital twin vehicle and the real vehicle to obtain the state data difference list ΔMC = (ΔMC1, ΔMC2, ..., ΔMC2). j , …, ΔMC k ); where ΔMC j For MA j and MB j The corresponding state data difference; ΔMC j =|MA j -MB j |

[0206] In this embodiment, the real vehicle's state data may include the real vehicle's current position, speed, and attitude, while the digital twin vehicle's state data may include the digital twin vehicle's current position, speed, and attitude; ΔMC j It can be a vector containing multiple state data differences; for example, it can include position data differences, velocity data differences, and attitude data differences.

[0207] D400, based on ΔMC, determine the target difference list ΔMD = (ΔMD1, ΔMD2, ..., ΔMD) corresponding to ΔMC. r , ..., ΔMD k-1 ); where ΔMD r For ΔMC r With ΔMC r+1 The corresponding target difference; ΔMD r =ΔMC r -ΔMC r+1 r = 1, 2, ..., k-1.

[0208] Subtracting the differences between two adjacent state data points yields a target difference list ΔMD, which reflects the changes in the differences between the state data of the digital twin vehicle and the state data of the real vehicle.

[0209] D500, if LA=QR, and ΔMC k If the target difference in ≥BH or ΔMD increases or changes randomly, the registration module is determined to have failed; where BH is a preset state data difference threshold.

[0210] The D600 fuses the virtual perception data corresponding to the digital twin vehicle with the perception data corresponding to the real vehicle to obtain fused data.

[0211] The D700 will send the fused data to the actual vehicle.

[0212] When LA = QR, if ΔMC k ≥BH indicates that the state data of the digital twin vehicle and the real vehicle are not synchronized. If the target difference in ΔMD increases or changes randomly, it means that the virtual environment information perceived by the digital twin vehicle is different from the physical environment information perceived by the real vehicle. In both of these cases, the virtual perception data corresponding to the digital twin vehicle cannot be sent to the real vehicle. Instead, the virtual perception data corresponding to the digital twin vehicle and the perception data corresponding to the real vehicle need to be fused to obtain fused data. The fused data is then sent to the real vehicle so that the real vehicle can perform obstacle avoidance actions based on the fused data, ensuring the accuracy of the obstacle avoidance actions.

[0213] Furthermore, after step D500, the method further includes the following steps:

[0214] D510, if LA=QR, ΔMC k If <BH, and the target difference in ΔMD decreases sequentially, then the virtual perception data corresponding to the digital twin vehicle will be sent to the real vehicle.

[0215] When LA = QR, the digital twin car can already sense virtual obstacles. Ideally, the real car should also be able to sense virtual obstacles at this point. Only then can the digital twin car and the real car perform obstacle avoidance maneuvers simultaneously during subsequent obstacle avoidance tests. Therefore, if the target difference in ΔMD decreases sequentially, it indicates that the state data of the digital twin car and the state data of the real car are gradually synchronizing. ΔMC k <BH indicates that the digital twin vehicle has synchronized its state data with the real vehicle. At this point, it can be determined that the virtual perception data perceived by the digital twin vehicle is the same as the physical environment information perceived by the real vehicle. The virtual perception data corresponding to the digital twin vehicle can then be sent to the real vehicle, enabling the real vehicle to use the virtual perception data of the digital twin vehicle to perform obstacle avoidance actions. This eliminates the need to fuse the virtual perception data and physical environment information, and also eliminates the need to use the real vehicle's own perception module to perceive the physical environment information, thus reducing computing power consumption and improving the efficiency of autonomous decision-making and obstacle avoidance.

[0216] Furthermore, after step D500, the method further includes the following steps:

[0217] D800, if LA < QR, then obtain the current state data MA_now of the digital twin vehicle and the current state data MB_now of the real vehicle.

[0218] D810, based on MA_now and MB_now, determine the difference in current state data between the digital twin vehicle and the real vehicle, ΔMC_now=|MA_now-MB_now|.

[0219] D820, if ΔMC_now>BH, then the virtual perception data corresponding to the digital twin vehicle and the perception data corresponding to the real vehicle will be fused to obtain fused data.

[0220] The D830 sends the fused data to the actual vehicle.

[0221] In this embodiment, when LA < QR, it means that both the real vehicle and the digital twin vehicle can perceive the virtual obstacle. During this period, the current state data MA_now of the digital twin vehicle and the current state data MB_now of the real vehicle need to be acquired in real time. Then, the difference ΔMC_now between the current state data of the digital twin vehicle and the real vehicle is obtained. If ΔMC_now > BH, the virtual perception data corresponding to the digital twin vehicle and the perception data corresponding to the real vehicle are fused to obtain fused data. That is, the data fusion operation is started immediately so that the real vehicle can perform obstacle avoidance actions through the fused data to ensure the accuracy of the obstacle avoidance actions.

[0222] Furthermore, after step D510, the method further includes the following steps:

[0223] D900, if LA=QR, ΔMCk If <BH, and the target difference in ΔMD decreases sequentially, then reduce the data sampling frequency of the actual vehicle.

[0224] At this point, it can be determined that the virtual perception data perceived by the digital twin vehicle and the physical environment information perceived by the real vehicle are the same. The computational power consumption can be reduced by lowering the data sampling frequency of the real vehicle, thereby further improving the execution efficiency of obstacle avoidance actions.

[0225] Furthermore, the actual vehicle includes an autonomous decision-making module, which is used to generate vehicle control commands based on the corresponding status data of the actual vehicle.

[0226] This embodiment supports a parallel testing method for a real vehicle and its digital twin vehicle in cases of registration failure. The distance LA between the digital twin vehicle and the target obstacle is obtained. If QR < LA ≤ QR + ΔL, the state data of the digital twin vehicle and the real vehicle are acquired sequentially at a first preset frequency. Based on the state data of the digital twin vehicle and the real vehicle acquired at each moment, the state data difference between the digital twin vehicle and the real vehicle at that moment is determined, resulting in several state data differences, forming a state data difference list ΔMC. Based on ΔMC, a target difference list ΔMD corresponding to ΔMC is determined. When LA = QR, if ΔMC... k If the target difference in ≥BH or ΔMD increases or changes randomly, the registration module is determined to have failed. At this time, the virtual perception data corresponding to the digital twin vehicle and the perception data corresponding to the real vehicle are fused to obtain fused data. The fused data is then sent to the real vehicle. Thus, when the registration module fails, the real vehicle can make autonomous decisions by combining virtual environment information and physical environment information, so that the digital twin vehicle and the real vehicle maintain state synchronization.

[0227] In an exemplary embodiment, during multi-vehicle joint testing, real obstacles are set in the physical environment and virtual obstacles are set in the virtual environment. In order to enable the virtual vehicle in the virtual environment to simultaneously perceive both real and virtual obstacles, this embodiment provides a non-intrusive testing method for unmanned vehicles that supports virtual-real fusion testing. The method is applied to a non-intrusive testing system for unmanned vehicles that supports virtual-real fusion testing. The system includes: a digital twin vehicle and a real vehicle, which are communicatively connected. The real vehicle includes: a real vehicle perception module, a real vehicle autonomous decision-making module, and an information transceiver module. The real vehicle perception module is communicatively connected to the information transceiver module, and the information transceiver module is communicatively connected to the real vehicle autonomous decision-making module.

[0228] The digital twin vehicle includes a virtual environment perception module and a virtual-real fusion module; the virtual environment perception module is used to perceive virtual environment information and send the virtual environment information to the virtual-real fusion module; the virtual-real fusion module is communicatively connected to the information transceiver module.

[0229] In this embodiment, the virtual environment perception module can perceive virtual obstacles within a preset range around the digital twin vehicle, and add each virtual obstacle to a preset matrix according to the position of each virtual obstacle to obtain the grid information KS of the digital twin vehicle; the grid information GS of the real vehicle is generated in the same way as the grid information KS of the digital twin vehicle, and will not be described in detail here.

[0230] The method includes the following steps:

[0231] E100, the virtual-real fusion module acquires the grille information GS of the real vehicle and the grille information KS of the digital twin vehicle; where GS and KS both include a u-row v-column grid, and each grid includes a first identifier or a second identifier; GS corresponds to a preset area around the real vehicle, and KS corresponds to a preset area around the digital twin vehicle; the area corresponding to GS and the area corresponding to KS have the same area; the first identifier is used to indicate that there are obstacles in the corresponding grid, and the second identifier is used to indicate that there are no obstacles in the corresponding grid.

[0232] In this embodiment, the grid information can be a matrix. The preset range that the vehicle's surrounding perception module can perceive, such as a rectangular range, is divided into several grids to obtain a matrix including u rows and v columns. Each grid corresponds to a region in the physical environment. If there is an obstacle in the region, the identifier in the grid is set as the first identifier; otherwise, it is set as the second identifier.

[0233] E200, obtain the location of the real vehicle WZ1 and the location of the digital twin vehicle WZ2.

[0234] During real-vehicle testing, a test area can be set first. Then, a corresponding Cartesian coordinate system can be set for this test area. The location information of the real vehicle can be obtained through the GPS or BD module preset on the real vehicle, and then the actual location information can be converted into location information in the Cartesian coordinate system. The digital twin vehicle is in a virtual environment, and the Cartesian coordinate system location information can be directly imported into the virtual environment, so that the real vehicle and the digital twin vehicle are in the same Cartesian coordinate system, which is convenient for subsequent coordinate calculations.

[0235] It should be noted that those skilled in the art can use existing coordinate system transformation methods to convert the actual coordinates of the vehicle into Cartesian coordinates according to actual needs, which will not be elaborated here.

[0236] E300, based on WZ1 and WZ2, determines whether the positions of the digital twin vehicle and the real vehicle are the same.

[0237] Specifically, WZ1 = (WZ 1，x WZ 1，y ); WZ2 = (WZ 2，xWZ 2，y ); among them, WZ 1，x Let WZ be the X-axis coordinate of the actual vehicle in the target coordinate system ZB. 1，y This represents the Y-axis coordinates of the actual vehicle in ZB; WZ 2，x WZ represents the X-axis coordinates of the digital twin vehicle in ZB. 2，y The Y-axis coordinate of the digital twin vehicle in ZB is given; the target coordinate system corresponds to the pre-defined test area of ​​the real vehicle.

[0238] Furthermore, step E300 includes the following steps:

[0239] E310, based on WZ1 and WZ2, determine the position difference ΔAU between the real vehicle and the digital twin vehicle = (ΔAU) x ΔAU y ); where ΔAU x ΔAU represents the difference in X-axis coordinates between the real vehicle and the digital twin. y The difference in Y-axis coordinates between the real vehicle and the digital twin vehicle; ΔAU x =|WZ 1，x -WZ 2，x |;ΔAU y =|WZ 1，y -WZ 2，y |

[0240] E320, if PL / 2≤ΔAU x <AU' x And PL / 2≤ΔAU y <AU' y If the digital twin vehicle and the real vehicle are in the same position, then the digital twin vehicle and the real vehicle are in different positions; otherwise, the digital twin vehicle and the real vehicle are in different positions; where AU' x AU' is the preset threshold for the difference between the X-axis coordinates. y PL is the preset Y-axis coordinate difference threshold; PL is the side length of each grid.

[0241] In this embodiment, if PL / 2≤ΔAU x <AU' x And PL / 2≤ΔAU y <AU' y This indicates that the positional deviation between the digital twin and the real vehicle is small, within half the grid side length, thus allowing for approximate positional synchronization between the two.

[0242] For E400, if the digital twin vehicle and the real vehicle are in the same position, the first or second identifier in each grid within the GS is updated according to GS and KS to obtain the target grid information.

[0243] Furthermore, step E400 includes the following steps:

[0244] E410: For any grid in GS, if the grid contains the second identifier and a grid in KS at the same position as the grid contains the second identifier, then the grid is determined to contain the second identifier; otherwise, the grid is set to contain the first identifier.

[0245] In this embodiment, the first identifier is 1 and the second identifier is 0; for example, the grid in the first row and first column of GS is 0, and the grid in the first row and first column of KS is also 0; then it can be determined that there are no actual obstacles in the actual area corresponding to the grid in the first row and first column, and there are no virtual obstacles in the corresponding area of ​​the virtual environment. Therefore, the grid is set to 0; otherwise, the grid is set to 1, indicating that there are actual obstacles in the actual area corresponding to the grid and / or there are virtual obstacles in the corresponding area of ​​the virtual environment.

[0246] Furthermore, after step E400 and before step S500, the method further includes the following steps:

[0247] E420, if ΔAU x >PL / 2, or ΔAU y If the distance is greater than PL / 2, then the distance from the target vertex to each side of the target mesh is obtained, in the distance list KN = (KN1, KN2, KN3, KN4); where KN1 is the distance from the target vertex to the first side of the target mesh, KN2 is the distance from the target vertex to the second side of the target mesh, KN3 is the distance from the target vertex to the third side of the target mesh, and KN4 is the distance from the target vertex to the fourth side of the target mesh; the first and second sides are parallel, and the third and fourth sides are parallel; the target vertex is the vertex in GS among the four vertices of KS, and the target mesh is the mesh in GS that contains the target vertex.

[0248] E430, if KN1≤KN2, then move KS towards the first side by a distance of KN1; otherwise, move KS towards the second side by a distance of KN2.

[0249] E440, if KN3≤KN4, then move KS towards the third side by a distance of KN3; otherwise, move KS towards the fourth side by a distance of KN4, so that some meshes in GS coincide with some meshes in KS.

[0250] E450 updates the first or second identifier in the GS memory within the overlapping grid.

[0251] In this embodiment, there may be a large positional deviation between the digital twin vehicle and the real vehicle. Therefore, it is necessary to make local fine adjustments to KS so that KS coincides with GS, and then update the identifiers in each overlapping grid in GS. This allows some virtual obstacle information in KS to be added to the corresponding grid in GS, making the obstacle information obtained by the real vehicle more accurate.

[0252] Furthermore, step E450 includes the following steps:

[0253] E451, for any overlapping grid in GS, if the grid contains the second identifier and the overlapping grid contains the second identifier, then the grid is determined to contain the second identifier; otherwise, the grid is set to the first identifier.

[0254] The method in this step is the same as that in step E410, and will not be repeated here.

[0255] The E500 sends the target grid information to the vehicle's autonomous decision-making module.

[0256] In the E600, the vehicle autonomous decision-making module generates vehicle control commands based on the target grid information; these vehicle control commands are used to control the vehicle's movement.

[0257] The autonomous decision-making module of the actual vehicle only needs to parse the identification information in each grid of the target grid information to obtain the obstacle information of the area corresponding to each grid; thus greatly improving the efficiency of data transmission and processing.

[0258] This embodiment provides a non-intrusive testing method for unmanned vehicles that supports virtual-real fusion experiments. The virtual-real fusion module acquires the grid information of the real vehicle and the digital twin vehicle. Based on the position coordinates of the real vehicle and the digital twin vehicle, it determines whether the positions of the real vehicle and the digital twin vehicle are the same. If they are the same, the grid information of the real vehicle is updated according to the grid information of the digital twin vehicle, so that the grid information of the real vehicle includes both virtual obstacle information and actual obstacle information. Since the grid information only records the identification information of the obstacles and does not record the complete obstacle information, the transmission efficiency is high when transmitting grid information containing obstacle identification information.

[0259] Furthermore, in this invention, the virtual obstacle information is updated to the corresponding grid information of the real vehicle, so that the real vehicle can simultaneously obtain both the actual obstacle information and the virtual obstacle information.

[0260] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin, characterized in that, The system includes: The system includes a virtual simulation environment, a virtual-real sensor fusion module, and a registration module. The virtual simulation environment contains a digital twin vehicle and a virtual vehicle. The digital twin vehicle is simultaneously connected to both the registration module and the virtual-real sensor fusion module. A pre-set real vehicle is also simultaneously connected to both the registration module and the virtual-real sensor fusion module. Each digital twin vehicle uniquely corresponds to one real vehicle. The virtual-real sensor fusion module is used to acquire virtual environment information perceived by the digital twin vehicle and physical environment information perceived by the real vehicle, and to fuse the virtual environment information perceived by the digital twin vehicle and the physical environment information perceived by the real vehicle to obtain fused data, and send the fused data to the real vehicle; the real vehicle generates real vehicle control commands based on the fused data and real vehicle status data, and sends the real vehicle control commands to the registration module; The digital twin vehicle is communicatively connected to the registration module. The registration module can acquire the status data of the real vehicle and the status data of the digital twin vehicle, and correct the control commands of the real vehicle based on the status data of the real vehicle and the status data of the digital twin vehicle, generate corrected control commands, and send the corrected control commands to the digital twin vehicle, so that the digital twin vehicle keeps its status synchronized with the real vehicle according to the corrected control commands. The registration module is also used to perform the following steps: D100, obtain the distance LA between the digital twin vehicle and the target obstacle; where the target obstacle is the virtual obstacle in the virtual environment that is closest to the digital twin vehicle in front of it; the digital twin vehicle corresponds to a real vehicle, and the real vehicle can generate real vehicle control commands based on the perceived environmental information, and the digital twin vehicle drives according to the real vehicle control commands; one digital twin vehicle corresponds to one real vehicle. D200, if QR < LA ≤ QR + ΔL, then the state data of the digital twin vehicle is acquired sequentially at a first preset frequency to obtain a digital twin vehicle state data list MA = (MA1, MA2, ..., MA...). j MA k ), j=1, 2, ..., k; and sequentially acquire the actual vehicle's state data at a first preset frequency to obtain the actual vehicle state data list MB=(MB1, MB2, ..., MB j ... MB k ); where MA j To obtain the j-th state data of the digital twin vehicle, MB j The j-th state data of the actual vehicle is obtained; k is the number of state data of the digital twin vehicle obtained; QR is the maximum detection distance of the distance sensor of the actual vehicle, and ΔL is the distance threshold. D300, based on MA and MB, determines the state data difference between the digital twin vehicle and the real vehicle to obtain the state data difference list ΔMC = (ΔMC1, ΔMC2, ..., ΔMC2). j , …, ΔMC k ); where ΔMC j For MA j and MB j The corresponding state data difference; ΔMC j =|MA j -MB j |; D400, based on ΔMC, determine the target difference list ΔMD = (ΔMD1, ΔMD2, ..., ΔMD) corresponding to ΔMC. r , ..., ΔMD k-1 ); where ΔMD r For ΔMC r With ΔMC r+1 The corresponding target difference; ΔMD r =ΔMC r -ΔMC r+1 r = 1, 2, ..., k-1; D500, if LA=QR, and ΔMC k If the target difference in ≥BH or ΔMD increases or changes randomly, the registration module is determined to have failed; where BH is a preset state data difference threshold. The D600 fuses the virtual perception data corresponding to the digital twin vehicle with the perception data corresponding to the real vehicle to obtain fused data. The D700 will send the fused data to the actual vehicle; The virtual simulation environment also includes several virtual vehicles, each of which is connected to a preset virtual vehicle autonomous decision-making module. Each virtual vehicle can sense virtual environment information and send the virtual environment information to the corresponding virtual vehicle autonomous decision-making module. The virtual vehicle autonomous decision-making module generates virtual vehicle control commands based on the virtual environment information and sends the virtual vehicle control commands to the corresponding virtual vehicle, so that the virtual vehicle drives according to the virtual vehicle control commands.

2. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The digital twin vehicle includes: The system comprises a digital twin vehicle status acquisition module, a digital twin vehicle virtual sensor module, and a digital twin vehicle controller module; wherein, the digital twin vehicle status acquisition module is used to acquire status data of the digital twin vehicle and send the status data of the digital twin vehicle to the registration module; The digital twin vehicle virtual sensor module is used to sense virtual environment information and send the virtual environment information to the virtual-real sensor fusion module; The digital twin vehicle controller module is used to receive correction control commands sent by the registration module and control the digital twin vehicle to drive according to the correction control commands.

3. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The virtual vehicle includes: The system includes a virtual sensor module, a hardware / software-in-the-loop interface, and a virtual vehicle controller module. The virtual sensor module is used to sense virtual environment information and send the virtual environment information to the virtual vehicle autonomous decision-making module through the hardware / software-in-the-loop interface. The software / hardware-in-the-loop interface is used to receive virtual vehicle control commands generated by the virtual vehicle autonomous decision-making module and send the virtual vehicle control commands to the virtual vehicle control module; The virtual vehicle controller module is used to control the virtual vehicle's movement according to the virtual vehicle control commands.

4. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The actual vehicle mentioned includes: The system includes a real vehicle status data acquisition module, a real vehicle perception module, a real vehicle autonomous decision-making module, and a real vehicle control module. The real vehicle status data acquisition module is used to collect the status data of the real vehicle and send the real vehicle status data to the registration module. The vehicle perception module is used to perceive physical environment information and send the physical environment information to the vehicle autonomous decision-making module; The real vehicle autonomous decision-making module is used to generate real vehicle control commands based on the fused data and the real vehicle status data, and send the real vehicle control commands to the real vehicle control module and the registration module; The vehicle control module is used to control the vehicle's movement according to the vehicle control commands.

5. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The virtual environment information includes virtual obstacle information, and each virtual vehicle and digital twin vehicle can simultaneously acquire virtual obstacle information.

6. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The actual vehicle control commands include the desired speed and desired curvature corresponding to the actual vehicle.

7. The virtual-real fusion testing system comprising a virtual vehicle, a physical vehicle, and their digital twin vehicle as described in claim 1, characterized in that, The real vehicle status data includes the real vehicle's position data, attitude data, and speed data; the real vehicle's attitude data includes the real vehicle's tilt angle and front-facing orientation; the digital twin vehicle status data includes the digital twin vehicle's position data, attitude data, and speed data; the digital twin vehicle's attitude data includes the digital twin vehicle's tilt angle and front-facing orientation.

Images