Networked vehicle safety interaction test and evaluation system and method based on digital twinning

Abstract

The invention provides a networked vehicle safety interaction test and evaluation system based on digital twinning. The system comprises a central control unit module; the digital twin modeling module is used for carrying out modeling on the mapped network connection vehicle, and the digital twin modeling module is in communication signal connection with the central control unit module; the network attack and defense simulation module is in communication connection with the central control unit module and is used for executing vehicle networking attack and defense scenes; and the virtual-real interaction device module is in communication signal connection with the central control unit module and is used for simulating the physical movement of the networked vehicle. According to the invention, the technical problem that part of network connection vehicle safety assessment methods need destructive influence on vehicle hardware on the physical level, so that subsequent continuous assessment is difficult can be solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle safety technology, and in particular to a connected vehicle safety interactive test and evaluation system and method based on digital twins. Background Art

[0002] As an important part of the modern transportation system, Intelligent Connected Vehicle (ICV) integrates sensing technology, network communication technology, artificial intelligence and automatic control technology, and realizes real-time information interaction between vehicles, road infrastructure, cloud platforms, etc. This highly networked and intelligent feature enables ICV to perform functions such as environmental perception, intelligent decision-making, and collaborative control, greatly improving the safety and efficiency of travel.

[0003] However, the highly integrated and interconnected nature of intelligent connected vehicles also brings many cybersecurity risks. Its network attack surface is extensive, including vehicle systems, on-board diagnostic interfaces, cellular network access, vehicle wireless access points, etc., and the attack methods cover multiple types such as remote attacks, physical attacks, and supply chain attacks. Once the intelligent connected vehicle system is attacked, it may lead to serious consequences such as illegal control of the vehicle, leakage of privacy information, and failure of safety functions, which seriously threatens the safety of passengers and pedestrians and personal privacy protection. The existing network security assessment methods for intelligent connected vehicles mainly include risk assessment, penetration testing, threat simulation, physical layer analysis, and compliance auditing. Among them, some assessment methods, such as physical layer analysis assessment methods, may also have a destructive impact on the hardware of intelligent connected vehicles, which is not conducive to subsequent continuous assessment and repair work.

[0004] In view of this, the inventor proposes a connected vehicle safety interactive test and evaluation system and method based on digital twins. Summary of the invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes a connected vehicle safety interactive test and evaluation system based on digital twins to solve the technical problem that some connected vehicle safety assessment methods in the existing technology require destructive impact on the vehicle hardware at the physical level, making subsequent continuous evaluation difficult.

[0006] The technical solution adopted by the present invention is that, in a first aspect, a connected vehicle safety interactive test and evaluation system based on digital twin is provided, comprising: a central control unit module;

[0007] A digital twin modeling module, the digital twin modeling module is used to model the mapped connected vehicle, and the digital twin modeling module is connected to the central control unit module by communication signals;

[0008] A network attack and defense simulation module, which is in communication with the central control unit module and is used to execute vehicle network attack and defense scenarios; and

[0009] A virtual-reality interaction device module, which is connected to the central control unit module via communication signals and is used to simulate the physical movement of the networked vehicle.

[0010] Furthermore, the digital twin modeling module includes a 3D laser scanner, a multi-constellation combined navigation module, a multi-spectral camera array and a vehicle-mounted sensor simulator module. The construction process of the digital twin modeling module for modeling a networked vehicle includes:

[0011] Use 3D laser scanners to scan the connected vehicle and the surrounding environment to obtain the first digital twin model of the connected vehicle and the vehicle environment;

[0012] The multi-constellation combined navigation module is used to measure the spatial information of the networked vehicle to obtain spatial information data, and the spatial information data is integrated into the first geometric three-dimensional model to obtain the second digital twin model;

[0013] A multispectral camera array is used to collect images of the connected vehicle and the surrounding environment, to generate digital material texture data of the connected vehicle and the environmental information, and the digital material texture data of the connected vehicle and the environmental information is mapped to the second digital twin model to obtain a third digital twin model;

[0014] The virtual vehicle-mounted sensing device is added to the connected vehicle digital twin model in the third digital twin model using the vehicle-mounted sensor simulator module to obtain the fourth digital twin model.

[0015] Furthermore, the network attack and defense simulation module includes a network traffic generator, a vulnerability injection module, an attack vector library and a network defense simulator. The network traffic generator is used to form network traffic, the vulnerability injection module is used to implant common vehicle system vulnerabilities into the fourth digital twin model, the attack vector library is used to form attack scenarios and attack chains, and the network defense simulator is used to input network defense mechanisms into the fourth digital twin model to simulate the defense response process of the mapped connected vehicle in an attack scenario.

[0016] Furthermore, the virtual-reality interaction device module includes a virtual demonstration system and an operation feedback system. The virtual demonstration system is used to demonstrate the scene of the vehicle in the virtual space in the fourth digital twin model to the operator, and the operation feedback system is used to feed back the operator's actions to the fourth digital twin model.

[0017] Furthermore, the virtual-reality interaction device module also includes an external display device for displaying the scene in the virtual space to the staff who do not participate in the direct interaction.

[0018] Furthermore, the virtual-reality interaction device module also includes a vehicle motion simulation platform, which is used to simulate the real-time motion of the networked vehicle in the fourth digital twin model.

[0019] Furthermore, it also includes an edge computing unit module, which is connected to the central control unit module by communication signals and is used to process vehicle information of the connected vehicle and upload it to the central control unit after processing.

[0020] Furthermore, it also includes a cloud computing server, which is communicatively connected to the central control unit module, and is used to assist the central control unit module in processing data information of each module.

[0021] Furthermore, the cloud computing server includes a data processing cluster module, a data analysis platform and a machine learning training server. The data processing cluster module is used to process the data uploaded by the central unit module, the data analysis platform is used to analyze the data, and the machine learning training server is used to acquire the data processed by the data processing cluster module and the data analysis platform and optimize the neural network model.

[0022] In the second aspect, a connected vehicle safety interaction test and evaluation method based on digital twin is provided, which is realized by using the connected vehicle safety interaction test and evaluation system based on digital twin provided in the first aspect.

[0023] It can be seen from the above technical solution that the beneficial technical effects of the present invention are as follows:

[0024] The present invention provides a networked vehicle safety interactive test and evaluation system based on digital twins. First, a central control unit module and a digital twin modeling module are set. The mapped networked vehicle and its surrounding environment are digitally modeled through the digital twin modeling module to obtain a digital twin model including the networked vehicle and the surrounding environment of the networked vehicle. A network attack and defense simulation module and a virtual-reality interaction device module are set. A virtual defense mechanism is set for the networked vehicle through the network attack and defense simulation module, and a network traffic source containing attack scenarios is set. The attack scenarios encountered by the networked vehicle in the digital twin model are displayed to staff participating in the evaluation experiment through the virtual-reality interaction device module. At the same time, the staff perform feedback interaction on the digital twin model through the virtual-reality interaction device module, so that the test and evaluation of the mapped networked vehicle can be carried out through the virtual digital twin model, thereby avoiding destructive experiments on the car at the physical level, thereby solving the technical problem that some networked vehicle safety evaluation methods in the prior art need to have a destructive impact on the car hardware at the physical level, which makes subsequent continuous evaluation difficult. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0026] Figure 1 A schematic diagram of the overall structure of a digital twin-based connected vehicle safety interactive test and evaluation system according to Embodiment 1 of the present invention;

[0027] Figure 2 This is a schematic diagram of the construction process flow of a digital twin modeling module for modeling a connected vehicle provided by a digital twin-based connected vehicle safety interactive test and evaluation system in Example 1 of the present invention. DETAILED DESCRIPTION

[0028] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0029] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0030] To solve the technical problem that some of the existing connected vehicle safety assessment methods need to physically affect the vehicle hardware, making subsequent continuous assessment difficult, such as Figure 1 As shown, this embodiment provides a connected vehicle safety interactive test and evaluation system based on digital twins, including: a central control unit module, the central control unit module is used to coordinate the work of various functional units, including a main controller as the core of the entire system, coordinating the work of various units, and also includes a high-performance computing cluster for complex data processing and simulation calculations, and also includes a storage array for storing evaluation data uploaded by each module.

[0031] A digital twin modeling module, the digital twin modeling module is used to model the mapped connected vehicle, the digital twin modeling module is connected to the central control unit module by communication signals, as a preferred solution of this embodiment, Figure 1 to Figure 2As shown, a digital twin modeling module is used to perform digital twin modeling of the connected vehicle and the surrounding environment of the connected vehicle to construct a mapping body of the connected vehicle and the surrounding environment in the virtual space. The digital twin modeling module includes a 3D laser scanner, a multi-constellation combined navigation module, a multi-spectral camera array and an on-board sensor, wherein the 3D laser scanner is used to scan to capture the three-dimensional geometric model of the connected vehicle and its surrounding environment. The multi-constellation combined navigation module in this embodiment is preferably any one or more of the GPS, Beidou or Galileo navigation modules, which are used to obtain the positioning data of the connected vehicle and its environmental model. The multi-spectral camera array is used to perform secondary image acquisition on the mapped connected vehicle and its environment to generate digital material textures of the connected vehicle and the environment. The on-board sensor simulator is used to simulate various sensing devices such as on-board radars, cameras, IMUs, etc. Specifically, this embodiment is preferably, Figure 2 As shown in the figure, the construction process of the digital twin modeling module for modeling the connected vehicle includes the following steps:

[0032] A 3D laser scanner is used to scan the connected vehicle and the surrounding environment to obtain the first digital twin model of the connected vehicle and the vehicle environment. The first digital twin model serves as the model basis for the initial version.

[0033] A multi-constellation combined navigation module is used to measure the spatial information of the connected vehicle to obtain spatial information data, and the spatial information data is integrated into the first geometric three-dimensional model to obtain a second digital twin model. At this time, the digital twin model includes spatial information data, so that the initial version of the model base clearly defines its specific position in the virtual space.

[0034] A multispectral camera array is used to capture images of the connected vehicle and the surrounding environment, generate digital material texture data of the connected vehicle and environmental information, and map the digital material texture data of the connected vehicle and environmental information to the second digital twin model to obtain the third digital twin model. This further upgrades the material of the second digital twin model, allowing staff to obtain a more realistic immersive experience when obtaining images of the digital twin model.

[0035] The vehicle-mounted sensor simulator module is used to add the virtual vehicle-mounted sensing device to the connected vehicle digital twin model in the third digital twin model to obtain the fourth digital twin model, so that the digital twin model of the connected vehicle can obtain data in the virtual environment through the virtual vehicle-mounted sensing device, mapping the real vehicle sensors of the connected vehicle in reality.

[0036] A network attack and defense simulation module is communicatively connected to the central control unit module and is used to execute vehicle network attack and defense scenarios. This embodiment is preferred, such as Figure 1As shown, the network attack and defense simulation module includes a network traffic generator, a vulnerability injection module, an attack vector library and a network defense simulator. The network traffic generator is used to form network traffic, the vulnerability injection module is used to implant common vehicle system vulnerabilities into the fourth digital twin model, the attack vector library is used to form attack scenarios and attack chains, and the network defense simulator is used to input network defense mechanisms into the fourth digital twin model to simulate the defense response process of the mapped connected vehicle in the attack scenario. When in use, common types of system vulnerabilities are implanted into the digital twin model of the connected vehicle through the vulnerability injection module, and the network defense simulator is used to implant the common defense mechanisms of the mapped connected vehicle into the digital twin model of the connected vehicle. After completing the vehicle preparation operation, the attack scenario and attack chain are selected from the attack vector library, and the attack scenario and attack chain are used to form traffic for attacking the digital twin model of the connected vehicle through the network traffic generator. Then the digital twin model of the connected vehicle receives the network attack and defends through its own defense mechanism, thereby realizing a digital twin network attack and defense process. The digital twin model of the network vehicle generates corresponding vehicle reactions in the digital virtual environment of the virtual connected vehicle for easy observation by the operator.

[0037] A virtual-reality interaction device module, which is connected to the central control unit module by communication signals, is used to simulate the physical movement of the networked vehicle. As a preferred solution of this embodiment, Figure 1 As shown, the virtual-reality interaction device module includes a virtual demonstration system and an operation feedback system. The virtual demonstration system is used to demonstrate the scene of the vehicle in the fourth digital twin model in the virtual space to the operator, and the operation feedback system is used to feed back the operator's actions to the fourth digital twin model. In this embodiment, the virtual demonstration system is preferably an AR and / or VR head display device. When the operator conducts experimental evaluation, the head display device is put on the operator to show the operator the virtual scene of the networked vehicle in the virtual space. In this embodiment, the operation feedback system is preferably a tactile feedback glove. The operator can use the tactile feedback glove to manipulate the digital twin model of the networked vehicle when it is attacked by the network, thereby providing real-time feedback.

[0038] Furthermore, the virtual-reality interaction device module also includes an external display device for displaying scenes in the virtual space to staff who do not participate in direct interaction. In this embodiment, the external display device preferably adopts a holographic projection system to display the image of the operator's head display device to the outside, so that staff other than the operator can observe the virtual scene that the operator can observe.

[0039] Furthermore, this embodiment preferably also includes an edge computing unit module and a cloud computing server. The edge computing unit module is connected to the central control unit module by communication signals, and is used to process vehicle information of the connected vehicle and upload it to the central control unit after processing; the cloud computing server includes a data processing cluster module, a data analysis platform and a machine learning training server. The data processing cluster module is used to process the data uploaded by the central unit module, the data analysis platform is used to analyze the data, and the machine learning training server is used to acquire the data processed by the data processing cluster module and the data analysis platform and optimize the neural network model.

[0040] That is, this embodiment provides a connected vehicle safety interactive test and evaluation system based on digital twins. First, a central control unit module and a digital twin modeling module are set. The mapped connected vehicle and its surrounding environment are digitally modeled through the digital twin modeling module to obtain a digital twin model including the connected vehicle and the surrounding environment of the connected vehicle. A network attack and defense simulation module and a virtual-reality interaction device module are set. A virtual defense mechanism is set for the connected vehicle through the network attack and defense simulation module, and a network traffic source containing an attack scenario is set. The attack scenario encountered by the connected vehicle in the digital twin model is displayed to the staff participating in the evaluation experiment through the virtual-reality interaction device module. At the same time, the staff interacts with the digital twin model through the virtual-reality interaction device module, so that the test and evaluation of the mapped connected vehicle can be carried out through the virtual digital twin model, thereby avoiding destructive experiments on the car at the physical level. This solves the technical problem that some connected vehicle safety assessment methods in the prior art need to have a destructive impact on the car hardware at the physical level, making subsequent continuous evaluation difficult.

[0041] As a preferred solution of this embodiment, this embodiment also provides a connected vehicle safety interaction test and evaluation method based on digital twin, which is realized by using the above-mentioned connected vehicle safety interaction test and evaluation system based on digital twin.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A connected vehicle safety interactive test and evaluation system based on digital twins, characterized in that: include: Central control unit module; A digital twin modeling module, the digital twin modeling module is used to model the mapped connected vehicle, and the digital twin modeling module is connected to the central control unit module by communication signals; A network attack and defense simulation module, which is in communication with the central control unit module and is used to execute vehicle network attack and defense scenarios; and A virtual-reality interaction device module, which is connected to the central control unit module via communication signals and is used to simulate the physical movement of the networked vehicle.

2. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 1 is characterized in that: The digital twin modeling module includes a 3D laser scanner, a multi-constellation combined navigation module, a multi-spectral camera array and an on-board sensor simulator module. The construction process of the digital twin modeling module for modeling a networked vehicle includes: Use 3D laser scanners to scan the connected vehicle and the surrounding environment to obtain the first digital twin model of the connected vehicle and the vehicle environment; The multi-constellation combined navigation module is used to measure the spatial information of the networked vehicle to obtain spatial information data, and the spatial information data is integrated into the first geometric three-dimensional model to obtain the second digital twin model; A multispectral camera array is used to collect images of the connected vehicle and the surrounding environment, to generate digital material texture data of the connected vehicle and the environmental information, and the digital material texture data of the connected vehicle and the environmental information is mapped to the second digital twin model to obtain a third digital twin model; The virtual vehicle-mounted sensing device is added to the connected vehicle digital twin model in the third digital twin model using the vehicle-mounted sensor simulator module to obtain the fourth digital twin model.

3. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 2 is characterized in that: The network attack and defense simulation module includes a network traffic generator, a vulnerability injection module, an attack vector library and a network defense simulator. The network traffic generator is used to form network traffic, the vulnerability injection module is used to implant common vehicle system vulnerabilities into the fourth digital twin model, the attack vector library is used to form attack scenarios and attack chains, and the network defense simulator is used to input network defense mechanisms into the fourth digital twin model to simulate the defense response process of the mapped connected vehicle in an attack scenario.

4. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 2 is characterized in that: The virtual-reality interaction device module includes a virtual demonstration system and an operation feedback system. The virtual demonstration system is used to demonstrate the scene of the vehicle in the fourth digital twin model in the virtual space to the operator, and the operation feedback system is used to feed back the operator's actions to the fourth digital twin model.

5. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 4 is characterized in that: The virtual-reality interaction device module also includes an external display device for displaying scenes in the virtual space to staff members who do not participate in direct interaction.

6. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 5 is characterized in that: The virtual-reality interaction device module also includes a vehicle motion simulation platform, which is used to simulate the real-time motion of the networked vehicle in the fourth digital twin model.

7. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 1 is characterized in that: It also includes an edge computing unit module, which is connected to the central control unit module via communication signals and is used to process vehicle information of the networked vehicle and upload it to the central control unit after processing.

8. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 1 is characterized in that: It also includes a cloud computing server, which is communicatively connected to the central control unit module and is used to assist the central control unit module in processing data information of each module.

9. The digital twin-based connected vehicle safety interactive test and evaluation system according to claim 8, characterized in that: The cloud computing service end includes a data processing cluster module, a data analysis platform and a machine learning training server. The data processing cluster module is used to process the data uploaded by the central unit module, the data analysis platform is used to analyze the data, and the machine learning training server is used to acquire the data processed by the data processing cluster module and the data analysis platform and optimize the neural network model.

10. A connected vehicle safety interactive test and evaluation method based on digital twins, characterized in that: The connected vehicle safety interactive test and evaluation system based on digital twins as described in claims 1-9 is realized.

Images