Virtual-real combined unmanned vehicle indoor experiment system

Abstract

The invention relates to the field of digital simulation, in particular to a virtuality and reality combined unmanned vehicle indoor experiment system. Comprising a cloud server, a large-screen display system, a physical unmanned vehicle platform and an edge computing node, the cloud server comprises a scene simulation and data generation module and a user interaction module, and the large-screen display system is used for displaying an image signal on a screen; the physical unmanned vehicle platform collects an image on a screen and sends the obtained image information to the edge computing node; the edge computing node is used for receiving the image information sent by the physical unmanned vehicle platform and preprocessing the image information to obtain preprocessed data; and according to the preprocessed data and the content of the test task, making a driving decision by using a decision algorithm, and sending the driving decision and the preprocessed data to a physical unmanned vehicle platform. The method has the advantages that space and resources are saved, experiment controllability and safety are high, and scene simulation is flexible.

Description

Technical Field

[0001] The present invention relates to the field of digital simulation, and in particular to a virtual-real combined unmanned vehicle indoor experiment system. Background Art

[0002] Under the current wave of intelligence and networking, unmanned vehicle swarms, as a key component of future smart transportation, are gradually becoming a hot spot for research and application. However, its development faces many challenges, especially in terms of collaborative control, decision optimization, and safety and reliability in complex environments. Traditional testing methods often rely on field road tests, which are not only costly and time-consuming, but also difficult to reproduce various extreme or rare traffic scenarios, limiting the rapid iteration and optimization of unmanned vehicle technology. Summary of the invention

[0003] In response to the above problems, the integration of end-edge-cloud collaborative architecture and digital simulation has opened up a new path for the development of distributed systems for unmanned vehicle swarms. End-edge-cloud collaboration emphasizes the efficient allocation and collaboration of data processing and computing capabilities between terminal devices (ends), edge servers (edges), and the cloud, enabling unmanned vehicles to respond to complex environmental changes in real time, while using the powerful computing power of the cloud to perform big data analysis and model training to improve decision-making accuracy and efficiency. This architecture not only reduces the computing burden of a single unmanned vehicle, but also promotes efficient communication and collaborative operations between vehicles.

[0004] The introduction of digital simulation technology has further accelerated the development of unmanned vehicle swarm technology. Through the construction of high-precision virtual environments and complex traffic flow simulations, researchers can test the perception, planning, decision-making and control algorithms of unmanned vehicle swarms under highly simulated conditions without the need for frequent field tests. This not only greatly reduces testing costs and risks, but also achieves comprehensive coverage of various extreme scenarios, ensuring that the algorithm remains efficient and stable when facing complex and changeable real road conditions.

[0005] The unmanned vehicle swarm distributed system that combines the end-edge-cloud collaborative architecture with digital simulation provides an innovative solution to address the limitations of actual road testing, accelerates the maturity and commercialization of autonomous driving technology, and drives the intelligent transportation system to a higher level.

[0006] The present invention provides a technical solution:

[0007] A virtual and real unmanned vehicle indoor experimental system, including a cloud server, a large-screen display system, a physical unmanned vehicle platform, and an edge computing node;

[0008] The cloud server includes a scene simulation and data generation module and a user interaction module. The scene simulation and data generation module is used to generate a virtual test scene based on the real scene image information and corresponding location information collected in the test environment, and to generate an image signal corresponding to the state data of the physical unmanned vehicle platform based on the virtual test scene, and then send the image signal to the large-screen display system; the user interaction module is used to receive test tasks from users and display the system status to users;

[0009] A large-screen display system is used to receive image signals sent by the cloud server and display them on the screen;

[0010] A physical unmanned vehicle platform includes a sensor, wherein the sensor includes a camera, the camera collects images on the screen of a large-screen display system, and sends the acquired image information to an edge computing node; the physical unmanned vehicle platform also receives pre-processed data sent by the edge computing node, and based on the pre-processed data, uses computer vision technology to perform semantic segmentation, and then uses a local decision algorithm to generate local decisions based on the results of the semantic segmentation, and the local decisions are used to handle emergency situations; the physical unmanned vehicle generates corresponding motion control commands based on the local decisions and the driving decisions sent by the edge computing node, and then sends the physical unmanned vehicle platform's own state data to a cloud server; during the test, the moving parts of the physical unmanned vehicle platform move according to the motion control commands, but the physical unmanned vehicle platform does not actually move, but only collects the motion data of the moving parts as the physical unmanned vehicle platform's own state data;

[0011] The edge computing node is used to receive image information sent by the physical unmanned vehicle platform, and pre-process the image information to obtain pre-processed data; and, according to the pre-processed data and the content of the test task, use the decision algorithm to make driving decisions; and send the driving decisions and pre-processed data to the physical unmanned vehicle platform.

[0012] Preferably, when the cloud server sends an image signal to the large-screen display system, the data stream is optimized using the NDI protocol via optical fiber.

[0013] Preferably, the local decision algorithm is based on a rule engine model.

[0014] Preferably, the decision algorithm is based on a reinforcement learning model.

[0015] Preferably, the preprocessing includes data cleaning and / or image denoising.

[0016] Preferably, the method for performing data cleaning includes outlier detection.

[0017] Beneficial effects: 1. Save space and resources: Traditionally, testing unmanned vehicle groups usually requires a large outdoor space, which is not only limited by geographical location and weather conditions, but also may involve high site rental and maintenance costs. The present invention uses a large screen to simulate a complex and changeable outdoor environment indoors, which greatly saves physical space requirements, allowing experiments to be carried out in a controlled indoor environment, reducing dependence on external conditions and saving resources.

[0018] 2. Enhanced experimental controllability and safety: Conducting unmanned vehicle group experiments indoors in a simulated environment can more accurately control experimental conditions, such as traffic flow, lighting changes, weather effects, etc., thereby improving the repeatability of the experiment and the accuracy of the results. In addition, this method can effectively avoid unforeseen risks in the real world, such as accidental collisions with other vehicles or pedestrians, and improve the safety of the experimental process.

[0019] 3. Flexible scenario simulation and rapid iteration: Large-screen simulation technology can quickly switch between different test scenarios and environmental conditions, from city streets to complex intersections, and even extreme weather conditions, which can be set up instantly. This means that researchers can complete a variety of test cases in a short period of time, accelerating the development and iteration cycle of algorithms and systems. Compared with arranging test scenarios one by one in a real environment, this method greatly improves R&D efficiency and makes the verification and optimization of unmanned vehicle technology faster and more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 4 is a system structure diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, a virtual and real unmanned vehicle indoor experimental system includes a cloud server, a large-screen display system, a physical unmanned vehicle platform, and an edge computing node;

[0023] Among them, the cloud server includes a scene simulation and data generation module and a user interaction module. Among them, the scene simulation and data generation module is used to generate a virtual test scene according to the real-scene image information and corresponding position information collected in the test environment, and generate an image signal corresponding to the physical unmanned vehicle platform's own state data according to the virtual test scene, and then send the image signal to the large-screen display system; the user interaction module is used to receive test tasks from the user and display the system status to the user; among them, the image signal physical unmanned vehicle platform's own state data refers to information such as speed and direction.

[0024] A large-screen display system is used to receive image signals sent by the cloud server and display them on the screen;

[0025] A physical unmanned vehicle platform includes a sensor, wherein the sensor includes a camera, the camera captures an image on the screen of a large-screen display system, and sends the acquired image information to an edge computing node; the physical unmanned vehicle platform also receives pre-processed data sent by the edge computing node, and based on the pre-processed data, uses computer vision technology to perform semantic segmentation, the content of semantic segmentation includes identifying specific logos and objects, and then generates local decisions using a local decision algorithm based on the result of semantic segmentation, the local decision algorithm is used to handle emergency situations, for example, in the result of semantic segmentation, if an obstacle is found ahead, the content of the local decision is to immediately slow down or stop; the local decision algorithm is based on a rule engine model; the physical unmanned vehicle generates corresponding motion control commands based on the local decisions and the driving decisions sent by the edge computing node, and then sends the physical unmanned vehicle platform's own state data to a cloud server; during the test process, the wheels and other moving parts of the physical unmanned vehicle platform move according to the motion control commands, but the physical unmanned vehicle platform does not actually move, but only collects the motion data of the moving parts such as wheel-side motion information as the physical unmanned vehicle platform's own state data.

[0026] The edge computing node is used to receive image information sent by the physical unmanned vehicle platform and preprocess the image information to obtain preprocessed data, wherein the preprocessing includes data cleaning and / or image denoising, wherein the method for data cleaning includes outlier detection, and there are many image denoising algorithms, such as many in OpenCV, which will not be repeated here; and, according to the preprocessed data and the content of the test task, use the decision algorithm to make driving decisions, and the decision algorithm is based on the reinforcement learning model; and send the driving decision and preprocessed data to the physical unmanned vehicle platform; the specific content of the driving decision is to plan the route, avoid congestion, etc.

[0027] In some embodiments, when the cloud server sends an image signal to the large-screen display system, the data stream is optimized for transmission via optical fiber using the NDI protocol;

[0028] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A virtual and real combined unmanned vehicle indoor experimental system, characterized in that: Includes cloud servers, large-screen display systems, physical unmanned vehicle platforms, and edge computing nodes; The cloud server includes a scene simulation and data generation module and a user interaction module. The scene simulation and data generation module is used to generate a virtual test scene based on the real scene image information and corresponding location information collected in the test environment, and to generate an image signal corresponding to the state data of the physical unmanned vehicle platform based on the virtual test scene, and then send the image signal to the large-screen display system; the user interaction module is used to receive test tasks from users and display the system status to users; A large-screen display system is used to receive image signals sent by the cloud server and display them on the screen; A physical unmanned vehicle platform includes a sensor, wherein the sensor includes a camera, the camera collects images on the screen of a large-screen display system, and sends the acquired image information to an edge computing node; the physical unmanned vehicle platform also receives pre-processed data sent by the edge computing node, and based on the pre-processed data, uses computer vision technology to perform semantic segmentation, and then uses a local decision algorithm to generate local decisions based on the results of the semantic segmentation, and the local decisions are used to handle emergency situations; the physical unmanned vehicle generates corresponding motion control commands based on the local decisions and the driving decisions sent by the edge computing node, and then sends the physical unmanned vehicle platform's own state data to a cloud server; during the test, the moving parts of the physical unmanned vehicle platform move according to the motion control commands, but the physical unmanned vehicle platform does not actually move, but only collects the motion data of the moving parts as the physical unmanned vehicle platform's own state data; The edge computing node is used to receive image information sent by the physical unmanned vehicle platform, and pre-process the image information to obtain pre-processed data; and, according to the pre-processed data and the content of the test task, use the decision algorithm to make driving decisions; and send the driving decisions and pre-processed data to the physical unmanned vehicle platform.

2. The virtual-real combined unmanned vehicle indoor experiment system according to claim 1 is characterized in that: When the cloud server sends image signals to the large-screen display system, the NDI protocol is used to optimize the data flow through optical fiber.

3. The virtual-real combined unmanned vehicle indoor experiment system according to claim 1 is characterized in that: The local decision algorithm is based on a rule engine model.

4. The virtual-real combined unmanned vehicle indoor experiment system according to claim 1 is characterized in that: The decision-making algorithm is based on a reinforcement learning model.

5. The virtual-real combined unmanned vehicle indoor experiment system according to claim 1 is characterized in that: The preprocessing includes data cleaning and / or image denoising.

6. The virtual-real combined unmanned vehicle indoor experiment system according to claim 5 is characterized in that: One of the methods for data cleaning is outlier detection.

Images