Multi-vehicle hybrid test simulation system and method for unmanned vehicles

Abstract

The invention is suitable for the technical field of computer application, and provides a multi-vehicle hybrid test simulation system and method for unmanned vehicles, and the system comprises a scene definition regulation and control module which is used for responding to the scene configuration operation of a user, generating scene information, and transmitting the scene information to a virtual environment perception module and a virtual vehicle control module; the virtual environment sensing module is used for receiving the scene information, generating sensing fusion information according to the scene information and sending the sensing fusion information to the virtual vehicle control module; and the virtual vehicle control module is used for receiving the scene information and the sensing fusion information, generating a virtual signal according to the scene information and the sensing fusion information, and controlling and planning the unmanned vehicle according to the virtual signal. Therefore, through the system, the hybrid test of a plurality of unmanned vehicles in the scene containing the background traffic flow is realized, the test cost is saved, the test efficiency is improved, and the test risk is reduced.

Description

Technical Field

[0001] This application belongs to the field of computer application technology, and in particular relates to a multi-vehicle mixed-program testing simulation system and method for unmanned vehicles. Background Technology

[0002] As a crucial component of smart mine construction, unmanned driving technology in mining has broad development prospects and significant application value. With continuous technological advancements and the coordinated development of the industry chain, unmanned driving technology will play an increasingly important role in the mining industry. Among these developments, testing unmanned vehicles is of paramount importance.

[0003] Among related technologies, real-vehicle testing of driverless vehicles can be conducted, but this method suffers from high testing costs, low efficiency, and high risks. Summary of the Invention

[0004] The purpose of this application is to provide a multi-vehicle mixed testing simulation system and method for unmanned vehicles, which can solve the problems of high testing cost, low efficiency and high risk in the real vehicle testing of unmanned vehicles in related technologies.

[0005] In a first aspect, embodiments of this application provide a multi-vehicle hybrid testing simulation system for unmanned vehicles, comprising: a scene definition and control module, a virtual environment perception module, and N virtual vehicle control modules, wherein N is an integer greater than 1; wherein, the scene definition and control module is used to generate scene information in response to the user's scene configuration operation, and send the scene information to the virtual environment perception module and the virtual vehicle control module, wherein the scene information includes configuration information of N unmanned vehicles, background traffic flow, obstacles, and task equipment; the i-th virtual vehicle control module is used to receive the scene information sent by the scene definition and control module and the perception fusion information sent by the virtual environment perception module, generate virtual signals according to the scene information and the perception fusion information, and control and plan the i-th unmanned vehicle according to the virtual signals, wherein i is an integer greater than or equal to 1 and less than or equal to N; the virtual environment perception module is used to receive the scene information sent by the scene definition and control module, generate perception fusion information according to the scene information, and send the perception fusion information to the virtual vehicle control module, wherein the perception fusion information includes all vehicle information and obstacle information within the perception range of the unmanned vehicle.

[0006] In one possible implementation of the first aspect, the virtual vehicle control module includes a virtual vehicle motion module and an autonomous driving system, wherein the virtual vehicle motion module is used for:

[0007] It receives vehicle control commands from the autonomous driving system and scene information from the scene definition and control module. Based on the scene information and vehicle control commands, it generates real-time information of the autonomous vehicle and sends the real-time information of the autonomous vehicle to the virtual environment perception module. It also receives perception fusion information from the virtual environment perception module and generates virtual signals based on the scene information, vehicle control commands, and perception fusion information. Finally, it sends the virtual signals to the autonomous driving system.

[0008] The aforementioned autonomous driving system is used for:

[0009] The vehicle control commands are sent to the virtual vehicle motion module, and the virtual signals sent by the virtual vehicle motion module are received. The unmanned vehicle is controlled and planned according to the virtual signals.

[0010] The aforementioned virtual environment perception module is also used for:

[0011] It receives real-time information about the unmanned vehicle from the virtual vehicle motion module and generates perception fusion information based on the real-time information of the unmanned vehicle and the scene information.

[0012] Optionally, in another possible implementation of the first aspect, the aforementioned virtual signals include virtual positioning signals, virtual underlying signals, virtual vehicle-to-vehicle (V2V) signals, and virtual sensing signals. The aforementioned virtual vehicle motion module is specifically used for:

[0013] The scene information and vehicle control commands are input into the preset vehicle dynamics model of the virtual vehicle motion module to generate virtual positioning signals and virtual underlying signals.

[0014] Based on the sensor fusion information, virtual V2V signals and virtual sensing signals are generated.

[0015] Optionally, in another possible implementation of the first aspect, the aforementioned multi-vehicle mixed-system test simulation system for unmanned vehicles also includes a computing power resource allocation module;

[0016] The above-described scenario-defined control module is also used for:

[0017] Send the scene information to the computing resource allocation module;

[0018] The aforementioned computing resource allocation module is used for:

[0019] The system receives scenario information, generates computing power resource allocation information based on the scenario information, and determines the computing power devices used to simulate each unmanned vehicle based on the computing power resource allocation information. The computing power devices corresponding to the unmanned vehicles are used to carry the virtual vehicle control module corresponding to the unmanned vehicles.

[0020] Optionally, in another possible implementation of the first aspect, the aforementioned multi-vehicle hybrid test simulation system for unmanned vehicles also includes a data dynamic monitoring module;

[0021] The aforementioned dynamic data monitoring module is used for:

[0022] The system uses a visual interface to display the input and output data of the virtual vehicle control module, as well as the status data of background traffic, obstacles, and task equipment.

[0023] Optionally, in another possible implementation of the first aspect, the aforementioned scenario information also includes task information, and the aforementioned virtual vehicle control module is further used for:

[0024] Based on the task information, the system sends real-time information of the unmanned vehicle and the task application to the task equipment, receives the task approval signal from the task equipment, generates task instructions, controls the task vehicle in the task information to execute the task, and receives the task completion signal from the task equipment.

[0025] Optionally, in another possible implementation of the first aspect, the aforementioned task information includes loading task information, crushing task information, and charging task information; the aforementioned task equipment includes loading equipment, crushing equipment, and charging equipment; and the aforementioned virtual vehicle control module is further used for:

[0026] Based on the loading task information, the system controls the work vehicle to perform the loading task. After receiving the end-loading signal from the loading equipment, the system controls the work vehicle to perform the crushing task based on the crushing task information. After receiving the end-crushing signal from the crushing equipment, the system controls the work vehicle to perform the charging task based on the charging task information. The system receives the end-charging signal from the charging equipment.

[0027] Optionally, in another possible implementation of the first aspect, the aforementioned multi-vehicle hybrid test simulation system for unmanned vehicles further includes a multi-vehicle collaborative hybrid algorithm module;

[0028] The aforementioned virtual vehicle control module is also used for:

[0029] The system sends real-time information from the unmanned vehicle to the multi-vehicle collaborative hybrid algorithm module and receives vehicle control commands from the multi-vehicle collaborative hybrid algorithm module.

[0030] The aforementioned multi-vehicle collaborative hybrid programming algorithm module is used for:

[0031] Receive real-time information from unmanned vehicles, generate vehicle control commands based on the real-time information, and send the vehicle control commands to the virtual vehicle control module.

[0032] Secondly, this application also provides a multi-vehicle hybrid test simulation method for unmanned vehicles, applied to a virtual vehicle control module, including: receiving scene information sent by a scene definition and control module and perception fusion information sent by a virtual environment perception module; generating virtual signals based on the scene information and perception fusion information; and controlling and planning the unmanned vehicle based on the virtual signals.

[0033] In one possible implementation of the second aspect, the aforementioned scenario information also includes job task information, and the method further includes:

[0034] Based on the task information, send real-time information of the unmanned vehicle and the task application to the task equipment;

[0035] Receive the work consent signal sent by the task device and generate the work task instruction;

[0036] Control the work vehicles in the work task information to execute the task, and receive the work completion signal sent by the task equipment.

[0037] The beneficial effects of the embodiments of this application compared with the prior art are: the multi-vehicle mixed test simulation system for unmanned vehicles provided by this application can realize the mixed simulation test of multiple unmanned vehicles and background traffic, which saves test costs, improves test efficiency and reduces test risks. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of a multi-vehicle hybrid test simulation system for unmanned vehicles provided in one embodiment of this application;

[0040] Figure 2 This is a functional schematic diagram of the scene definition and control module provided in one embodiment of this application;

[0041] Figure 3 This is a functional schematic diagram of a virtual environment perception module provided in an embodiment of this application;

[0042] Figure 4 This is a functional schematic diagram of a virtual vehicle motion module provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of a multi-vehicle hybrid test simulation system for unmanned vehicles provided in another embodiment of this application;

[0044] Figure 6 This is a functional schematic diagram of a computing resource allocation module provided in an embodiment of this application;

[0045] Figure 7 This is a functional schematic diagram of a data dynamic monitoring module provided in one embodiment of this application;

[0046] Figure 8 This is a flowchart illustrating a multi-vehicle mixed-system test simulation method for unmanned vehicles provided in an embodiment of this application. Detailed Implementation

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0048] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0049] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0050] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0051] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0053] The multi-vehicle mixed-system test simulation system and method for unmanned vehicles provided in this application are described in detail below with reference to the accompanying drawings.

[0054] Figure 1 The diagram shows a structural schematic of a multi-vehicle hybrid test simulation system for unmanned vehicles provided in an embodiment of this application.

[0055] like Figure 1 As shown, the multi-vehicle hybrid test simulation system 10 for the unmanned vehicle includes: a scene definition and control module 11, a virtual environment perception module 13, and N virtual vehicle control modules 12, where N is an integer greater than 1;

[0056] The scene definition and control module 11 is used to respond to the user's scene configuration operation, generate scene information, and send the scene information to the virtual environment perception module and the virtual vehicle control module. The scene information includes the configuration information of N unmanned vehicles, background traffic flow, obstacles and task equipment.

[0057] The i-th virtual vehicle control module 12 is used to receive scene information sent by the scene definition and control module and perception fusion information sent by the virtual environment perception module, generate virtual signals based on the scene information and perception fusion information, and control and plan the i-th unmanned vehicle based on the virtual signals. The perception fusion information includes all vehicle information and obstacle information within the perception range of the unmanned vehicle, and i is an integer greater than or equal to 1 and less than or equal to N.

[0058] The virtual environment perception module 13 is used to receive scene information sent by the scene definition and control module, generate perception fusion information based on the scene information, and send the perception fusion information to the virtual vehicle control module.

[0059] As one possible implementation of this application, the user can configure the test simulation scenario through scene configuration operations such as input or selection. The scene definition and control module 11 can respond to the user's scene configuration operation and generate scene information. The scene information may include configuration information of N unmanned vehicles, background traffic flow configuration information, obstacle and task equipment configuration information, etc. Among them, the unmanned vehicles may be driverless vehicles used for transportation in the mine, and the background traffic flow may be manned vehicles. The scene definition and control module 11 can send the generated scene information to the virtual environment perception module 13 and the virtual vehicle control module 12.

[0060] As an example, such as Figure 2 As shown, scene information can be divided into static definition information and dynamic control information. The configuration information of autonomous vehicles can include: dynamic characteristic parameters, initial pose, range performance, sensor model, longitudinal velocity, lateral offset, and enabled status; the configuration information of background traffic (e.g., delivery vehicles) can include: scheduling task, initial pose, driving style, lifting characteristics, range performance, start / stop, remaining energy, longitudinal velocity, and lateral offset; the configuration information of obstacles can include: target point or target trajectory, initial pose, random characteristics, physical size, longitudinal velocity, lateral offset, type (e.g., pedestrian, cart, rock, passenger car, etc.), longitudinal velocity, lateral offset, and enabled status; the task equipment can include loading equipment (e.g., excavators, loaders, etc.), charging equipment (e.g., charging piles, gas stations, etc.) and crushing equipment (e.g., crushing plants, etc.); the configuration information of loading equipment can include: loading pose, loading mode, loading efficiency, and loading time; the configuration information of charging equipment can include: total energy, equipment pose, and charging efficiency; the configuration information of crushing equipment can include: total capacity, equipment pose, crushing efficiency, and unloading enabled.

[0061] It should be noted that the types or configuration information of unmanned vehicles, background traffic, obstacles and task equipment listed above are merely illustrative. In actual use, they can be determined according to actual testing needs and testing scenarios. This application embodiment does not limit this.

[0062] As one possible implementation of this application, each unmanned vehicle corresponds to a virtual vehicle control module 12. When the scene information includes N unmanned vehicles, the system includes N virtual vehicle control modules 12. The i-th virtual vehicle control module 12 (i is an integer greater than or equal to 1 and less than or equal to N) can receive scene information sent by the scene definition and control module 11 and perception fusion information sent by the virtual environment perception module 13. The perception fusion information is all vehicle information and obstacle information perceived by the i-th unmanned vehicle in the virtual environment perception module through the paired sensors (such as truth sensors or noise sensors). The vehicle information includes information of other unmanned vehicles and background traffic information. After fusion, the perception fusion information is obtained. The virtual vehicle control module 13 generates unmanned vehicle functional characteristics that meet user needs based on the scene information, generates virtual signals based on the perception fusion information, and drives the control, planning, prediction, and main control module operation of the corresponding unmanned vehicle based on the virtual signals.

[0063] In one possible implementation, the virtual environment perception module 13 can receive scene information and simulate the test scene configured by the user based on the scene information. This simulation may include obstacle simulation, background traffic flow simulation, and task equipment simulation (loading equipment simulation, crushing equipment simulation, and charging equipment simulation). It can also perceive vehicles and obstacles within the perception range of each unmanned vehicle through the sensors of the unmanned vehicle based on the simulated scene. Combining the vehicle-to-vehicle (V2V) vehicle network information, the perceived information is converted into coordinate system information to generate perception fusion information. The perception fusion information is then sent to the virtual vehicle control module 12 corresponding to each unmanned vehicle.

[0064] For example, such as Figure 3As shown, the virtual environment perception module 13 may include obstacle simulation, background traffic flow simulation, loading equipment simulation, crushing equipment simulation, charging equipment simulation, and vehicle perception fusion. Among them, obstacle simulation can be modeled based on kinematic theory, realizing the motion characteristics of obstacles according to physical parameters and preset trajectories, and has the dynamic control function of longitudinal speed and lateral offset; background traffic flow simulation can be modeled by combining micro and macro traffic flow theories, realizing the operation cycle of vehicles through loading and unloading task scheduling configuration, forming a mixed test scenario with unmanned vehicles, and has the dynamic control function of start-stop and offset; loading equipment simulation can interact with loading vehicles by triggering two state signal flow commands of start loading and end loading based on the set loading posture, loading mode and loading efficiency, until the loading operation process is completed; charging equipment simulation can interact with charging vehicles by triggering two state signal flow commands of start charging and end charging based on the set equipment posture, charging efficiency and total charging energy, and judging the energy status of charging vehicles, until the charging operation process is completed; crushing equipment simulation can judge the current capacity of crushing equipment by the start unloading and end unloading state signals corresponding to vehicle commands based on the set equipment posture, crushing efficiency and total capacity of crushing equipment, and output the corresponding unloading enable state according to the crushing efficiency.

[0065] Furthermore, the virtual vehicle control module 12 can also control the unmanned vehicle to perform task testing based on the user-configured scenario information, thereby making the simulation system more comprehensive and improving the accuracy and realism of the unmanned vehicle simulation test. That is, in one possible implementation of this application embodiment, the aforementioned scenario information also includes task information, and the virtual vehicle control module 12 can also be used for:

[0066] Based on the task information, the system sends real-time information of the unmanned vehicle and the task application to the task equipment, receives the task approval signal from the task equipment, generates task instructions, controls the task vehicle in the task information to execute the task, and receives the task completion signal from the task equipment.

[0067] As one possible implementation, the scenario information includes task information, which can be configured by the user according to the actual task requirements of the unmanned vehicle. The virtual vehicle control module 12 can determine the task type based on the task information, and then send the unmanned vehicle's real-time information (such as the unmanned vehicle's pose status) and task application to the task device involved in the task type. After the task device agrees, it can send an agreement signal to the virtual vehicle control module 12. After receiving the signal, the virtual vehicle control module 12 can generate a task instruction and send it to the task vehicle in the task information, i.e., the unmanned vehicle that needs to perform the task, and control the unmanned vehicle to perform the corresponding task. After the unmanned vehicle completes the task, the task device can send an end-of-task signal to the virtual vehicle control module 12. The virtual vehicle control module 12 indicates that the task is completed after receiving the end-of-task signal.

[0068] For example, task information can include loading task information, which is related to the unmanned vehicle's loading task, such as loading time and loading efficiency.

[0069] Furthermore, considering the actual use of unmanned mining vehicles, the operational tasks can be loading, crushing, and charging. That is, the aforementioned operational task information includes loading, crushing, and charging task information, and the task equipment includes loading equipment, crushing equipment, and charging equipment. The aforementioned virtual vehicle control module 12 can also be used for:

[0070] Based on the loading task information, the system controls the work vehicle to perform the loading task. After receiving the end-loading signal from the loading equipment, the system controls the work vehicle to perform the crushing task based on the crushing task information. After receiving the end-crushing signal from the crushing equipment, the system controls the work vehicle to perform the charging task based on the charging task information. The system receives the end-charging signal from the charging equipment.

[0071] One possible implementation is to perform a crushing task after the loading task. That is, based on the loading task information, the unmanned vehicle is controlled to drive to the loading equipment to perform the loading task. After receiving the end-loading operation signal sent by the loading equipment, the unmanned vehicle is controlled to drive to the crushing equipment to perform the crushing task. After receiving the end-crushing operation signal sent by the crushing equipment, the unmanned vehicle is controlled to drive to the crushing equipment to perform the crushing operation. After receiving the end-crushing operation signal sent by the crushing equipment, the charging task information is used to determine whether the battery is low and needs to be charged. If so, the unmanned vehicle is controlled to drive to the charging equipment to perform the charging task and receives the end-charging operation signal sent by the charging equipment.

[0072] Furthermore, the virtual vehicle control module 12 may include a virtual vehicle motion module and an autonomous driving system. The virtual vehicle motion module and the autonomous driving system process different data and transmit the processing results, thereby further improving the accuracy of autonomous vehicle simulation testing. That is, in one possible implementation of this application embodiment, the above-mentioned virtual vehicle control module includes a virtual vehicle motion module and an autonomous driving system. The virtual vehicle motion module can be used for:

[0073] It receives vehicle control commands from the autonomous driving system and scene information from the scene definition and control module. Based on the scene information and vehicle control commands, it generates real-time information of the autonomous vehicle and sends the real-time information of the autonomous vehicle to the virtual environment perception module. It also receives perception fusion information from the virtual environment perception module and generates virtual signals based on the scene information, vehicle control commands, and perception fusion information. Finally, it sends the virtual signals to the autonomous driving system.

[0074] The aforementioned autonomous driving system can be used for:

[0075] The vehicle control commands are sent to the virtual vehicle motion module, and the virtual signals sent by the virtual vehicle motion module are received. The unmanned vehicle is controlled and planned according to the virtual signals.

[0076] The aforementioned virtual environment awareness module can also be used for:

[0077] It receives real-time information about the unmanned vehicle from the virtual vehicle motion module and generates perception fusion information based on the real-time information of the unmanned vehicle and the scene information.

[0078] As one possible implementation, the virtual vehicle control module 12 may include a virtual vehicle motion module and an autonomous driving system. The autonomous driving system can send vehicle control commands, such as throttle, brake, steering angle, gear position, and handbrake commands, to the virtual vehicle motion module. Based on the received scene information and vehicle control commands, the virtual vehicle motion module generates real-time information about the autonomous vehicle corresponding to the virtual vehicle control module 12, such as the autonomous vehicle's pose, speed, perception enable status, and sensor type, and sends this real-time information to the virtual environment perception module 13. The virtual environment perception module 13 then transmits the real-time information to the virtual environment perception module 13. The system uses real-time and scene information to perceive vehicles and obstacles around the autonomous vehicle and fuses the perceived information to generate perception fusion information. This information can include the pose, velocity, acceleration, and confidence level of the perceived elements. The perception fusion information is then sent to the virtual vehicle motion module. The virtual vehicle motion module uses scene information, vehicle control commands, and perception fusion information to generate virtual signals, which are then sent to the autonomous driving system. The autonomous driving system uses the virtual signals to determine the autonomous vehicle's own state and the surrounding perceived environment, thereby making corresponding autonomous vehicle behavior planning and control, and generating the next vehicle control command, which is then repeated in a loop.

[0079] Furthermore, the virtual signals may include virtual positioning signals, virtual underlying signals, virtual vehicle-to-vehicle (V2V) signals, and virtual perception signals. This allows the autonomous driving system to make more precise plans for the autonomous vehicle based on the virtual signals, thereby further improving the simulation efficiency of the autonomous vehicle. Specifically, in one possible implementation of this application embodiment, the aforementioned virtual vehicle motion module is used for:

[0080] The scene information and vehicle control commands are input into the preset vehicle dynamics model of the virtual vehicle motion module to generate virtual positioning signals and virtual underlying signals.

[0081] Based on the sensor fusion information, virtual V2V signals and virtual sensing signals are generated.

[0082] As one possible implementation, such as Figure 4 As shown, the virtual vehicle motion module can input scene information and vehicle control commands into a preset vehicle dynamics model, and output the unmanned vehicle state signal through model deduction and calculation. According to the communication protocol of the unmanned driving system, it can be converted into virtual positioning signals and virtual underlying signals adapted to the unmanned driving system. It can also convert perception fusion information into virtual V2V signals and virtual perception signals adapted to the unmanned driving system.

[0083] Furthermore, since the test scenario includes testing of multiple unmanned vehicles, computing resources can be allocated, thereby further improving the efficiency of unmanned vehicle mixed testing. That is, in one possible implementation of the embodiments of this application, the above-mentioned unmanned vehicle multi-vehicle mixed testing simulation system 10 may also include a computing resource allocation module 14;

[0084] The scene definition and control module can also be used for:

[0085] Send the scene information to the computing resource allocation module;

[0086] The computing resource allocation module can be used for:

[0087] Receive scene information, generate computing power resource allocation information based on the scene information, and determine the computing power equipment used to simulate each unmanned vehicle based on the computing power resource allocation information.

[0088] Among them, the computing power equipment corresponding to the unmanned vehicle can be used to carry the virtual vehicle control module corresponding to the unmanned vehicle.

[0089] As one possible implementation, such as Figure 5As shown, the scenario definition and control module 11 can send scenario information to the computing power resource allocation module 14. The computing power resource allocation module 14 dynamically allocates and releases computing power resources according to the number of unmanned vehicles to be started and the names of the unmanned vehicles in the scenario information, generates computing power resource allocation information, and determines the computing power equipment corresponding to each unmanned vehicle for simulation based on the computing power resource allocation information. The corresponding virtual vehicle control module 12 can run on each computing power equipment.

[0090] As an example, such as Figure 6 As shown, the computing power resource allocation module 14 controls M computing power devices to run N virtual vehicle control modules 12. The virtual vehicle control module 12 running for each computing power resource can be determined according to the computing power resource allocation information.

[0091] Furthermore, the data from each simulation module can be displayed through a visual interface, facilitating user monitoring. Specifically, in one possible implementation of this application embodiment, the aforementioned multi-vehicle mixed-system test simulation system 10 for unmanned vehicles may further include a data dynamic monitoring module 15.

[0092] The data dynamic monitoring module can be used for:

[0093] The system uses a visual interface to display the input and output data of the virtual vehicle control module, as well as the status data of background traffic, obstacles, and task equipment.

[0094] As one possible implementation of this application, such as Figure 5 As shown, each simulation module can send input and output data to the data dynamic monitoring module 15. The data dynamic monitoring module 15 can display the input and output data of the virtual vehicle control module, as well as the status data of background traffic, obstacles and task equipment to the user through a visual interface.

[0095] For example, such as Figure 7 As shown, the data dynamic monitoring module 15 can display the input of the collaborative hybrid algorithm module (such as the driving instructions of each unmanned vehicle, including target speed, target pose, etc.), the input and output of the virtual vehicle control module (such as the pose, speed, and driving trajectory of the unmanned vehicle, etc.), obstacle status, background traffic status, energy charging equipment status, crushing equipment status, and status equipment status, etc.

[0096] It should be noted that the content displayed by the data dynamic monitoring module listed above is only exemplary. In actual use, it can be determined according to actual testing needs and testing scenarios. This application embodiment does not limit this.

[0097] Furthermore, the coordinated operation of multiple unmanned vehicles and background traffic can be calculated using algorithms, thereby making the testing of multiple unmanned vehicles more accurate. That is, in one possible implementation of the embodiments of this application, the above-mentioned multi-vehicle mixed-operation test simulation system 10 may further include a multi-vehicle cooperative mixed-operation algorithm module 16;

[0098] The virtual vehicle control module 12 can also be used for:

[0099] The system sends real-time information from the unmanned vehicle to the multi-vehicle collaborative hybrid algorithm module and receives vehicle control commands from the multi-vehicle collaborative hybrid algorithm module.

[0100] Multi-vehicle collaborative hybrid algorithm module 16 can be used for:

[0101] Receive real-time information from unmanned vehicles, generate vehicle control commands based on the real-time information, and send the vehicle control commands to the virtual vehicle control module.

[0102] In one possible implementation, the virtual vehicle control module 12 can send the real-time information of each unmanned vehicle (such as the pose, speed, and driving trajectory of each unmanned vehicle) to the multi-vehicle cooperative hybrid algorithm module 16. The multi-vehicle cooperative hybrid algorithm module 16 generates vehicle control commands (such as target speed and target pose) for each unmanned vehicle based on the real-time information of the unmanned vehicles, and sends the vehicle control commands to the corresponding virtual vehicle control module 12.

[0103] The multi-vehicle mixed testing simulation system for unmanned vehicles provided in this application embodiment, through a scene definition and control module, a virtual environment perception module, and multiple virtual vehicle control modules, enables flexible mixed testing of multiple unmanned vehicles with background traffic flow. This saves testing costs, improves testing efficiency, and reduces testing risks. Furthermore, it can control unmanned vehicles to perform tasks, improving the comprehensiveness of simulation testing. Through a computing resource allocation module, computing resources are rationally allocated, enabling parallel testing of multiple unmanned vehicles, improving the efficiency and practicality of testing simulation. Through a multi-vehicle collaborative mixed-system algorithm module, the synergy of multi-vehicle testing simulation is enhanced, and through a dynamic data monitoring module, the user experience is improved.

[0104] To achieve the above embodiments, this application also proposes a multi-vehicle mixed-system test simulation method for unmanned vehicles.

[0105] Figure 8 This is a flowchart illustrating a multi-vehicle hybrid testing simulation method for unmanned vehicles provided in an embodiment of this application.

[0106] like Figure 8 As shown, this multi-vehicle hybrid testing simulation method for unmanned vehicles, applied to a virtual vehicle control module, includes the following steps:

[0107] Step 801: Receive scene information sent by the scene definition and control module and perception fusion information sent by the virtual environment perception module.

[0108] In one possible implementation, the user can configure the test simulation scenario through scene configuration operations such as input or selection. The scene definition and control module can respond to the user's scene configuration operation and generate scene information. The scene information can include the configuration information of N unmanned vehicles, the configuration information of the background traffic flow, the configuration information of obstacles and task equipment, etc. Among them, the unmanned vehicles can be driverless vehicles used for transportation in the mine, the background traffic flow can be manned vehicles, and the perception fusion information can be all vehicle information and obstacle information perceived by the unmanned vehicles through the paired sensors (such as truth sensors or noise sensors). The vehicle information includes information of other unmanned vehicles and background traffic flow information.

[0109] Step 802: Generate virtual signals based on scene information and perception fusion information.

[0110] In one possible implementation, virtual signals can be generated based on scene information and perception fusion information.

[0111] In one possible implementation of this application, the virtual signals may include virtual positioning signals, virtual underlying signals, virtual vehicle-to-vehicle (V2V) signals, and virtual perception signals. It can receive vehicle control commands, input scene information and vehicle control commands into a preset vehicle dynamics model, calculate through model deduction, output unmanned vehicle state signals, and convert them into virtual positioning signals and virtual underlying signals adapted to the unmanned driving system according to the communication protocol of the unmanned driving system. It can also convert perception fusion information into virtual V2V signals and virtual perception signals adapted to the unmanned driving system.

[0112] Step 803: Control and plan the unmanned vehicle based on the virtual signals.

[0113] In one possible implementation, the control, planning, prediction, and main control module of the corresponding unmanned vehicle can be driven by virtual signals, thereby controlling and planning the unmanned vehicle.

[0114] Furthermore, based on user-configured scenario information, the unmanned vehicle can be controlled to perform task testing, thereby making the testing simulation method more comprehensive and improving the accuracy and realism of unmanned vehicle simulation testing. Specifically, in one possible implementation of this application embodiment, the aforementioned scenario information also includes task information, and the method may further include:

[0115] Based on the task information, send real-time information of the unmanned vehicle and the task application to the task equipment;

[0116] Receive the work consent signal sent by the task device and generate the work task instruction;

[0117] Control the work vehicles in the work task information to execute the task, and receive the work completion signal sent by the task equipment.

[0118] In one possible implementation, the scenario information includes task information, which can be configured by the user according to the actual task requirements of the unmanned vehicle. Based on the task information, the task type can be determined, and then the task device involved in the task type can send the unmanned vehicle's real-time information (such as the unmanned vehicle's pose status) and task application. After the task device agrees, it can send a work consent signal. After receiving the signal, the virtual vehicle control module can generate a task instruction and send it to the task vehicle in the task information, i.e., the unmanned vehicle that needs to perform the task, and control the unmanned vehicle to perform the corresponding task. After the unmanned vehicle completes the task, the task device can send a work end signal to the virtual vehicle control module. The virtual vehicle control module receives the work end signal, which indicates that the task is completed.

[0119] For example, task information can include loading, crushing, and charging information, such as the time and efficiency of loading, crushing, and charging. Task equipment can include loading equipment, crushing equipment, and charging equipment. The crushing task can be performed after the loading task. Specifically, based on the loading task information, the unmanned vehicle (RV) is controlled to travel to the loading equipment to perform the loading task. After receiving a signal indicating the end of loading from the loading equipment, the RV is controlled to travel to the crushing equipment to perform the crushing task. After receiving a signal indicating the end of crushing from the crushing equipment, the RV is controlled to travel to the crushing equipment to perform the crushing task. After receiving a signal indicating the end of crushing from the crushing equipment, the RV is controlled to travel to the charging equipment to perform the charging task, based on the charging task information. Finally, the RV is controlled to travel to the charging equipment to perform the charging task, and receives a signal indicating the end of charging from the charging equipment.

[0120] The multi-vehicle mixed testing simulation method for unmanned vehicles provided in this application realizes flexible mixed testing of multiple unmanned vehicles with background traffic by perceiving the environment around each unmanned vehicle and planning and controlling each unmanned vehicle. This saves testing costs, improves testing efficiency, and reduces testing risks. In addition, it can control the unmanned vehicles to perform work tasks, thereby improving the comprehensiveness of simulation testing.

[0121] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0123] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0124] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-vehicle hybrid test simulation system for unmanned vehicles, characterized in that, include: The system includes a scene definition and control module, a virtual environment perception module, and N virtual vehicle control modules, where N is an integer greater than 1. The scene definition and control module is used to respond to the user's scene configuration operation, generate scene information, and send the scene information to the virtual environment perception module and the virtual vehicle control module. The scene information includes the configuration information of N unmanned vehicles, background traffic flow, obstacles, and task equipment. The i-th virtual vehicle control module is used to receive the scene information sent by the scene definition and control module and the perception fusion information sent by the virtual environment perception module, generate a virtual signal according to the scene information and the perception fusion information, and control and plan the i-th unmanned vehicle according to the virtual signal. The perception fusion information includes all vehicle information and obstacle information within the perception range of the i-th unmanned vehicle, where i is an integer greater than or equal to 1 and less than or equal to N. The virtual environment perception module is used to receive the scene information sent by the scene definition and control module, generate perception fusion information based on the scene information, and send the perception fusion information to the virtual vehicle control module.

2. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 1, characterized in that, The virtual vehicle control module includes a virtual vehicle motion module and an autonomous driving system. The virtual vehicle motion module is used for: The system receives vehicle control commands sent by the autonomous driving system and scene information sent by the scene definition and control module. Based on the scene information and the vehicle control commands, it generates real-time information of the autonomous vehicle and sends the real-time information of the autonomous vehicle to the virtual environment perception module. It also receives perception fusion information sent by the virtual environment perception module, generates virtual signals based on the scene information, the vehicle control commands, and the perception fusion information, and sends the virtual signals to the autonomous driving system. The unmanned driving system is used for: The vehicle control command is sent to the virtual vehicle motion module, and the virtual signal sent by the virtual vehicle motion module is received. The unmanned vehicle is controlled and planned according to the virtual signal. The virtual environment perception module is also used for: The system receives real-time information about the unmanned vehicle sent by the virtual vehicle motion module, and generates the perception fusion information based on the real-time information about the unmanned vehicle and the scene information.

3. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 2, characterized in that, The virtual signals include virtual positioning signals, virtual underlying signals, virtual vehicle-to-vehicle (V2V) signals, and virtual sensing signals. The virtual vehicle motion module is specifically used for: The scene information and the vehicle control commands are input into the preset vehicle dynamics model of the virtual vehicle motion module to generate the virtual positioning signal and the virtual underlying signal; Based on the perception fusion information, the virtual V2V signal and the virtual perception signal are generated.

4. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 1, characterized in that, The multi-vehicle hybrid test simulation system for unmanned vehicles also includes a computing power resource allocation module; The scenario definition and control module is also used for: The scene information is sent to the computing power resource allocation module; The computing power resource allocation module is used for: The system receives the scene information, generates computing power resource allocation information based on the scene information, and determines the computing power devices used to simulate each of the unmanned vehicles based on the computing power resource allocation information. The computing power devices corresponding to the unmanned vehicles are used to carry the virtual vehicle control module corresponding to the unmanned vehicles.

5. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 1, characterized in that, The multi-vehicle hybrid test simulation system for unmanned vehicles also includes a data dynamic monitoring module; The data dynamic monitoring module is used for: The input and output data of the virtual vehicle control module, as well as the status data of the background traffic, obstacles, and task equipment, are displayed through a visual interface.

6. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 1, characterized in that, The scene information also includes task information, and the virtual vehicle control module is further used for: Based on the task information, the system sends the real-time information of the unmanned vehicle and the task application to the task device, receives the consent signal from the task device, generates a task instruction, controls the task vehicle in the task information to perform the task, and receives the end-of-task signal from the task device.

7. The multi-vehicle mixed-system test simulation system for unmanned vehicles as described in claim 6, characterized in that, The task information includes loading task information, crushing task information, and charging task information; the task equipment includes loading equipment, crushing equipment, and charging equipment; and the virtual vehicle control module is further used for: Based on the loading task information, the system controls the work vehicle to perform the loading task. After receiving the end-loading operation signal sent by the loading equipment, the system controls the work vehicle to perform the crushing task based on the crushing task information. After receiving the end-crushing operation signal sent by the crushing equipment, the system controls the work vehicle to perform the charging task based on the charging task information, and receives the end-charging operation signal sent by the charging equipment.

8. The multi-vehicle hybrid test simulation system for unmanned vehicles as described in claim 1, characterized in that, The multi-vehicle hybrid programming test simulation system for unmanned vehicles also includes a multi-vehicle collaborative programming algorithm module; The virtual vehicle control module is also used for: The unmanned vehicle's real-time information is sent to the multi-vehicle collaborative hybrid algorithm module, and the vehicle control commands sent by the multi-vehicle collaborative hybrid algorithm module are received. The multi-vehicle collaborative hybrid programming algorithm module is used for: The system receives real-time information from the unmanned vehicle, generates vehicle control commands based on the real-time information, and sends the vehicle control commands to the virtual vehicle control module.

9. A multi-vehicle mixed-program testing simulation method for unmanned vehicles, applied to a virtual vehicle control module, characterized in that, include: The system receives scene information sent by the scene definition and control module and perception fusion information sent by the virtual environment perception module. The scene information includes configuration information of multiple unmanned vehicles, background traffic flow, obstacles and task equipment. The perception fusion information includes information of all vehicles and obstacles within the perception range of the unmanned vehicles. Based on the scene information and the perception fusion information, a virtual signal is generated; The unmanned vehicle is controlled and planned based on the virtual signals.

10. The method as described in claim 9, characterized in that, The scenario information also includes task information, and the method further includes: Based on the task information, the unmanned vehicle's real-time information and task request are sent to the task equipment. Receive the work consent signal sent by the task device and generate a work task instruction; Control the work vehicle in the work task information to perform the task, and receive the work end signal sent by the task equipment.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by an electronic device, it implements the method as described in any one of claims 9 to 10.

Images