Multi-system interactive control methods, devices, electronic equipment, and storage media

Abstract

This disclosure provides a multi-system interactive control method, apparatus, and electronic device. The method includes: determining the processing frequency of each system in the multi-system interaction in response to the same event node; allocating clock trigger signals corresponding to the processing frequencies of each system according to their respective processing frequencies, with each system in the multi-system processing data using its own clock trigger signal; and triggering the systems in the multi-system interaction to synchronize their data processing if the data processing is asynchronous. This disclosure can guarantee synchronization of data processing between systems, simulate scenarios at any given time, and simulate corresponding scenarios for past and future time periods.

Description

Technical Field

[0001] This disclosure relates to multi-system interactive control technology, and more particularly to a multi-system interactive control method, device, electronic device, and storage medium. Background Technology

[0002] Currently, autonomous driving technology is maturing, but in reality, it remains in the experimental stage due to factors such as waiting for government approval and potential legal disputes. Besides the need for human control on roads...

[0003] In addition to simulations, there are also various autonomous driving simulation systems that use simulation platforms and autonomous driving platforms to simulate the operation of vehicles in the environment. However, in current simulation systems, the running speed of the simulation physical model depends on the hardware computing power and system load, and the results cannot be accurately reproduced, nor can they adapt to the simulation of real hardware-in-the-loop computing; moreover, the smoothness of the simulation is constrained by the computing power of autonomous driving, resulting in a poor simulation experience. Summary of the Invention

[0004] The present invention discloses a multi-system interactive control method, device, electronic device and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this disclosure, a multi-system interactive control method is provided, wherein the multi-systems are configured with a common clock source, the method comprising:

[0006] Determine the processing frequency of each system in a multi-system response to the same event node;

[0007] 0. Based on the processing frequency of each system in the multi-system, a clock trigger signal corresponding to the processing frequency of each system is allocated, and each system in the multi-system performs data processing with its own clock trigger signal.

[0008] If it is determined that the data processing of each system is not synchronized, the systems in the multi-system framework will be triggered to adjust to synchronized data processing.

[0009] After the systems in the multi-system framework are synchronized for data processing, the method further includes:

[0010] Receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal;

[0011] In response to the triggering time of the same event node, the data processing results of each system in the multi-systems corresponding to the triggering time are fused together.

[0012] In some possible implementations, the method further includes:

[0013] The frequency of the clock trigger signal allocated to each system in the multi-system is adjusted proportionally to obtain the data processing results that are ahead or behind the same event node or set time. The data processing results of each system are then merged to perform ahead or behind simulation.

[0014] In some possible implementations, the method further includes:

[0015] The system controls the interruption of the clock trigger signal output to each system in the multi-system, obtains the data processing results of the same event node corresponding to the interruption time, merges the obtained data processing results of each system, and performs interruption simulation.

[0016] In some possible implementations, allocating clock trigger signals corresponding to the processing frequency of each system in the multi-system configuration includes:

[0017] Based on the computing power of each system in the multi-system, the computational step size of each system in the multi-system in response to the same event node is determined, the processing frequency of each system in the multi-system is determined based on the computational step size, and a clock trigger signal corresponding to the processing frequency of each system is assigned to each system in the multi-system.

[0018] In some implementations, in response to asynchronous data processing in multiple systems, the clock trigger signal output by the clock source to the asynchronous system is adjusted according to the computing power of the asynchronous system, thereby triggering the asynchronous system to synchronize its data processing with that of other systems.

[0019] In some possible implementations, the method further includes:

[0020] The fused data processing results are sent to the designated system in the multi-system.

[0021] In some possible implementations, the multi-system includes at least an autonomous driving simulation platform and a simulation platform for real-world environments.

[0022] According to a second aspect of this disclosure, a multi-system interactive control device is provided, comprising:

[0023] A configuration unit is used to configure a common clock source for the multiple systems;

[0024] The determining unit is used to determine the processing frequency of each system in a multi-system response to the same event node;

[0025] The allocation unit is used to allocate clock trigger signals corresponding to the processing frequency of each system in the multi-system, and each system in the multi-system performs data processing with its own clock trigger signal.

[0026] The triggering unit is used to trigger each system in the multi-system to adjust to synchronize data processing when the data processing of each system is not synchronized.

[0027] In some possible implementations, the device further includes:

[0028] The receiving unit is used to receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal;

[0029] The fusion unit is used to fuse the data processing results of each system in the multi-system that correspond to the trigger time in response to the trigger time of the same event node.

[0030] In some possible implementations, the device further includes:

[0031] An adjustment unit is used to proportionally adjust the frequency of the clock trigger signal allocated to each system in the multi-system configuration.

[0032] The fusion unit is also used to obtain the data processing results of the same event node corresponding to the interruption time, fuse the obtained data processing results of each system, and perform interruption simulation.

[0033] In some possible implementations, the device further includes:

[0034] A control unit is used to control the interruption of the clock trigger signal output to each system in the multi-system;

[0035] The fusion unit is also used to obtain the data processing results of the same event node corresponding to the interruption time, fuse the obtained data processing results of each system, and perform interruption simulation.

[0036] In some possible implementations, the allocation unit is further configured to:

[0037] Based on the computing power of each system in the multi-system, the computational step size of each system in the multi-system in response to the same event node is determined, the processing frequency of each system in the multi-system is determined based on the computational step size, and a clock trigger signal corresponding to the processing frequency of each system is assigned to each system in the multi-system.

[0038] In some possible implementations, the allocation unit is further configured to:

[0039] In response to asynchronous data processing in multiple systems, the clock source adjusts the clock trigger signal output to the asynchronous system based on the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with other systems.

[0040] In some possible implementations, the device further includes:

[0041] The sending unit is used to send the fused data processing results to a designated system in the multi-system.

[0042] In some possible implementations, the multi-system includes at least an autonomous driving simulation platform and a simulation platform for real-world environments.

[0043] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0044] At least one processor; and

[0045] A memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the multi-system interactive control method described in this disclosure.

[0047] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the steps of the multi-system interactive control method described in this disclosure.

[0048] This disclosure discloses a multi-system interactive control method, apparatus, device, and storage medium. By configuring a common clock source for multiple systems and fusing the data results of each system corresponding to the trigger time of the same event node, the results are output as simulation data. Since this disclosure uses a common clock source with a higher processing frequency than each system, clock signals matching the processing frequency of each system can be set. Each system processes data according to its own frequency. If the data processing of the systems is not synchronized, the system is adjusted to synchronize its data processing. When data fusion is required, the processing data of each system corresponding to the set time is fused and output as the output data. This disclosure ensures synchronization of data processing between systems, enabling simulation of scenarios at any time, including past and future time periods. This disclosure combines the computing power of each system in the simulation and controls the calculation step size of each system through trigger signals to achieve a balance between accuracy and speed. This achieves a sense of time consistent with the real physical world and allows for proportional acceleration and deceleration between autonomous driving and simulation by controlling the signal source frequency. It also allows for the entire autonomous driving system to enter a breakpoint by controlling the on / off state of the signal source, supporting reproducible debugging.

[0049] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0050] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0051] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0052] Figure 1 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 1 ;

[0053] Figure 2 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 1 ;

[0054] Figure 3 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 2 ;

[0055] Figure 4 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 3 ;

[0056] Figure 5 A schematic diagram illustrating the implementation of the multi-system interactive control method according to an embodiment of this disclosure is shown;

[0057] Figure 6 A schematic diagram of the composition structure of the multi-system interactive control device according to an embodiment of the present disclosure is shown;

[0058] Figure 7 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0059] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0060] Figure 1 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 1 ,like Figure 1 As shown, the multi-system interactive control method of this disclosure includes the following processing steps:

[0061] Step 101: Determine the processing frequency of each system in the multi-system response to the same event node.

[0062] In this embodiment of the disclosure, a common clock source is configured for each system in the multi-system configuration. Specifically, by setting a common clock source, clock trigger signals corresponding to the processing frequency of each system are allocated, enabling each system to process data using its own clock trigger signal and send the processed data results to the central processing module at the corresponding response node.

[0063] By using a common clock source, a processing frequency that is compatible with the data processing frequency of each system in the multi-system can be set.

[0064] In this embodiment of the disclosure, as an example, the multiple systems may include an autonomous driving simulation system, such as a simulation platform represented by Carla, and an autonomous driving simulation system that collaboratively uses autonomous driving platforms represented by ROS / ROS2. It may also include a service platform and numerous access terminals, such as an online game system. That is, the technical solutions of this embodiment of the disclosure can be applied to real-time processing systems that require the fusion of data results processed by multiple processing systems.

[0065] Step 102: Based on the processing frequency of each system in the multi-system, allocate clock trigger signals corresponding to the processing frequency of each system, and each system in the multi-system performs data processing using its own clock trigger signal.

[0066] In this embodiment of the disclosure, after a common clock source is set, the common clock source is used as the clock trigger signal for data processing of each system in the multi-system.

[0067] As an example, the processing frequency of each system responding to the same event node can be set to the maximum processing frequency of each system itself, or the processing frequency of each system responding to the same event node can be set to 0.5 times, 0.4 times, 0.2 times, 0.1 times, etc. of the maximum processing frequency of each system.

[0068] Step 103: If it is determined that the data processing of each system is not synchronized, trigger the systems in the multi-system to adjust to synchronized data processing.

[0069] In this embodiment of the disclosure, when it is determined that the data processing of various systems is asynchronous, one or more systems in the multi-system framework can be used as the synchronization standard for data processing. The data processing progress of other systems can then be synchronized to the corresponding standard system, achieving synchronization of the entire system. One implementation method is to directly send synchronization instructions to other systems to be synchronized based on the data processing progress of the standard system, causing the other systems to synchronize with the standard system's data processing progress. Alternatively, another implementation method is to send synchronization instructions to the core system (such as a backend processing system) in the multi-system framework, which then sends synchronization instructions to other systems to be synchronized, synchronizing the data processing progress of the other systems with the backend processing system. Yet another implementation method is to send synchronization instructions to the systems to be synchronized, causing them to actively initiate data processing progress synchronization with the core system (such as a backend processing system), thus maintaining data processing synchronization among the systems in the multi-system framework.

[0070] In this embodiment of the disclosure, when it is determined that the data processing of each system is asynchronous, the computing power of the asynchronous system can be reassessed, and the clock trigger signal output by the clock source to the asynchronous system can be adjusted according to the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with that of other systems. This synchronization of data processing can be triggered by the clock source, by another system, or by synchronization between the different system and another system, where any one of these systems refers to the previously synchronized system.

[0071] Figure 2 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 2 ,like Figure 2 As shown, the multi-system interactive control method of this disclosure includes the following processing steps:

[0072] Step 201: Determine the processing frequency of each system in the multi-system response to the same event node.

[0073] In this embodiment of the disclosure, a common clock source is configured for each system in the multi-system configuration. Specifically, by setting a common clock source, clock trigger signals corresponding to the processing frequency of each system are allocated, enabling each system to process data using its own clock trigger signal and send the processed data results to the central processing module at the corresponding response node.

[0074] By using a common clock source, a processing frequency that is compatible with the data processing frequency of each system in the multi-system can be set.

[0075] In this embodiment of the disclosure, as an example, the multiple systems may include an autonomous driving simulation system, such as a simulation platform represented by Carla, and an autonomous driving simulation system that collaboratively uses autonomous driving platforms represented by ROS / ROS2. It may also include a service platform and numerous access terminals, such as an online game system. That is, the technical solutions of this embodiment of the disclosure can be applied to real-time processing systems that require the fusion of data results processed by multiple processing systems.

[0076] Step 202: Based on the processing frequency of each system in the multi-system, allocate clock trigger signals corresponding to the processing frequency of each system, and each system in the multi-system performs data processing using its own clock trigger signal.

[0077] In this embodiment of the disclosure, after a common clock source is set, the common clock source is used as the clock trigger signal for data processing of each system in the multi-system.

[0078] As an example, the processing frequency of each system responding to the same event node can be set to the maximum processing frequency of each system itself, or the processing frequency of each system responding to the same event node can be set to 0.5 times, 0.4 times, 0.2 times, 0.1 times, etc. of the maximum processing frequency of each system.

[0079] Step 203: If it is determined that the data processing of each system is not synchronized, trigger each system in the multi-system to adjust to synchronized data processing.

[0080] In this embodiment of the disclosure, when it is determined that the data processing of various systems is asynchronous, one or more systems in the multi-system framework can be used as the synchronization standard for data processing. The data processing progress of other systems can then be synchronized to the corresponding standard system, achieving synchronization of the entire system. One implementation method is to directly send synchronization instructions to other systems to be synchronized based on the data processing progress of the standard system, causing the other systems to synchronize with the standard system's data processing progress. Alternatively, another implementation method is to send synchronization instructions to the core system (such as a backend processing system) in the multi-system framework, which then sends synchronization instructions to other systems to be synchronized, synchronizing the data processing progress of the other systems with the backend processing system. Yet another implementation method is to send synchronization instructions to the systems to be synchronized, causing them to actively initiate data processing progress synchronization with the core system (such as a backend processing system), thus maintaining data processing synchronization among the systems in the multi-system framework.

[0081] In this embodiment of the disclosure, when it is determined that the data processing of each system is asynchronous, the computing power of the asynchronous system can be reassessed, and the clock trigger signal output by the clock source to the asynchronous system can be adjusted according to the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with that of other systems. This synchronization of data processing can be triggered by the clock source, by another system, or by synchronization between the different system and another system, where any one of these systems refers to the previously synchronized system.

[0082] Step 204: Receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal.

[0083] In this embodiment of the disclosure, after each system in the multi-system generates a corresponding data processing result based on its own clock trigger signal, the result of the processed data is sent to the data center processing module so that the data center processing module responds to the simulation command, merges the data processing results sent by each system in the multi-system to generate simulation data, and outputs it to the corresponding output interface.

[0084] Step 205: In response to the triggering time of the same event node, merge the data processing results of each system in the multi-system that correspond to the triggering time.

[0085] In this embodiment, the data processing results of each system are fused as needed. This fusion can be a simulation fusion of data processing results triggered by the simulation cycle, or a simulation fusion of data processing results in response to an external fusion command. As an example, when the multiple systems are Carla and ROS systems, the scene in the cockpit and the scene in real time can be fused based on the real physical world scene returned by Carla and the simulated cockpit scene returned by the ROS system, and used as the output of the driving simulation. This can realize the simulation of autonomous driving and improve the user experience.

[0086] In this embodiment of the disclosure, the fused data processing result can be sent to a designated system in the multi-system, so that the designated system outputs the fused data processing result, which can be a simulated image or simulated video, etc.

[0087] In this embodiment of the disclosure, the same event may be the arrival of the fusion moment or the arrival of the fusion command, etc.

[0088] Figure 3 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 3 ,like Figure 3 As shown, the multi-system interactive control method of this disclosure includes the following processing steps:

[0089] Step 301: Determine the processing frequency of each system in the multi-system response to the same event node.

[0090] In this embodiment of the disclosure, a common clock source is configured for each system in the multi-system configuration. Specifically, by setting a common clock source, clock trigger signals corresponding to the processing frequency of each system are allocated, enabling each system to process data using its own clock trigger signal and send the processed data results to the central processing module at the corresponding response node.

[0091] By using a common clock source, a processing frequency that is compatible with the data processing frequency of each system in the multi-system can be set.

[0092] Step 302: Adjust the frequency of the clock trigger signal allocated to each system in the multi-system proportionally.

[0093] In this embodiment of the disclosure, a common clock source is configured for each system in the multi-system configuration. Specifically, by setting a common clock source, clock trigger signals corresponding to the processing frequency of each system are allocated, enabling each system to process data using its own clock trigger signal and send the processed data results to the central processing module at the corresponding response node.

[0094] By using a common clock source, the processing frequency of each system responding to the same event node is set to 0.5 times, 0.4 times, 0.2 times, 0.1 times, etc. of the maximum processing frequency of each system, so as to achieve proportional adjustment of the processing frequency of each system responding to the same event node.

[0095] In this embodiment of the disclosure, as an example, the multiple systems may include an autonomous driving simulation system, such as a simulation platform represented by Carla, and an autonomous driving simulation system that collaboratively uses autonomous driving platforms represented by ROS / ROS2. It may also include a service platform and numerous access terminals, such as an online game system. That is, the technical solutions of this embodiment of the disclosure can be applied to real-time processing systems that require the fusion of data results processed by multiple processing systems.

[0096] Step 303: Obtain the data processing results that are ahead or behind the same event node or time.

[0097] In this embodiment of the disclosure, after proportionally adjusting the frequency of the clock trigger signals allocated to each system in the multi-system configuration, the simulation data can be ahead of or behind the time by speeding up or slowing down the frequency of the clock trigger signals allocated to each system. That is, by adjusting the processing frequency of each system, a simulated scenario of the vehicle speed and driving state in the simulated cockpit can be achieved that is ahead of or behind the time, allowing for simulation of driving scenarios that are ahead of or behind the time, as needed.

[0098] As an example, the processing frequency of each system responding to the same event node is set to 0.5 times, 0.4 times, 0.2 times, 0.1 times, etc., the maximum processing frequency of each system. This makes the simulation data of each system in the multi-system lead or lag behind the simulation scenario corresponding to the vehicle speed driving process in the cockpit, so as to realize the driving simulation of autonomous driving.

[0099] Step 304: The data processing results of each system are merged to perform advanced or delayed simulations.

[0100] In this embodiment of the disclosure, after each system in the multi-system generates a corresponding data processing result based on its own clock trigger signal, the result of the processed data is sent to the data center processing module so that the data center processing module responds to the simulation command, merges the data processing results sent by each system in the multi-system to generate simulation data, and outputs it to the corresponding output interface.

[0101] In this embodiment of the disclosure, the data processing results of each system are fused as needed. This fusion can be based on the simulation fusion of the data processing results triggered by the simulation cycle. For example, after receiving the data processing results sent by the system with the lowest processing frequency, the data processing results can be fused and the corresponding fused simulation image can be output to realize the simulation of autonomous driving.

[0102] In this embodiment of the disclosure, the fused data processing result can be sent to a designated system in the multi-system, so that the designated system outputs the fused data processing result, which can be a simulated image or simulated video, etc.

[0103] Figure 4 This illustration shows the implementation flow of the multi-system interactive control method according to an embodiment of the present disclosure. Figure 4 ,like Figure 4 As shown, the multi-system interactive control method of this disclosure includes the following processing steps:

[0104] Step 401: Determine the processing frequency of each system in the multi-system response to the same event node.

[0105] In this embodiment of the disclosure, a common clock source is configured for each system in the multi-system configuration. Specifically, by setting a common clock source, clock trigger signals corresponding to the processing frequency of each system are allocated, enabling each system to process data using its own clock trigger signal and send the processed data results to the central processing module at the corresponding response node.

[0106] By using a common clock source, a processing frequency that is compatible with the data processing frequency of each system in the multi-system can be set.

[0107] In this embodiment of the disclosure, as an example, the multiple systems may include an autonomous driving simulation system, such as a simulation platform represented by Carla, and an autonomous driving simulation system that collaboratively uses autonomous driving platforms represented by ROS / ROS2. It may also include a service platform and numerous access terminals, such as an online game system. That is, the technical solutions of this embodiment of the disclosure can be applied to real-time processing systems that require the fusion of data results processed by multiple processing systems.

[0108] Step 402: Based on the processing frequency of each system in the multi-system, allocate a clock trigger signal corresponding to the processing frequency of each system, and each system in the multi-system performs data processing with its own clock trigger signal.

[0109] In this embodiment, after setting a common clock source, this common clock source is used as the clock trigger signal for data processing in each system of the multi-system system. Specifically, it is necessary to determine the calculation step size of each system in the multi-system system in response to the same event node based on the computing power of each system in the multi-system system, determine the processing frequency of each system in the multi-system system based on the calculation step size, and allocate a clock trigger signal corresponding to the processing frequency of each system in the multi-system system. As an example, the processing frequency of each system can be determined based on the calculation step size of each system processing its own data and sending the data processing result to the central processing module according to the same simulation instruction. It is necessary to determine the response step size of all systems in the multi-system system for processing their corresponding data, and then set the processing frequency of each system in a coordinated manner according to the set processing data step size of each system. As an example, if the processing step size of system 1 is 20ms and the processing step size of system 2 is 10ms, the processing frequency of each system in response to the same event node can be set to 1s / 20ms, etc. Of course, as an example, it can also be set to 1s / 40ms, 1s / 60ms, or 1s / 80ms, etc. This means that the processing frequency set for each system needs to meet the needs of the system with the slowest processing speed, ensuring that each system can complete data processing during data fusion. Of course, the processing frequency can be set according to the processing step size of each system. If the processing step size of system 1 is 10ms and the processing step size of system 2 is 5ms, the processing frequency of system 1 responding to the same event node can be set to 1s / 10ms, and the processing frequency of system 2 responding to the same event node can be set to 1s / 5ms, and so on.

[0110] Step 403: Control the interruption of the clock trigger signal output to each system in the multi-system, and obtain the data processing result of the same event node corresponding to the interruption time.

[0111] In this embodiment of the disclosure, after each system in the multi-system generates a corresponding data processing result based on its own clock trigger signal, the result of the processed data is sent to the data center processing module so that the data center processing module responds to the simulation command, merges the data processing results sent by each system in the multi-system to generate simulation data, and outputs it to the corresponding output interface.

[0112] Upon receiving an interrupt triggered by a clock signal, the system obtains the data processing results of each system at the time corresponding to the interrupt, which are used as the interrupt simulation results.

[0113] This disclosure supports the output of simulation results at any given time. When it is necessary to output the simulation results at a certain time, an interrupt command can be sent to each system. In response to the interrupt command, each system sends the current data processing results to the central processing module, so that the central processing module can fuse the data processing results corresponding to the interrupt time to generate the simulation image at the corresponding interrupt time.

[0114] Step 404: The data processing results of each system are merged and interrupt simulation is performed.

[0115] In this embodiment of the disclosure, the data processing results at the time of interruption of each system are fused to obtain the corresponding simulation image.

[0116] The following specific examples further illustrate the essence of the technical solutions in the embodiments of this disclosure.

[0117] For current simulation systems, such as simulation platforms represented by Carla and autonomous driving platforms like ROS / ROS2 collaborating in autonomous driving simulation systems, the use of their own clock signals by each system leads to two main simulation modes: asynchronous and synchronous. In asynchronous mode, the simulation platform and the autonomous driving platform run asynchronously, with the simulation speed determined by the computing power of the simulation computer. In synchronous mode, the simulation platform and the autonomous driving platform run synchronously, with the simulation speed determined by the frequency of the autonomous driving platform's output data (typically vehicle control data). Thus, in asynchronous mode, the running speed of the simulation physical model within the simulation platform depends on the hardware computing power and system load, resulting in inaccurate simulation results and an inability to adapt to real hardware-in-the-loop simulations. In synchronous mode, the smoothness of the simulation depends on whether the simulation physical model within the simulation platform can undergo sufficient iterative computation, constrained by the algorithms and computing power of the autonomous driving platform. The embodiments disclosed herein use a common clock source with a higher frequency to serve as the clock trigger signal for each system in the multi-system. The common clock source triggers the clock signals for each system in the multi-system. High-frequency timing messages are used as the trigger signal source, and frequency division and routing are applied to the simulation physical model and autonomous driving platform in the simulation platform. By controlling the signal source, the rhythm of the entire autonomous driving simulation is precisely controlled, and fully reproducible and accurate debugging is achieved.

[0118] Figure 5 A schematic diagram illustrating the implementation of the multi-system interactive control method according to an embodiment of this disclosure is shown, such as... Figure 5 As shown in the embodiments of this disclosure, the implementation of multi-system interactive control includes a high-frequency software clock source that can run on any computer, such as the autonomous driving platform or simulation platform itself, and sends messages in the form of timed messages. The divider represents a software frequency divider, which can divide the high-frequency clock source using a counting method and provide it to the scene physical model in the autonomous driving platform or simulation platform. This disclosure implements the configuration of the aforementioned clock source and frequency divider signals in software. The frequency of the clock trigger signals of each system can be set as needed to control the simulation results of the system. For example, it supports the output of leading or lagging simulation results as described in the previous embodiments, and can obtain the simulation results at the interruption time through interrupt instructions. Therefore, the technical solution of this disclosure can flexibly configure the clock source switch and frequency divider frequency, achieving precise control of the simulation results and enabling multi-mode simulation according to user needs.

[0119] This disclosure, through its embodiments, sets a shared clock source for each system, allowing each system to be configured with a clock signal matching its processing frequency. Each system processes data according to its own frequency, and when data fusion is required, the processed data from each system at a set time is merged and output as the final output data. This disclosure can simulate scenarios at any given time, including past and future time periods. By combining the computing power of each system in the simulation and controlling the calculation step size of each system through trigger signals, this disclosure achieves a balance between accuracy and speed, providing a sense of time consistent with the real physical world. It can also control the acceleration and deceleration of autonomous driving and simulation proportionally by controlling the signal source frequency; and it can control the on / off state of the signal source to allow the entire autonomous driving system to enter a breakpoint, supporting reproducible debugging.

[0120] Figure 6 A schematic diagram of the composition structure of the multi-system interactive control device according to an embodiment of the present disclosure is shown, such as... Figure 6 As shown, the multi-system interactive control device of this disclosure includes:

[0121] Configuration unit 60 is used to configure a common clock source for the multiple systems;

[0122] The determining unit 61 is used to determine the processing frequency of each system in the multi-system response to the same event node;

[0123] Allocation unit 62 is used to allocate clock trigger signals corresponding to the processing frequency of each system in the multi-system according to the processing frequency of each system in the multi-system, and each system in the multi-system performs data processing with its own clock trigger signal.

[0124] Triggering unit 63 is used to trigger each system in the multi-system to adjust to synchronize data processing when the data processing of each system is not synchronized.

[0125] As one implementation method, such as Figure 6 As shown, the multi-system interactive control device in this embodiment of the present disclosure further includes:

[0126] The receiving unit 64 is used to receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal;

[0127] The fusion unit 65 is used to fuse the data processing results of each system in the multi-system that correspond to the trigger time in response to the trigger time of the same event node.

[0128] As one implementation method, in the aforementioned Figure 6 Based on the multi-system interactive control device shown, the multi-system interactive control device of this disclosure embodiment further includes:

[0129] Adjustment unit ( Figure 6 (not shown in the image), used to proportionally adjust the frequency of the clock trigger signal allocated to each system in the multi-system;

[0130] The fusion unit 65 is also used to obtain the data processing results of the same event node corresponding to the interruption time, and to fuse the obtained data processing results of each system to perform interruption simulation.

[0131] As one implementation method, in the aforementioned Figure 6 Based on the multi-system interactive control device shown, the multi-system interactive control device of this disclosure embodiment further includes:

[0132] Control unit ( Figure 6 (Not shown in the image), used to control the interruption of the clock trigger signal output to each system in the multi-system;

[0133] The fusion unit 65 is also used to obtain the data processing results of the same event node corresponding to the interruption time, and to fuse the obtained data processing results of each system to perform interruption simulation.

[0134] In some possible implementations, the allocation unit 62 is further configured to:

[0135] Based on the computing power of each system in the multi-system, the computational step size of each system in the multi-system in response to the same event node is determined, the processing frequency of each system in the multi-system is determined based on the computational step size, and a clock trigger signal corresponding to the processing frequency of each system is assigned to each system in the multi-system.

[0136] In some possible implementations, the allocation unit 62 is further configured to:

[0137] In response to asynchronous data processing in multiple systems, the clock source adjusts the clock trigger signal output to the asynchronous system based on the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with other systems.

[0138] As one implementation method, in the aforementioned Figure 6 Based on the multi-system interactive control device shown, the multi-system interactive control device of this disclosure embodiment further includes:

[0139] Transmitting unit ( Figure 6 (Not shown in the image), used to send the fused data processing results to a designated system in the multi-system.

[0140] In some possible implementations, the multi-system includes at least an autonomous driving simulation platform and a simulation platform for real-world environments.

[0141] In an exemplary embodiment, the configuration unit 60, determination unit 61, allocation unit 62, triggering unit 63, receiving unit 64, fusion unit 65, adjustment unit, control unit, and sending unit may be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components.

[0142] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0143] According to embodiments of this disclosure, this disclosure also describes an electronic device and a readable storage medium.

[0144] Figure 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0145] like Figure 7As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0146] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as multi-system interaction control methods. For example, in some embodiments, the multi-system interaction control method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the multi-system interaction control method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the steps of the multi-system interaction control method by any other suitable means (e.g., by means of firmware).

[0148] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0152] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0153] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0154] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0155] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0156] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A multi-system interactive control method, characterized in that, The multiple systems are configured with a common clock source, which has a higher processing frequency than each system, to set a clock signal for each system that matches its processing frequency; the method includes: Determine the processing frequency of each system in the multi-system response to the same event node; set the processing frequency of each system in response to the same event node to the maximum processing frequency of each system, or set it to a certain multiple of the maximum processing frequency of each system; the multiple is greater than 0 and less than 1. Based on the processing frequency of each system in the multi-system, a clock trigger signal corresponding to the processing frequency of each system is allocated, and each system in the multi-system performs data processing with its own clock trigger signal; based on the computing power of each system, the calculation step size of each system is controlled by the trigger signal to achieve a sense of time consistent with the real physical world. If it is determined that the data processing of each system is not synchronized, the systems in the multi-system framework will be triggered to adjust to synchronized data processing.

2. The method according to claim 1, characterized in that, After the systems in the multi-system framework are adjusted to synchronize data processing, the method further includes: Receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal; In response to the triggering time of the same event node, the data processing results of each system in the multi-systems corresponding to the triggering time are fused together.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The frequency of the clock trigger signal allocated to each system in the multi-system is adjusted proportionally to obtain the data processing results that are ahead or behind the same event node or set time. The data processing results of each system are then merged to perform ahead or behind simulation.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The system controls the interruption of the clock trigger signal output to each system in the multi-system, obtains the data processing results of the same event node corresponding to the interruption time, merges the obtained data processing results of each system, and performs interruption simulation.

5. The method according to claim 1 or 2, characterized in that, The clock trigger signal allocated to each system corresponding to its processing frequency includes: Based on the computing power of each system in the multi-system, the computational step size of each system in the multi-system in response to the same event node is determined, the processing frequency of each system in the multi-system is determined based on the computational step size, and a clock trigger signal corresponding to the processing frequency of each system is assigned to each system in the multi-system.

6. The method according to claim 5, characterized in that, The method further includes: In response to asynchronous data processing in multiple systems, the clock source adjusts the clock trigger signal output to the asynchronous system based on the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with other systems.

7. The method according to claim 2, characterized in that, The method further includes: The fused data processing results are sent to the designated system in the multi-system.

8. The method according to claim 1 or 2, characterized in that, The multi-system includes at least an autonomous driving simulation platform and a simulation platform for real-world environments.

9. A multi-system interactive control device, characterized in that, The device includes: A configuration unit is used to configure a common clock source for the multiple systems; the clock source is higher than the processing frequency of each system, so as to set a clock signal that matches the processing frequency of each system. The determining unit is used to determine the processing frequency of each system in the multi-system response to the same event node; and to set the processing frequency of each system in response to the same event node to its own maximum processing frequency, or to set it to a certain set multiple of the maximum processing frequency of each system; the set multiple is greater than 0 and less than 1. The allocation unit is used to allocate clock trigger signals corresponding to the processing frequency of each system in the multi-system configuration. Each system in the multi-system configuration performs data processing with its own clock trigger signal. Based on the computing power of each system, the calculation step size of each system is controlled by the trigger signal to achieve a sense of time consistent with the real physical world. The triggering unit is used to trigger each system in the multi-system to adjust to synchronize data processing when the data processing of each system is not synchronized.

10. The apparatus according to claim 9, characterized in that, The device further includes: The receiving unit is used to receive the data processing results triggered by each system in the multi-system based on its own clock trigger signal; The fusion unit is used to fuse the data processing results of each system in the multi-system that correspond to the trigger time in response to the trigger time of the same event node.

11. The apparatus according to claim 10, characterized in that, The device further includes: An adjustment unit is used to proportionally adjust the frequency of the clock trigger signal allocated to each system in the multi-system configuration. The fusion unit is also used to obtain the data processing results of the same event node corresponding to the interruption time, fuse the obtained data processing results of each system, and perform interruption simulation.

12. The apparatus according to claim 10, characterized in that, The device further includes: A control unit is used to control the interruption of the clock trigger signal output to each system in the multi-system; The fusion unit is also used to obtain the data processing results of the same event node corresponding to the interruption time, fuse the obtained data processing results of each system, and perform interruption simulation.

13. The apparatus according to claim 9 or 10, characterized in that, The allocation unit is further configured to: Based on the computing power of each system in the multi-system, the computational step size of each system in the multi-system in response to the same event node is determined, the processing frequency of each system in the multi-system is determined based on the computational step size, and a clock trigger signal corresponding to the processing frequency of each system is assigned to each system in the multi-system.

14. The apparatus according to claim 13, characterized in that, The allocation unit is further configured to: In response to asynchronous data processing in multiple systems, the clock source adjusts the clock trigger signal output to the asynchronous system based on the computing power of the asynchronous system, triggering the asynchronous system to synchronize its data processing with other systems.

15. The apparatus according to claim 10, characterized in that, The device further includes: The sending unit is used to send the fused data processing results to a designated system in the multi-system.

16. The apparatus according to claim 9 or 10, characterized in that, The multi-system includes at least an autonomous driving simulation platform and a simulation platform for real-world environments.

17. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the multi-system interactive control method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the steps of the multi-system interactive control method according to any one of claims 1 to 8.

Images