Heterogeneous model joint simulation system and synchronization method based on virtual scene

Abstract

The invention discloses a heterogeneous model joint simulation system and synchronization method based on a virtual scene, and the system comprises a Ros communication module which is used for transmitting a simulation result issued by an FMU simulation system to a control strategy end, and transmitting control information issued by the control strategy end to the FMU simulation system; the FMU simulation system is used for converting the control information published by the control strategy end into FMU simulation parameters, driving the FMU to execute simulation, and converting simulation results, simulation time, a three-dimensional model of a virtual scene system and environment information into message bodies corresponding to subscribers to be published; and the virtual scene system is used for providing a three-dimensional model and environment information. The simulation system provided by the invention can reduce the complexity of model integration and optimize the simulation efficiency through an FMU integration mode. According to the synchronization method based on the simulation system, real-time data transmission and response of the simulation model and the algorithm can be ensured, so that the accuracy and the reliability of the algorithm are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a virtual scene-based heterogeneous model joint simulation system and a synchronization method. Background Art

[0002] With the advancement of digital transformation, the application of system simulation and virtual reality technology in engineering design, testing and optimization has gradually gained attention. Especially in the research of complex systems (such as electromechanical equipment, vehicle power systems, etc.), the traditional single simulation method can no longer meet the demand for high restoration of real scenes. Therefore, joint simulation has become a development trend.

[0003] In this context, FMU (Functional Mock-up Unit), as an open standard, provides the ability for interoperability between different engineering tools, enables seamless integration of multiple modeling environments, and greatly promotes cross-disciplinary and cross-platform joint simulation.

[0004] Combining virtual scenes with FMU models involves multiple aspects such as graphics rendering, sensor simulation, and human-computer interaction. Modern virtual reality technology can visualize complex physical models and provide an intuitive operating interface through high-performance computing and graphics processing. By transmitting dynamic data in heterogeneous models to virtual scenes in real time, an intuitive display of system behavior can be achieved, greatly improving the user's understanding and decision-making ability. In addition, simulations of sensors and actuators can be embedded in virtual scenes for real-time control and feedback. This real-time interaction can not only be used for system performance evaluation, but also help develop more intelligent control strategies to achieve better performance in practical applications.

[0005] However, when performing joint simulation between FMU models and virtual scenes, the integration of multiple heterogeneous models requires adjustment of model positions and connections. The integration of various heterogeneous models is relatively complex, which increases time and technical costs. The simulation results of heterogeneous models are analyzed by viewing data text and curves. The simulation of heterogeneous models only has simple data and cannot intuitively display the system behavior. Due to the complexity of the system itself, the simulation speed is slow, resulting in the control strategy executing multiple steps, but the model simulation only completes a single step, which distorts the control strategy and affects the accuracy of the control strategy test. Summary of the invention

[0006] The purpose of the present invention is to provide a virtual scene-based heterogeneous model joint simulation system and synchronization method to solve the problems of high complexity of system model integration and time inconsistency between the simulation end and the algorithm end in current multidisciplinary simulation. The system can reduce the complexity of model integration and optimize simulation efficiency through FMU integration.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] In one aspect, the present invention provides a virtual scene-based heterogeneous model joint simulation system, comprising:

[0009] Ros communication module is used for communication between the joint simulation system and the control strategy end, transferring the simulation results published by the FMU simulation system to the control strategy end, and transferring the control information published by the control strategy end to the FMU simulation system;

[0010] The FMU simulation system is used to convert the control information published by the control strategy end into FMU simulation parameters, drive the FMU to perform simulation, and convert the simulation results, time, and three-dimensional model and environmental information of the virtual scene system into the corresponding message body of the subscriber for publication;

[0011] The mining truck virtual scene system is used to provide mining truck 3D models and environmental information.

[0012] Preferably, the FMU simulation system comprises:

[0013] RosComu module is used to receive the FMU simulation results, time and model status and environment information transmitted by the three-dimensional model of the mining truck transmitted by the Control module, convert them into the corresponding Ros message body of the subscriber for publication, and convert the control information published by the control strategy end into FMU simulation parameters and pass them to the Control module;

[0014] The Control module is used to obtain the simulation results and time from the FMU driver module and pass them to the RosComu module and the mining truck 3D model, as well as obtain the FMU simulation parameters from the RosComu module and assign them to the FMU driver module to control the execution of the FMU driver module;

[0015] The FMU driver module is used to execute the call of FMU, obtain FMU simulation parameters from the Control module, assign them to FMU and drive FMU to perform simulation, and pass the simulation results and time to the Control module.

[0016] Preferably, the Ros communication module is implemented via TCP or UDP.

[0017] Preferably, the RosComu module includes one or more Ros publishers and subscribers, and the corresponding Ros topics and message bodies are set according to requirements.

[0018] Preferably, the Control module can be used to drive multiple mining truck three-dimensional models.

[0019] In a second aspect, the present invention provides a synchronization method for virtual scene-based heterogeneous model joint simulation, which is implemented by the above-mentioned virtual scene-based heterogeneous model joint simulation system, and the method includes:

[0020] Establish the communication connection between the joint simulation system and the control strategy end, import the FMU and initialize it;

[0021] When subscribing to the control message of the control strategy end, the control information is converted into FMU simulation parameters through the RosComu module and passed to the Control module, and then passed to the FMU driver module through the Control module, and the simulation parameters are assigned to the corresponding FMU through the FMU driver module;

[0022] Each FMU performs a single-step simulation according to its own simulation parameters;

[0023] When all FMUs complete a step of simulation, the simulation results and time are obtained through the FMU driver module, and the simulation results, time, and the status and environmental information of the mining truck's three-dimensional model are converted into the corresponding Ros message body of the subscriber for publication through the RosComu module.

[0024] Preferably, the establishing of a communication connection between the joint simulation system and the control strategy terminal, importing the FMU and initializing it, includes:

[0025] Establish a communication connection between the joint simulation system and the control strategy end through the network communication protocol, and import And initialize, and set the simulation flag Assigns a value of false and sends an enable signal to the control strategy end ,in Expressed as indivual , express The number of

[0026] Preferably, the method further comprises:

[0027] After a single step of simulation is completed, the simulation is paused and the simulation flag is set to true.

[0028] Preferably, the method further comprises:

[0029] When all FMUs complete a step of simulation, the simulation results are obtained through the FMU driver module and assigned to the mining truck 3D model through the Control module. At the same time, the simulation flag is reset to false, and the simulation time is incremented by a single step.

[0030] Preferably, the size of the single step is adjusted according to simulation requirements.

[0031] Preferably, the data format of the simulation flag is set according to requirements.

[0032] Preferably, the status and environmental information of the three-dimensional model of the mining truck are set according to the simulation requirements, wherein the status information of the three-dimensional model of the mining truck includes posture information, angle information and pressure information; the environmental information is collected by virtual sensors, including lidar point cloud data, GPS positioning and camera information.

[0033] Preferably, the method further comprises:

[0034] After establishing the communication connection between the joint simulation system and the control strategy end, the control strategy end runs and the simulation time Set to zero;

[0035] Wait for the subscription enable signal. If the subscription is successful, execute the next step. Otherwise, wait.

[0036] Subscribe to simulation results and calculate the control strategy simulation time difference and reset the control strategy simulation time ,in, is the simulation time;

[0037] like , control strategy execution, and publish control strategy information, where, is the simulation step size.

[0038] Preferably, the control strategy end monitors the enable signal and simulation result issued by the simulation end during execution, and the control strategy can automatically subscribe when a corresponding message is issued.

[0039] Preferably, the control strategy simulation time difference Set to the simulation step size at the simulation end Consistent.

[0040] Beneficial effects brought by the technical solution of the present invention:

[0041] (1) The present invention proposes a virtual scene-based heterogeneous model joint simulation system, which can reduce the complexity of model integration and optimize simulation efficiency through FMU integration.

[0042] (2) The present invention proposes a synchronization method for joint simulation of heterogeneous models based on virtual scenes, which can ensure real-time data transmission and response between simulation models and algorithms, thereby improving the accuracy and reliability of the algorithms.

[0043] (3) The present invention adopts FMU and virtual scene joint simulation to enhance the user's visualization experience of simulation results, promote the innovative development of engineering design and verification process, and ultimately improve the product development quality and market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the architecture of a virtual scene-based heterogeneous model joint simulation system provided by the present invention; in the figure: 1-virtual scene-based heterogeneous model joint simulation system, 2-FMU simulation system, 3-virtual scene system;

[0045] Figure 2 In one embodiment of the present invention, whether the control strategy and the time axis of the simulation system under the synchronization method of the present invention are adopted, in which: (a) is the time axis of the control strategy and the simulation system under the synchronization method of the present invention is not adopted; (b) is the time axis of the control strategy and the simulation system under the synchronization method of the present invention;

[0046] Figure 3 A schematic flow chart of a synchronization method for virtual scene-based heterogeneous model joint simulation provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0048] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0049] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0050] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0051] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0052] It should be emphasized here that the step marks mentioned below are not intended to limit the order of the steps, but it should be understood that the steps can be executed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be executed simultaneously.

[0053] An embodiment of the present invention provides a virtual scene-based heterogeneous model joint simulation system 1, see Figure 1 , including: Ros communication module, FMU simulation system 2 and virtual scene system 3,

[0054] Among them, the FMU simulation system 2 includes the RosComu module, the Control module and the FMU driver module, and the virtual scene system includes the mining truck three-dimensional model and virtual sensors.

[0055] In the present invention, the Ros communication module is used to realize the communication between the joint simulation system and the control strategy end, and can transmit the simulation results published by the RosComu module to the control strategy end, and can transmit the control information published by the control strategy end to the RosComu module;

[0056] The RosComu module is used to receive the FMU simulation results, time, model and environment information transmitted by the three-dimensional model of the mining truck transmitted by the Control module, and can convert them into corresponding Ros message bodies for publication. At the same time, it can convert the control information published by the control strategy end into FMU simulation parameters and pass them to the Control module;

[0057] The Control module is used to obtain the simulation results and time from the FMU driver module and pass them to the RosComu module and the mining truck 3D model. It can also obtain the FMU simulation parameters from the RosComu module and assign them to the FMU driver module, and can control the execution of the FMU driver module.

[0058] The FMU driver module is used to call the FMU, which can initialize the FMU model. At the same time, it can obtain simulation parameters from the Control module, assign them to the FMU and drive the FMU to perform simulation, and can pass the simulation results and time to the Control module.

[0059] Based on this, the present invention provides an application example of a heterogeneous model joint simulation system of a mining truck virtual scene, which includes a Ros communication module, an FMU simulation system and a virtual scene system. The virtual scene system includes a mining truck three-dimensional model and virtual sensors such as cameras and laser radars.

[0060] The Ros communication module establishes a connection between the simulation system and the control strategy; the RosComu module transmits the simulation time, mining truck status information (engine speed, fuel tank temperature, hydraulic cylinder pressure, etc.) and environmental information perceived by the virtual sensor (point cloud data, image data, etc.) transmitted by the Control module to the control strategy through the Ros communication module, and can also convert the control information (throttle, steering, gear) issued by the control strategy into FMU simulation parameters and pass them to the Control module; the Control module obtains the FMU simulation results and time transmitted by the FMU drive module, and passes them to the RosComu module, and can control the mining truck posture according to the FMU simulation results, and can pass the FMU simulation parameters to the FMU drive module and drive the module to execute; the FMU drive module can initialize the mining truck FMU, assign values ​​to the FMU parameters and drive the FMU to execute, and pass the simulation results and time to the Control module.

[0061] In one embodiment of the present invention, the Ros communication module is implemented through TCP, UDP or other methods.

[0062] In one embodiment of the present invention, the RosComu module may include one or more Ros publishers and subscribers, and the corresponding Ros topics and message bodies may be set according to requirements.

[0063] In one embodiment of the present invention, the Control module can realize the driving of multiple mining truck three-dimensional models.

[0064] This embodiment proposes a heterogeneous model joint simulation system based on a virtual scene. The system can standardize the interfaces between different simulation tools and models through FMU integration, ensure the compatibility between models and the simplicity of data transmission, reduce the complexity of model integration, optimize simulation efficiency, and enhance the user's visualization experience of simulation results.

[0065] Based on the above heterogeneous model joint simulation system, the second embodiment of the present invention provides a synchronization method for heterogeneous model joint simulation based on a virtual scene, see Figure 3 , the method comprises the following steps:

[0066] Simulation side:

[0067] Step S1, establish a connection with the control strategy terminal through a network communication protocol, Import and initialize, and set the simulation flag Assigns a value of false and sends an enable signal to the control strategy end ,in Expressed as indivual , express the number of

[0068] Step S2, Execute a single step according to the respective simulation parameters Simulation; after the simulation is finished, pause the simulation and set the simulation flag Assigns a value of true;

[0069] Step S3, when the number of true values ​​of the simulation flag is When the simulation results are obtained through the FMU driver module And assign it to the mining card 3D model , set the simulation flag reset is false, and the FMU simulation time Increment single step , publish the simulation time to the control strategy end through the RosComu module , simulation results , the mining truck three-dimensional model status information and environmental information and continue to execute step S4. Otherwise, it falls into waiting;

[0070] Step S4: When subscribing to the control policy end control message When the control information is converted into FMU simulation parameters through the RosComu module and passed to the Control module, then to the FMU driver module through the Control module, and then to the FMU driver module through the FMU driver module. Assign to The simulation parameters are set and the process continues to execute step S2, otherwise the process falls into waiting.

[0071] In one embodiment of the present invention, the FMU in step S1 can be exported by various commercial software.

[0072] In one embodiment of the present invention, the single step length in step S2 is The size can be adjusted according to simulation requirements.

[0073] In one embodiment of the present invention, the simulation flag bit in step S2 Can be changed to other data formats, such as integers, strings, etc.

[0074] In one embodiment of the present invention, the three-dimensional model status information and scene information of the mining truck in step S3 can be set according to the simulation requirements, wherein the three-dimensional model status information of the mining truck includes but is not limited to the mining truck posture information, angle information, pressure information, etc.; the environmental information includes but is not limited to lidar point cloud data, GPS positioning and camera information, etc.

[0075] Control strategy side:

[0076] Step A1, control strategy operation, control strategy simulation time Zero adjustment;

[0077] Step A2, wait for the subscription enable signal, if the subscription enable signal is received, then execute step A3. Otherwise, the process continues to wait;

[0078] Step A3: Subscribe to simulation results and calculate the control strategy simulation time difference and reset the control strategy simulation time ,

[0079] Step A4: If , the control strategy is executed and continues to step A5, otherwise it returns to step A3;

[0080] Step A5: Release the control strategy information and continue to execute step A3.

[0081] In one embodiment of the present invention, the control strategy will always monitor the enable signal and simulation results issued by the simulation end when it is executed, and when a corresponding message is issued, the control strategy can automatically subscribe.

[0082] In one embodiment of the present invention, the control strategy simulation time difference in step A3 is Need to set the simulation step size of FMU on the simulation side Consistent.

[0083] This embodiment proposes a synchronization method for joint simulation of heterogeneous models based on virtual scenes, which can ensure real-time data transmission and response between simulation models and algorithms, thereby improving the accuracy and reliability of the algorithms.

[0084] Figure 2 FIG. 1 is a timeline comparison of the control strategy and the simulation system in an embodiment of the present invention when the synchronization method of the present invention is used or not. FIG. (a) is a timeline of the control strategy and the simulation system when the synchronization method of the present invention is not used. When the control strategy receives the three-dimensional model data, it performs calculation processing and then passes the next simulation parameter to the three-dimensional model. However, due to the slow model processing speed, the three-dimensional model completes one step calculation only after the control strategy executes multiple simulation steps with the data, thereby causing the control strategy and the three-dimensional model simulation to be out of sync, thereby causing the control strategy to be distorted. FIG. (b) is a timeline of the control strategy and the simulation system when the synchronization method of the present invention is used. When the control strategy receives the three-dimensional model data, it performs calculation processing. At this time, the three-dimensional model no longer performs the next simulation and performs a single-step simulation when receiving the control parameter. After the control strategy completes the calculation and passes the simulation parameter to the three-dimensional model, the control strategy no longer performs simulation and performs a single-step simulation after receiving the three-dimensional model data. This method ensures the synchronization of the control strategy and the three-dimensional model simulation and improves the accuracy and real-time performance of the algorithm test.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A virtual scene-based heterogeneous model joint simulation system, characterized in that: include: Ros communication module is used for communication between the joint simulation system and the control strategy end, transferring the simulation results published by the FMU simulation system to the control strategy end, and transferring the control information published by the control strategy end to the FMU simulation system; The FMU simulation system is used to convert the control information published by the control strategy end into FMU simulation parameters, drive the FMU to perform simulation, and convert the simulation results, time, and three-dimensional model and environmental information of the virtual scene system into the corresponding message body of the subscriber for publication; The mining truck virtual scene system is used to provide mining truck 3D models and environmental information.

2. The virtual scene-based heterogeneous model joint simulation system according to claim 1, characterized in that: The FMU simulation system comprises: RosComu module is used to receive the FMU simulation results, time and model status and environment information transmitted by the three-dimensional model of the mining truck transmitted by the Control module, convert them into the corresponding Ros message body of the subscriber for publication, and convert the control information published by the control strategy end into FMU simulation parameters and pass them to the Control module; The Control module is used to obtain the simulation results and time from the FMU driver module and pass them to the RosComu module and the mining truck 3D model, as well as obtain the FMU simulation parameters from the RosComu module and assign them to the FMU driver module to control the execution of the FMU driver module; The FMU driver module is used to execute the call of FMU, obtain FMU simulation parameters from the Control module, assign them to FMU and drive FMU to perform simulation, and pass the simulation results and time to the Control module.

3. The virtual scene-based heterogeneous model joint simulation system according to claim 2, characterized in that: The Ros communication module is implemented via TCP or UDP.

4. The virtual scene-based heterogeneous model joint simulation system according to claim 2, characterized in that: The RosComu module includes one or more Ros publishers and subscribers, and the corresponding Ros topics and message bodies are set according to requirements.

5. The virtual scene-based heterogeneous model joint simulation system according to claim 2, characterized in that: The Control module can be used to drive multiple mining truck three-dimensional models.

6. A synchronization method for joint simulation of heterogeneous models based on virtual scenes, characterized in that: The method is implemented by the virtual scene-based heterogeneous model joint simulation system according to any one of claims 1 to 5, and comprises: Establish the communication connection between the joint simulation system and the control strategy end, import the FMU and initialize it; When subscribing to the control message of the control strategy end, the control information is converted into FMU simulation parameters through the RosComu module and passed to the Control module, and then passed to the FMU driver module through the Control module, and the simulation parameters are assigned to the corresponding FMU through the FMU driver module; Each FMU performs a single-step simulation according to its own simulation parameters; When all FMUs complete a step of simulation, the simulation results and time are obtained through the FMU driver module, and the simulation results, time, and the status and environmental information of the mining truck's three-dimensional model are converted into the corresponding Ros message body of the subscriber for publication through the RosComu module.

7. The synchronization method for virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The establishing of the communication connection between the joint simulation system and the control strategy end, importing the FMU and initializing it, includes: Establish a communication connection between the joint simulation system and the control strategy end through the network communication protocol, and import And initialize, and set the simulation flag Assigns a value of false and sends an enable signal to the control strategy end ,in Expressed as indivual , express The number of 8. The synchronization method for virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The method further comprises: After a single step of simulation is completed, the simulation is paused and the simulation flag is set to true.

9. The synchronization method for virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The method further comprises: When all FMUs complete a step of simulation, the simulation results are obtained through the FMU driver module and assigned to the mining truck 3D model through the Control module. At the same time, the simulation flag is reset to false, and the simulation time is incremented by a single step.

10. The synchronization method of virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The size of the single step is adjusted according to simulation requirements.

11. The synchronization method for virtual scene-based heterogeneous model joint simulation according to claim 7, characterized in that: The data format of the simulation flag bit is set according to requirements.

12. The synchronization method of virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The status and environmental information of the three-dimensional model of the mining truck are set according to the simulation requirements, wherein the status information of the three-dimensional model of the mining truck includes posture information, angle information and pressure information; the environmental information is collected by virtual sensors, including lidar point cloud data, GPS positioning and camera information.

13. The synchronization method of virtual scene-based heterogeneous model joint simulation according to claim 6, characterized in that: The method further comprises, After establishing the communication connection between the joint simulation system and the control strategy end, the control strategy end runs and the simulation time Set to zero; Wait for the subscription enable signal. If the subscription is successful, execute the next step. Otherwise, wait. Subscribe to simulation results and calculate the control strategy simulation time difference and reset the control strategy simulation time ,in, is the simulation time; like , control strategy execution, and publish control strategy information, where, is the simulation step size.

14. The synchronization method for virtual scene-based heterogeneous model joint simulation according to claim 13, characterized in that: The control strategy end monitors the enable signal and simulation result issued by the simulation end during execution, and the control strategy can automatically subscribe when a corresponding message is issued.

15. The synchronization method of virtual scene-based heterogeneous model joint simulation according to claim 13, characterized in that: The control strategy simulation time difference Set to the simulation step size at the simulation end Consistent.

Images