Simulation optimization method and apparatus for automation system

Abstract

A simulation optimization method for an automation system. The automation system comprises an automation device and an environment. The method comprises: a reinforcement learning model (21) outputting an operation instruction; scheduling the operation instruction to a device simulation model (23) of an automation device, and the device simulation model generating device feedback on the basis of the operation instruction; scheduling the device feedback to an environment simulation model (24) of an environment, and the environment simulation model (24) generating environmental feedback on the basis of the device feedback; scheduling the environmental feedback to an interaction simulation module (25), and the interaction simulation module (25) generating a system state simulation result of the automation system on the basis of the environmental feedback; and scheduling the system state simulation result to the reinforcement learning model (21), and the reinforcement learning model (21) optimizing the operation instruction on the basis of the system state simulation result.

Description

Simulation optimization method and device for automation system Technical Field

[0001] The present invention mainly relates to the field of industrial digitalization, and in particular to a simulation optimization method and device for an automation system. Background Art

[0002] Automated devices such as robots and automated guided vehicles (AGVs) can learn new skills. Simulating their behavior in a virtual environment is an effective and cost-effective way to achieve this. However, current simulators use computer graphics, which only consider the overlap between graphics and ignore the physical and chemical changes between the automated device and its surroundings, such as deformation caused by stress and temperature. Consequently, many advanced automated skills, which involve interactions between the automated device and its environment, cannot be trained using existing simulation software.

[0003] In order to realize the simulation of automation systems, the existing technology adopts the method of configuring a physics engine. However, it can only imitate simple physical behaviors, the simulation scenarios are very limited, and the accuracy is insufficient, which cannot meet the level of industrial applications.

[0004] Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a simulation optimization method and device for an automation system to expand the application scenarios of automation system simulation and improve the simulation accuracy of the automation system.

[0006] To achieve the above-mentioned purpose, the present invention proposes a simulation optimization method for an automation system, wherein the automation system includes automation equipment and an environment, and the method includes: a reinforcement learning model outputs an operation instruction; dispatches the operation instruction to the equipment simulation model of the automation equipment, and the equipment simulation model generates an equipment feedback according to the operation instruction; dispatches the equipment feedback to the environment simulation model of the environment, and the environment simulation model generates an environment feedback according to the equipment feedback; dispatches the environment feedback to an interactive simulation module, and the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback; dispatches the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result. To this end, by dispatching between the reinforcement learning model, the automation equipment simulation model, the environment simulation model and the interactive simulation module, there is no need for complex model training, automation system simulation and interactive simulation, thereby achieving lightweight automation system simulation, reducing the physical requirements of automation system simulation, expanding the application scenarios of automation system simulation, and improving the simulation accuracy of the automation system.

[0007] Optionally, the method further comprises: synchronizing the reinforcement learning model, the device simulation model, the environment simulation model, and the interactive simulation module. To this end, by synchronizing the reinforcement learning model, the device simulation model, the environment simulation model, and the interactive simulation module, the accuracy and efficiency of the automation system simulation optimization are improved.

[0008] Optionally, the method further includes storing the operating instructions, the device feedback, the environmental feedback, and the system state simulation results. Storing the operating instructions, device feedback, environmental feedback, and system state simulation results enables data tracking, avoids repeated training and optimization, and improves the efficiency of automation system simulation optimization.

[0009] Optionally, the interactive simulation module generating a system state simulation result of the automation system based on the environmental feedback includes: the interactive simulation module generating the system state simulation result of the automation system based on the environmental feedback using finite element analysis. To this end, interactive simulation between the automation device and the environment in the automation system is achieved by using finite element analysis.

[0010] Optionally, the method further includes: dispatching the operation instructions to the device simulation model of the automation device on the edge computing device, dispatching the device feedback to the environment simulation model of the environment, dispatching the environment feedback to an interactive simulation module, and dispatching the system state simulation results to the reinforcement learning model. To this end, the scheduling action can be performed on the edge computing device without the need for cloud computing, and the edge computing device only needs to perform scheduling, without the need for complex model training, automation system simulation and interactive simulation, thus achieving lightweight automation system simulation.

[0011] The present invention also proposes a simulation optimization device for an automation system, wherein the automation system includes automation equipment and an environment, and is characterized in that the device includes: an initialization module, a reinforcement learning model outputs an operation instruction; a first scheduling module, scheduling the operation instruction to the equipment simulation model of the automation equipment, and the equipment simulation model generates an equipment feedback according to the operation instruction; a second scheduling module, scheduling the equipment feedback to the environment simulation model of the environment, and the environment simulation model generates an environment feedback according to the equipment feedback; a third scheduling module, scheduling the environment feedback to an interactive simulation module, and the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback; a fourth scheduling module, scheduling the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result.

[0012] The present invention also proposes a simulation optimization system for an automation system, wherein the automation system includes automation equipment and an environment, and the system includes a reinforcement learning model, a coordinator, a device simulation model of the automation equipment, an environment simulation model of the environment, and an interactive simulation module, wherein the reinforcement learning model outputs an operation instruction; the coordinator schedules the operation instruction to the device simulation model of the automation equipment, and the device simulation model generates a device feedback according to the operation instruction; the coordinator schedules the device feedback to the environment simulation model of the environment, and the environment simulation model generates an environment feedback according to the device feedback; the coordinator schedules the environment feedback to an interactive simulation module, and the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback; the coordinator schedules the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result.

[0013] Optionally, the coordinator further includes a synchronizer, which synchronizes the reinforcement learning model, the device simulation model, the environment simulation model and the interaction simulation module.

[0014] Optionally, the coordinator further includes a storage unit, which stores the operation instructions, the device feedback, the environment feedback and the system state simulation result.

[0015] Optionally, the coordinator is deployed in an edge computing device.

[0016] The present invention also provides an electronic device, comprising a processor, a memory, and instructions stored in the memory, wherein the instructions implement the above-mentioned method when executed by the processor.

[0017] The present invention also provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed, the method described above is executed.

[0018] The present invention also provides a computer program product, comprising a computer program, which implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The following drawings are only intended to illustrate and explain the present invention, and do not limit the scope of the present invention.

[0020] FIG1 is a flow chart of a simulation optimization method for an automation system according to an embodiment of the present invention;

[0021] FIG2 is a schematic diagram of a simulation optimization system for an automation system according to an embodiment of the present invention;

[0022] FIG3 is a schematic diagram of a simulation optimization device for an automation system according to an embodiment of the present invention;

[0023] FIG4 is a schematic diagram of an electronic device according to an embodiment of the present invention.

[0024] Description of Reference Numerals

[0025] 100 Simulation Optimization Methods

[0026] 110-150 steps

[0027] 21 Reinforcement Learning Model

[0028] 22 Coordinator

[0029] 221 Scheduler

[0030] 222 Synchronizer

[0031] 223 storage units

[0032] 23 Equipment Simulation Model

[0033] 24 Environmental Simulation Model

[0034] 25 Interactive Simulation Module

[0035] 300 simulation optimization device

[0036] 310 Initialization module

[0037] 320 First Scheduling Module

[0038] 330 Second Scheduling Module

[0039] 340 Third Scheduling Module

[0040] 350 Fourth Scheduling Module

[0041] 400 Electronic Equipment

[0042] 410 processor

[0043] 420 Memory DETAILED DESCRIPTION

[0044] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described with reference to the accompanying drawings.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0047] The present invention provides a simulation optimization method for an automation system. The automation system includes automation equipment and an environment. FIG1 is a flowchart of a simulation optimization method 100 for an automation system according to an embodiment of the present invention. As shown in FIG1 , the method 100 includes:

[0048] Step 110: The reinforcement learning model outputs an operation instruction.

[0049] The reinforcement learning model includes an agent. After the agent performs an action, the environment will switch to a new state. The environment will give a reward signal (positive reward or negative reward) for the new state. The agent then performs new actions according to a certain strategy based on the new state and the reward fed back by the environment. Through reinforcement learning, the agent can know what state it is in and what actions it should take to obtain the maximum reward. For the simulation of the automation system, the maximum reward is determined by the reinforcement learning model, so that the simulation optimization of the automation system can be achieved. A reinforcement learning model can be trained using historical data, and the reinforcement learning model can be configured according to the parameters of the automation system to initialize the reinforcement learning model. The initialized reinforcement learning model outputs an action instruction. For example, the action instruction can be to instruct the robotic arm to move down 1 cm.

[0050] FIG2 is a schematic diagram of a simulation optimization system for an automation system according to an embodiment of the present invention. As shown in FIG2 , a reinforcement learning model 21 outputs an operation instruction to a coordinator 22. The reinforcement learning model 21 can be deployed in the cloud, and the coordinator 22 can be deployed in an edge computing device.

[0051] Step 120 : dispatching the operation instruction to the device simulation model of the automation device, and the device simulation model generates a device feedback according to the operation instruction.

[0052] You can use simulation tools such as ROS (Robot operation system) to establish a device simulation model of the automation equipment by setting the parameters and properties of the automation equipment. The automation equipment can be a robotic arm, AGV, motor, CNC machine tool, etc. For example, for a robotic arm, you can set the parameters and properties of the robotic arm such as the number of joints, degrees of freedom, arm length, torque, and rotation in ROS to establish a device simulation model of the automation equipment. The operation instructions are dispatched to the device simulation model of the automation equipment. The operation instructions serve as the input of the device simulation model, and the device simulation model simulates the device feedback of the automation equipment based on the input. The device feedback can be a parameter corresponding to the operation instruction, or a parameter related to the operation instruction. Taking the operation instruction as an example of instructing the robotic arm to move down 1 cm, the device feedback can be the robotic arm moving down 1 cm, or it can be the rotation angle of each joint of the robotic arm.

[0053] As shown in FIG. 2 , the operation instruction is dispatched by the coordinator 22 (scheduler 221 ) to the device simulation model 23 . The device simulation model 23 generates a device feedback according to the operation instruction, and the device feedback is sent to the coordinator 22 .

[0054] Step 130 : The scheduling device feeds back to the environment simulation model of the environment, and the environment simulation model generates an environment feedback according to the device feedback.

[0055] Using simulation tools like ROS, you can create an environmental simulation model of the environment by setting its parameters and properties. For example, an environment can be defined as a spring-loaded button being pressed downward by a robotic arm. In ROS, you can set parameters such as the spring's elastic coefficient, maximum pressing stroke, and maximum pressing force to create an environmental simulation model of the environment. The device is then dispatched to the environmental simulation model, with the device feedback serving as input. The model then simulates the environment based on the input. For example, if the device feedback indicates that the robotic arm moves down 1 cm, the environmental feedback could indicate that the button's spring has been compressed by 1 cm.

[0056] As shown in FIG. 2 , the device feedback is dispatched by the coordinator 22 to the environment simulation model 24 . The environment simulation model 24 generates an environment feedback according to the device feedback, and the environment feedback is sent to the coordinator 24 .

[0057] Step 140 : The scheduling environment is fed back to an interactive simulation module, and the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback.

[0058] The interactive simulation module is used to simulate the interaction between automated equipment and the environment. Environmental feedback serves as input to the interactive simulation module, which simulates the interaction between the automated equipment and the environment based on the input environmental feedback and outputs the state simulation results of the automated system. For example, if the environmental feedback indicates that the button spring has compressed by 1 cm, the interactive simulation module's system state simulation result may indicate that the button spring has moved down by 0.8 cm, indicating that the system state remains normal. In other scenarios, the interactive simulation module's system state simulation result may indicate that the force applied to the button spring exceeds the load-bearing range, causing the button spring to be crushed, indicating that the system state has failed.

[0059] In some embodiments, the interactive simulation module generates a system state simulation result of the automation system based on environmental feedback, including: the interactive simulation module generates the system state simulation result of the automation system based on environmental feedback using finite element analysis. Specifically, finite element analysis (FEA) divides the automation system into many units, then performs an assembly solution on these units to generate the system state simulation result of the automation system. To this end, the use of finite element analysis enables interactive simulation between the automation equipment and the environment in the automation system.

[0060] As shown in FIG2 , the environment feedback is dispatched by the coordinator 22 to the interactive simulation module 25 , and the interactive simulation module 25 generates a system state simulation result of the automation system according to the environment feedback.

[0061] Step 150 : dispatching the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result.

[0062] The system state simulation results can be used as the state input of the reinforcement learning model. The reinforcement learning model calculates a reward value based on the system state simulation results. The reinforcement learning model outputs operation instructions, obtains state input, and calculates the reward value. This process can be regarded as the reinforcement learning model training process. During the reinforcement learning model training process, the reinforcement learning model can determine the operation instructions corresponding to the maximum reward. The operation instructions corresponding to the maximum reward are also the optimal simulation parameters of the automation system. At this point, the simulation optimization stage is completed. Subsequently, the simulation optimization parameters can be used to configure the automation system to achieve the optimal operation of the automation system.

[0063] As shown in FIG2 , the system state simulation result is dispatched by the coordinator 22 to the reinforcement learning model 21 , and the reinforcement learning model 21 optimizes the operation instructions according to the system state simulation result.

[0064] In some embodiments, the method further comprises: synchronizing a reinforcement learning model, a device simulation model, an environment simulation model, and an interactive simulation module. Specifically, the operation instructions and system state simulation results sent and received by the reinforcement learning model, the operation instructions and device feedback sent and received by the device simulation model, the device feedback and environment feedback sent and received by the environment simulation model, and the environment feedback and system state simulation results sent and received by the interactive simulation model are synchronized, that is, these data are synchronized to the same round to avoid confusion caused by data from different rounds. To this end, by synchronizing the reinforcement learning model, the device simulation model, the environment simulation model, and the interactive simulation module, the accuracy and efficiency of the automation system simulation optimization are improved.

[0065] In some embodiments, the method further includes storing operating instructions, device feedback, environmental feedback, and system state simulation results. Specifically, the operating instructions and system state simulation results sent and received by the reinforcement learning model, the operating instructions and device feedback sent and received by the device simulation model, the device feedback and environmental feedback sent and received by the environment simulation model, and the environmental feedback and system state simulation results sent and received by the interactive simulation model are stored. This data can be tracked in subsequent processes, avoiding repeated training and optimization. To this end, by storing operating instructions, device feedback, environmental feedback, and system state simulation results, data tracking can be achieved, repeated training and optimization can be avoided, and the efficiency of simulation optimization of the automated system can be improved.

[0066] In some embodiments, the method further includes: dispatching operation instructions on the edge computing device to a device simulation model of the automation device, dispatching device feedback to an environment simulation model of the environment, dispatching environment feedback to an interactive simulation module, and dispatching system state simulation results to a reinforcement learning model. Specifically, the dispatching action can be performed on the edge computing device, without having to be performed in the cloud. The edge computing device only needs to perform the dispatching, eliminating the need for complex model training, automation system simulation, and interactive simulation, thus achieving lightweight automation system simulation.

[0067] An embodiment of the present invention proposes a simulation optimization method for an automation system. By scheduling among a reinforcement learning model, an automation equipment simulation model, an environment simulation model and an interactive simulation module, there is no need for complex model training, automation system simulation and interactive simulation, thereby achieving lightweight automation system simulation, reducing the physical requirements of automation system simulation, expanding the application scenarios of automation system simulation, and improving the simulation accuracy of the automation system.

[0068] The present invention further provides a simulation optimization device for an automation system. The automation system includes automation equipment and an environment. FIG3 is a schematic diagram of a simulation optimization device 300 for an automation system according to an embodiment of the present invention. As shown in FIG3 , the device 300 includes:

[0069] Initialization module 310, the reinforcement learning model outputs an operation instruction;

[0070] The first scheduling module 320 schedules the operation instruction to the device simulation model of the automation device, and the device simulation model generates a device feedback according to the operation instruction;

[0071] The second scheduling module 330 schedules the device to feedback to the environment simulation model of the environment, and the environment simulation model generates an environment feedback based on the device feedback;

[0072] The third scheduling module 340 schedules the environment feedback to an interactive simulation module, and the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback;

[0073] The fourth scheduling module 350 schedules the system state simulation results to the reinforcement learning model, and the reinforcement learning model optimizes the operation instructions according to the system state simulation results.

[0074] The present invention also proposes a simulation optimization system for an automation system, which includes automation equipment and an environment. FIG2 is a schematic diagram of a simulation optimization system for an automation system according to an embodiment of the present invention. As shown in FIG2 , the system includes a reinforcement learning model 21, a coordinator 22, a device simulation model 23 of the automation equipment, an environment simulation model 24 of the environment, and an interactive simulation module 25, wherein:

[0075] The reinforcement learning model 21 outputs an operation instruction;

[0076] The coordinator 22 dispatches the operation instruction to the device simulation model 23 of the automation device, and the device simulation model 23 generates a device feedback according to the operation instruction;

[0077] The coordinator 22 schedules the device to feedback to the environment simulation model 24 of the environment, and the environment simulation model 24 generates an environment feedback based on the device feedback;

[0078] The coordinator 22 dispatches environmental feedback to an interactive simulation module 25, and the interactive simulation module 25 generates a system state simulation result of the automation system based on the environmental feedback;

[0079] The coordinator 22 dispatches the system state simulation results to the reinforcement learning model 21, and the reinforcement learning model 21 optimizes the operation instructions according to the system state simulation results.

[0080] In some embodiments, the coordinator 22 further includes a synchronizer 222 , which synchronizes the reinforcement learning model, the device simulation model, the environment simulation model, and the interaction simulation module.

[0081] In some embodiments, the coordinator 22 further includes a storage unit 223 that stores operation instructions, device feedback, environment feedback, and system status simulation results.

[0082] In some embodiments, the coordinator 22 is deployed in an edge computing device.

[0083] An embodiment of the present invention proposes a simulation optimization system for an automation system. By setting a coordinator to schedule between a reinforcement learning model, an automation equipment simulation model, an environment simulation model and an interactive simulation module, there is no need for complex model training, automation system simulation and interactive simulation, thereby achieving lightweight automation system simulation, reducing the physical requirements of automation system simulation, expanding the application scenarios of automation system simulation, and improving the simulation accuracy of the automation system.

[0084] The present invention also provides an electronic device 400. FIG4 is a schematic diagram of an electronic device 400 according to an embodiment of the present invention. As shown in FIG4 , the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores instructions, wherein the instructions, when executed by the processor 410, implement the method 100 described above.

[0085] The present invention further provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed, the method 100 described above is executed.

[0086] The present invention further provides a computer program product, comprising a computer program, which implements the method 100 described above when the computer program is executed by a processor.

[0087] Some aspects of the methods and apparatus of the present invention may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks," "modules," "engines," "units," "components," or "systems." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLCs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, various aspects of the present invention may be embodied as computer products in one or more computer-readable media, the product including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0088] Flowcharts are used herein to illustrate the operations performed by the methods according to the embodiments of the present application. It should be understood that the preceding operations are not necessarily performed in exact order. Instead, the various steps may be performed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0089] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0090] The above description is only an illustrative embodiment of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes, modifications and combinations made by those skilled in the art without departing from the concept and principle of the present invention shall fall within the scope of protection of the present invention.

[0091] Nouns and pronouns referring to persons in this patent application are not limited to a specific gender.

Claims

1. A simulation optimization method (100) for an automation system, wherein the automation system comprises automation equipment and an environment, characterized in that: The method (100) comprises: The reinforcement learning model outputs an operation instruction (110); Dispatching the operation instruction to a device simulation model of the automation device, wherein the device simulation model generates a device feedback according to the operation instruction (120); Scheduling the device to feedback to an environment simulation model of the environment, wherein the environment simulation model generates an environment feedback according to the device feedback (130); Scheduling the environment feedback to an interactive simulation module, the interactive simulation module generating a system state simulation result of the automation system according to the environment feedback (140); The system state simulation result is dispatched to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result (150).

2. The simulation optimization method (100) according to claim 1, characterized in that: The method (100) further comprises: synchronizing the reinforcement learning model, the device simulation model, the environment simulation model and the interaction simulation module.

3. The simulation optimization method (100) according to claim 1 or 2, characterized in that: The method (100) further comprises: storing the operation instruction, the device feedback, the environment feedback and the system state simulation result.

4. The simulation optimization method (100) according to claim 1, characterized in that: The interactive simulation module generates a system state simulation result of the automation system according to the environmental feedback, including: the interactive simulation module generates a system state simulation result of the automation system according to the environmental feedback using finite element analysis.

5. The simulation optimization method (100) according to claim 1, characterized in that: The method (100) also includes: scheduling the operation instructions to the device simulation model of the automation device on the edge computing device, scheduling the device feedback to the environment simulation model of the environment, scheduling the environment feedback to an interactive simulation module, and scheduling the system state simulation results to the reinforcement learning model.

6. A simulation optimization device (300) for an automation system, the automation system comprising automation equipment and an environment, characterized in that: The device (300) comprises: Initialization module (310), the reinforcement learning model outputs an operation instruction; A first scheduling module (320) schedules the operation instruction to a device simulation model of the automation device, wherein the device simulation model generates a device feedback according to the operation instruction; A second scheduling module (330) schedules the device to feedback to the environment simulation model of the environment, wherein the environment simulation model generates an environment feedback according to the device feedback; A third scheduling module (340) is configured to schedule the environment feedback to an interactive simulation module, wherein the interactive simulation module generates a system state simulation result of the automation system according to the environment feedback; A fourth scheduling module (350) schedules the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result.

7. A simulation optimization system for an automation system, the automation system comprising automation equipment and an environment, characterized in that: The system comprises a reinforcement learning model (21), a coordinator (22), a device simulation model (23) of the automation device, an environment simulation model (24) of the environment and an interactive simulation module (25), wherein: The reinforcement learning model (21) outputs an operation instruction; The coordinator (22) dispatches the operation instruction to the device simulation model (23) of the automation device, and the device simulation model (23) generates a device feedback according to the operation instruction; The coordinator (22) schedules the device to feedback to the environment simulation model (24) of the environment, and the environment simulation model (24) generates an environment feedback according to the device feedback; The coordinator (22) schedules the environment to be fed back to an interactive simulation module (25), and the interactive simulation module (25) generates a system state simulation result of the automation system according to the environment feedback; The coordinator (22) dispatches the system state simulation result to the reinforcement learning model, and the reinforcement learning model optimizes the operation instruction according to the system state simulation result.

8. The simulation optimization system according to claim 7, characterized in that: The coordinator (22) further comprises a synchronizer (222), wherein the synchronizer (222) synchronizes the reinforcement learning model (21), the device simulation model (23), the environment simulation model (24) and the interaction simulation module (25).

9. The simulation optimization system according to claim 7 or 8, characterized in that: The coordinator (22) further comprises a storage unit (223), wherein the storage unit (223) stores the operation instruction, the device feedback, the environment feedback and the system state simulation result.

10. The simulation optimization system according to claim 7, characterized in that: The coordinator (22) is deployed in an edge computing device.

11. An electronic device (400), comprising a processor (410), a memory (420), and instructions stored in the memory (420), wherein the instructions, when executed by the processor (410), implement the method (100) according to any one of claims 1 to 5.

12. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed, perform the method (100) according to any one of claims 1 to 5.

13. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the method (100) according to any one of claims 1 to 5.

Images