Simulation system based on unreal engine blueprint

Abstract

The invention provides a simulation system based on an unreal engine blueprint, which comprises an agent simulation device and an external control device, and is characterized in that the agent simulation device comprises a system initialization unit configured to read a json configuration file generated by the external control device, and initialize an unreal engine through the unreal engine blueprint by using the json configuration file; the agent initialization unit is configured to read an agent initialization list generated by an external control device and initialize an agent simulation model through an unreal engine blueprint by using the agent initialization list; the intelligent agent instruction unit is configured to receive an intelligent agent control instruction sent by an external control device and control the intelligent agent simulation model through the unreal engine blueprint by using the intelligent agent control instruction; and the agent data unit is configured to summarize and publish the agent data from the agent simulation model, so that the external control device transfers and subscribes to the agent data. According to some embodiments, the problem of low development efficiency during intelligent agent simulation is solved.

Description

Technical Field

[0001] The present application relates to the field of intelligent agent simulation, and more specifically, to a simulation system based on Unreal Engine blueprint. Background Art

[0002] With the rapid development of computer graphics and artificial intelligence technology, Unreal Engine (UE), as a powerful open source game development platform, has been widely used in game design, film and television production, virtual reality and other fields.

[0003] As an important means to study the behavior of intelligent agents in virtual environments, intelligent agent simulation technology is of great significance for in-depth exploration of the perception, decision-making, learning and interaction mechanisms of intelligent agents. However, the Unreal Engine still has shortcomings in supporting intelligent agent simulation communication. Traditional methods such as the UE C++ interface-based method have problems such as low development efficiency and difficulty in getting started, while the gRPC-based communication method is prone to jamming when facing high-frequency control signals.

[0004] For example: The application number is 202210026353.3, and the patent name is "A third-party communication library bridging method and system adapted to Unreal Engine applications". A third-party communication library bridging method adapted to Unreal Engine applications is proposed. The application number is 202110523205.8, and the patent name is "A data communication method between Unreal Engine and external programs". A data communication method between Unreal Engine and external programs is proposed. The Unreal Engine and external programs use efficient gRPC remote service calls as a communication method for data transmission between Unreal Engine and external programs. The application number is 201811645022.8, and the patent name is "A control method and related device based on Unreal Engine". It proposes a method of calling the Unreal Engine C++API through C# to call the control method of the Unreal Engine, and does not involve intelligent agent communication.

[0005] At present, intelligent agent simulation technology is still in the development stage at home and abroad, especially in terms of communication complexity and efficiency. Although a few foreign companies have mastered the core of intelligent agent simulation technology, China still faces huge challenges in this field. As an open source 3D simulation engine, Unreal Engine's powerful functions make it possible to develop intelligent agent simulation methods. However, Unreal Engine currently does not provide a direct intelligent agent simulation communication method, and this gap needs to be filled urgently.

[0006] Therefore, it is particularly urgent to develop an efficient and stable intelligent agent simulation device suitable for Unreal Engine. Summary of the invention

[0007] The present application aims to propose an intelligent agent simulation device based on Unreal Engine blueprint to solve the problems of low development efficiency and difficulty in getting started when using Unreal Engine for intelligent agent simulation.

[0008] According to one aspect of the present application, a simulation system based on Unreal Engine blueprint is proposed, including an agent simulation device and an external control device, characterized in that the agent simulation device includes: a system initialization unit, configured to read a json configuration file generated by the external control device, and use the json configuration file to initialize the Unreal Engine through the Unreal Engine blueprint; an agent initialization unit, configured to read an agent initialization list generated by the external control device, and use the agent initialization list to initialize multiple agent simulation models through the Unreal Engine blueprint; an agent instruction unit, configured to receive an agent control instruction sent from the external control device, and use the agent control instruction to control the agent simulation model through the Unreal Engine blueprint; and an agent data unit, configured to aggregate and publish agent data from the agent simulation model, so that the external control device transfers and subscribes to the agent data.

[0009] According to some embodiments, the external control device includes a Python SDK unit, wherein: the Python SDK unit is configured to generate a json configuration file using an IO control function of the Python SDK.

[0010] According to some embodiments, initializing the Unreal Engine includes initializing the following information:

[0011] Simulation name, simulation rate, UDP server communication information, MQTT server communication information, MongoDB communication information, and / or data export frequency.

[0012] According to some embodiments, the external control device includes a MongoDB client unit and a MongoDB server unit, wherein: the MongoDB client unit is configured to use the agent initialization control function of the Python SDK to insert the initialization information of multiple agents into the MongoDB server unit; the MongoDB server unit is configured to use the initialization information of the multiple agents to generate the agent initialization list.

[0013] According to some embodiments, the initialization information of the multiple agents includes: the name, type, initial position, initial posture and / or parameter attributes of the agent simulation model.

[0014] According to some embodiments, the external control device also includes an MQTT server unit and an MQTT subscriber unit, wherein: the agent data unit is further configured to transfer the agent data to the MQTT server unit; the MQTT subscriber unit subscribes to the agent data through the MQTT server unit and transfers it to the MongoDB server unit.

[0015] According to some embodiments, the Python SDK unit is further configured to read the agent data from the MongoDB server unit through the MongoDB client unit using the Python SDK according to the agent control instructions sent by the agent algorithm device, and send the read agent data to the agent algorithm device.

[0016] According to some embodiments, the intelligent agent control instruction is directly sent by the intelligent agent algorithm device to the Python SDK unit; or the intelligent agent algorithm device first transfers the intelligent agent control instruction through the publication and subscription of RESTful API, ROS or FastDDS, and then transmits it to the Python SDK unit, or the intelligent agent algorithm device directly calls the Python SDK unit according to the intelligent agent control instruction.

[0017] According to some embodiments, the external control device further comprises a UDP client unit, wherein: the UDP client unit is configured to send the agent control instructions sent by the agent algorithm device to the agent instruction unit via the UDP protocol.

[0018] According to some embodiments, the intelligent agent control instructions are directly sent by the intelligent agent algorithm device to the Python SDK unit; or the intelligent agent algorithm device first transfers the intelligent agent control instructions through the publication and subscription of RESTful API, ROS or FastDDS, and then transmits them to the Python SDK unit; the Python SDK unit is further configured to use Python SDK to send the intelligent agent control instructions to the UDP client unit.

[0019] According to some embodiments, the system initialization unit includes a first blueprint plug-in sub-unit and a blueprint initialization sub-unit, wherein: the first blueprint plug-in sub-unit is configured to read the json configuration file generated by the external control device; the blueprint initialization sub-unit is configured to parse the json configuration file to obtain parsed data, and initialize the Unreal Engine according to the parsed data.

[0020] According to some embodiments, the agent initialization unit includes a second blueprint plug-in sub-unit and a model initialization sub-unit, wherein: the second blueprint plug-in sub-unit is configured to read the agent initialization list generated by the external control device; the model initialization sub-unit is configured to use the agent initialization list to initialize multiple agent simulation models through the Unreal Engine blueprint.

[0021] According to some embodiments, the agent instruction unit includes a third blueprint plug-in sub-unit and a model control sub-unit, wherein: the third blueprint plug-in sub-unit is configured to receive the agent control instructions sent from the external control device; the model control sub-unit is configured to use the agent control instructions to control the agent simulation model through the Unreal Engine blueprint.

[0022] According to some embodiments, the agent data unit includes a fourth blueprint plug-in sub-unit and a model data sub-unit, wherein: the model data sub-unit is configured to aggregate the agent data from the agent simulation model; and the fourth blueprint plug-in sub-unit is configured to publish the aggregated agent data so that the external control device transfers and subscribes to the agent data.

[0023] According to the embodiments of the present application, the simulation of the intelligent body is achieved by interacting with the blueprint program of the Unreal Engine instead of the traditional development method using UE C++, thereby solving the problems of low development efficiency and difficulty in getting started when using the Unreal Engine for intelligent body simulation.

[0024] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. By describing the exemplary embodiments in detail with reference to the drawings, the above and other objectives, features and advantages of the present application will become more apparent.

[0026] Figure 1 A block diagram of a simulation system device based on Unreal Engine blueprint according to an exemplary embodiment of the present application is shown.

[0027] Figure 2 A flow chart of a method for simulating an intelligent agent using an Unreal Engine according to an example embodiment of the present application is shown.

[0028] Figure 3 A schematic diagram of a system interaction process for simulating an intelligent agent using Unreal Engine according to an example embodiment of the present application is shown.

[0029] Figure 4 A flow chart of a simulation process of an intelligent agent according to an exemplary embodiment of the present application is shown.

[0030] Figure 5 An electronic device according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. In the figures, the same figures show the same or similar parts, and thus their repeated description will be omitted.

[0032] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices or operations may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0035] Before describing the embodiments of the present application, the terms appearing in the present application are first explained.

[0036] Python SDK: is a software development kit. It provides tools and libraries for developing Python software, including Python modules, classes, and functions that connect to a service or use a technology. These modules, classes, and functions can help developers use the service or technology in Python programs.

[0037] UDP (User Datagram Protocol): It is a connectionless transport layer protocol in the OSI (Open System Interconnection) reference model, providing simple, transaction-oriented, unreliable information transmission services.

[0038] MQTT (Message Queuing Telemetry Transport): An instant messaging protocol developed by IBM that supports all platforms and can connect almost all networked objects to the outside world. It is used as a communication protocol for sensors and actuators (such as connecting a house to the Internet via Twitter).

[0039] MongoDB: It is a database based on distributed file storage and supports indexing of data.

[0040] RESTful API: An application programming interface based on the REST (Representational State Transfer) principle, used for the design of Web data interfaces. RESTful API transmits data through the HTTP protocol and uses standard HTTP methods (such as GET, POST, PUT, DELETE, etc.) to operate resources and transfer states.

[0041] ROS protocol: ROS protocol is an open source robot operating system used to develop and manage robot software, providing hardware abstract description, underlying driver management, execution of common functions, inter-program message passing, program distribution package management and other services. In the ROS protocol, communication is a key part of realizing robot functions, which allows data exchange and collaboration between different nodes. ROS protocol communication is mainly based on the publish / subscribe model and the service model.

[0042] FastDDS protocol: Provides an application programming interface and a communication protocol for deploying a data-centric publish-subscribe model with the goal of establishing efficient and reliable information distribution between real-time systems.

[0043] The specific embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0044] Figure 1 A simulation system device block diagram based on Unreal Engine blueprint according to an exemplary embodiment of the present application is shown, as shown in FIG. Figure 1 The agent simulation device shown in the figure includes an agent simulation device 101 and an external control device 103. The agent simulation device 101 includes a system initialization unit 1011, an agent initialization unit 1013, an agent instruction unit 1015 and an agent data unit 1017. Figure 1 Taking as an example, an intelligent agent simulation device based on Unreal Engine blueprint according to an exemplary embodiment of the present application is described in detail.

[0045] According to an embodiment of the present application, the system initialization unit 1011 is configured to read the json configuration file generated by the external control device 103, and use the external control device 103 to initialize the Unreal Engine through the Unreal Engine blueprint. Among them, the json configuration file includes Unreal Engine configuration information. The Unreal Engine configuration information includes simulation name, simulation rate, UDP server communication information, MQTT server communication information, MongoDB communication information, data export frequency, etc.

[0046] In some embodiments, the external control device 103 includes a Python SDK unit, wherein the Python SDK unit is configured to generate a json configuration file using an IO control function of the Python SDK.

[0047] In a specific embodiment, the system initialization unit 1011 parses the text information in the json configuration file after reading it, obtains the configuration information required for the Unreal Engine simulation runtime, and implements the system initialization configuration of the Unreal Engine through the Unreal Engine blueprint. In some embodiments, the system initialization unit 1011 includes a first blueprint plug-in subunit and a blueprint initialization subunit. Among them, the first blueprint plug-in subunit is configured to read the json configuration file generated by the external control device; the blueprint initialization subunit is configured to parse the json configuration file to obtain parsed data, and initialize the Unreal Engine according to the parsed data. In practice, initializing the Unreal Engine includes but is not limited to system configuration of the Unreal Engine and generating corresponding services.

[0048] According to an embodiment of the present application, the agent initialization unit 1013 is configured to read the agent initialization list generated by the external control device 103, and use the agent initialization list to initialize and generate multiple agent simulation models through the Unreal Engine blueprint. The initialization information of the multiple agents includes the name, type, initial position, initial posture and / or parameter attributes of the agent simulation model.

[0049] In some embodiments, the external control device 103 includes a MongoDB client unit and a MongoDB server unit. The MongoDB client unit is configured to insert the initialization information of multiple agents into the MongoDB server unit using the agent initialization control function of the Python SDK; the MongoDB server unit is configured to generate an agent initialization list using the initialization information of the multiple agents.

[0050] In a specific embodiment, after reading and parsing the agent initialization list, the agent initialization unit 1013 obtains the initialization information of the agent simulation model in the agent initialization list, and then generates multiple agent simulation models through the Unreal Engine blueprint.

[0051] In some embodiments, the agent initialization unit 1013 includes a second blueprint plug-in subunit and a model initialization subunit. The second blueprint plug-in subunit is configured to read an agent initialization list generated by an external control device; the model initialization subunit is configured to use the agent initialization list to initialize multiple agent simulation models through the Unreal Engine blueprint. In practice, initializing multiple agent simulation models includes but is not limited to generating agent simulation models, setting role relationships for agent simulation models, communication relationships between agent simulation models, and / or assembly equipment for agent simulation models.

[0052] It should be noted that the present application does not limit the method of generating the intelligent agent simulation model.

[0053] The agent instruction unit 1015 is configured to receive the agent control instruction sent from the external control device 103, and use the agent control instruction to control the agent simulation model through the Unreal Engine blueprint.

[0054] In practice, the Unreal Engine blueprint controls the intelligent agent simulation model according to the intelligent agent control instructions. Specifically, the intelligent agent instruction unit 1015 parses the intelligent agent control instructions and sends them to the Unreal Engine blueprint. Then, through the regulation of the Unreal Engine blueprint, the corresponding intelligent agent control function is called to enable the intelligent agent simulation model to perform related actions or tasks.

[0055] In some embodiments, the external control device 103 further includes a UDP client unit, wherein the UDP client unit is configured to send the agent control instruction sent by the agent algorithm device to the agent instruction unit via the UDP protocol.

[0056] In a specific embodiment, the intelligent agent control instructions are directly sent by the intelligent agent algorithm device to the Python SDK unit, and the Python SDK unit calls the Python SDK to send the intelligent agent control instructions to the Unreal Engine blueprint through the UDP client unit; or the intelligent agent algorithm device first publishes the intelligent agent control instructions through the RESTful API, ROS or FastDDS, and then calls the Python SDK through the Python SDK unit to send the intelligent agent control instructions to the Unreal Engine blueprint through the UDP client unit.

[0057] In some embodiments, the agent instruction unit 1015 includes a third blueprint plug-in subunit and a model control subunit. The third blueprint plug-in subunit is configured to receive an agent control instruction sent from an external control device, and the model control subunit is configured to use the agent control instruction to control the agent simulation model through the Unreal Engine blueprint. In practice, the model control subunit parses the agent control instruction and controls the agent simulation model through the Unreal Engine blueprint according to the parsed parameters.

[0058] The agent data unit 1017 is configured to collect and publish agent data from the agent simulation model so that the external control device 103 transfers and subscribes to the agent data.

[0059] In some embodiments, the external control device 103 also includes an MQTT server unit and an MQTT subscriber unit. The agent data unit is further configured to transfer the agent data to the MQTT server unit; the MQTT subscriber unit subscribes to the agent data through the MQTT server unit and transfers it to the MongoDB server unit. The Python SDK unit is further configured to read the agent data from the MongoDB server unit through the MongoDB client unit using Python SDK according to the agent control instruction sent by the agent algorithm device, and send the read agent data to the agent algorithm device. The read agent data includes the information topic name and information content of different agent simulation models. The information topic name includes information such as the agent name and the task name. The information content includes various attributes and real-time status information such as the position, posture, speed, angular velocity, time, loading condition, agent control instruction log data, etc. of the agent simulation model in the Unreal Engine simulator.

[0060] For example, after the intelligent agent simulation model generates the intelligent agent data, the intelligent agent data unit 1017 organizes the relevant real-time status and attribute information data of each intelligent agent simulation model in the intelligent agent data into data information in json format; then, the intelligent agent data is published to the MQTT server unit according to the sending frequency required in the Unreal Engine configuration information. After the MQTT server unit successfully subscribes to and stores the intelligent agent data, the intelligent agent data will continue to be published, and after being subscribed by the MQTT subscriber unit, it will be sent directly or indirectly to the intelligent agent algorithm through the Python SDK unit. Finally, the intelligent agent algorithm updates the intelligent agent control instructions in real time according to the intelligent agent data and sends them to the Python SDK unit. Through the service transfer method of the MQTT Broker server, the received data is cached, which alleviates the communication jam when the data communication volume is large. The MQTT server has the function of forwarding. The intelligent body data is first cached in the MQTT Broker server, and then more subscribers can subscribe to relevant data from the MQTT Broker service. At the same time, the MQTT Broker can also be deployed to other computers to alleviate the performance load; the MQTT Broker service plays the role of data buffering and transfer, so it needs to be subscribed first, then published, and then subscribed by the MQTT subscriber of the Python SDK. By choosing the MQTT communication protocol, the accuracy and integrity of the returned data can be ensured, and the reliability of data return can be improved.

[0061] In a specific embodiment, the agent control instructions are sent directly by the agent algorithm device to the Python SDK unit, which uses the Python SDK function to directly read the agent data or reads the agent data from the MongoDB database; or the Python SDK unit uses the RESTful API, ROS or FastDDS to transfer the agent data, and the agent algorithm obtains the agent data from the RESTfulAPI, ROS or FastDDS.

[0062] In some embodiments, the agent data unit 1017 includes a fourth blueprint plug-in subunit and a model data subunit. The model data subunit is configured to aggregate the agent data from the agent simulation model; the fourth blueprint plug-in subunit is configured to publish the aggregated agent data so that the external control device transfers and subscribes to the agent data.

[0063] In some embodiments, after receiving the agent data, the agent algorithm device generates a new agent control command based on the agent's position, posture, speed, angular velocity, time, loading status, agent control instruction log data and other attributes and real-time status information in the agent data, and sends it to the agent instruction unit 1015 through the PythonSDK unit and UDP client unit of the external control device 103 in sequence, so that the agent instruction unit 1015 uses the new agent control command to control the agent simulation model through the Unreal Engine blueprint, thereby continuously controlling the agent simulation model.

[0064] It should be noted here that Python SDK is different from the Python script that comes with Unreal Engine. It is an external program relative to Unreal Engine, not a Python script embedded in the Unreal Engine editor. The Python script that comes with Unreal Engine reads the json configuration file, and uses the native Python of the Unreal Engine editor mode to read the json configuration file; only the Unreal Engine editor mode supports native Python script access, and does not support direct Python script access at runtime. The Python script access in the editor mode can only support the editing operations of the Unreal Engine, and does not support the operations when the Unreal Engine is running, and is invalid for the running state of the intelligent agent simulation model. In the present embodiment, the system initialization unit 1011 reads and parses the contents of the json configuration file to support the operations of the Unreal Engine and the intelligent agent simulation model at runtime.

[0065] according to Figure 1 The embodiment shown realizes the simulation of the intelligent agent simulation model by using (for example, Python SDK) and the blueprint program of the Unreal Engine to interact instead of the traditional development method using UE C++, solving the problems of low development efficiency and difficulty in using the Unreal Engine for intelligent agent simulation. And by using the communication method of the UDP protocol, efficient data transmission can be achieved, communication delay can be reduced, and network congestion of control instructions can be reduced; by using the MQTT service transfer method, the received data is cached, which alleviates the communication jam when the data communication volume is large.

[0066] Figure 2 A flow chart of a method for simulating an intelligent agent using an Unreal Engine according to an exemplary embodiment of the present application is shown. Figure 2 The method shown includes steps S201, S203 and S205.

[0067] According to an embodiment of the present application, Figure 2The execution subject of the method shown is a simulation system device based on Unreal Engine blueprint. In a specific embodiment, each unit of the agent simulation device 101 in the simulation system device includes a corresponding blueprint plug-in and a blueprint program.

[0068] Blueprint plugins are used to interact directly with Unreal Engine's blueprint program. Blueprint plugins can add various new features to Unreal Engine's blueprint system, including but not limited to custom node types, tool sets, optimization algorithms, new input processing methods, etc. These plugins can greatly improve the work efficiency of developers, allowing them to implement specific game logic and interactive effects more quickly and flexibly.

[0069] The Blueprint program is a powerful feature in UE that provides a visual scripting system that allows developers to create gameplay elements in the Unreal Editor through a node-based interface. The Blueprint program is an object-oriented compiled visual programming language that uses wires to connect nodes, events, functions, and variables to create complex gameplay logic.

[0070] In step S201, in response to a simulation instruction of the agent simulation model, the agent simulation model is initialized in the Unreal Engine.

[0071] An intelligent agent is an intelligent entity, and its English name is Agent. It is an intelligent system with three-dimensional perception, global coordination, accurate judgment, continuous evolution, and openness, built with artificial intelligence as the core. In the field of artificial intelligence, any independent entity that can think and interact with the environment can be abstracted as an intelligent agent simulation model.

[0072] According to an embodiment of the present application, before step S201, the Unreal Engine needs to be initialized.

[0073] According to some embodiments, before step S201, it is also necessary to receive initialization information of the Unreal Engine and initialization information of the intelligent agent simulation model sent from an external control device.

[0074] The system initialization unit 1011 in the intelligent agent simulation device 101 includes a first Unreal Engine blueprint plug-in and a first blueprint program. In a specific embodiment, first, the external control device generates a json configuration file using the IO control function of the Python SDK, and then, when initializing the Unreal Engine, the first Unreal Engine blueprint plug-in reads the json configuration file, and the first blueprint program parses the file content based on the json configuration file read by the blueprint plug-in and configures the Unreal Engine.

[0075] According to some embodiments, the initialization configuration of the Unreal Engine includes, but is not limited to, a simulation name, a simulation rate, UDP server communication information, MQTT server communication information, MongoDB communication information, and / or data export frequency.

[0076] The agent initialization unit 1013 in the agent simulation device 101 includes a second Unreal Engine blueprint plug-in and a second blueprint program. According to other embodiments, when initializing the agent simulation model, first, the external control device stores the initialization information of the agent simulation model in the MongoDB database to generate an agent initialization table; then, the second Unreal Engine blueprint plug-in reads the initialization configuration information of the agent simulation model from the agent initialization table, and the second blueprint program parses and processes the read agent simulation model configuration information to generate at least one agent simulation model.

[0077] In a specific embodiment, the initialization information of the agent simulation model includes but is not limited to the agent simulation model name, the agent simulation model type, the agent simulation model initial position, the agent simulation model initial posture and / or the agent simulation model parameter attribute information.

[0078] According to the embodiments of the present application, according to specific experimental requirements, the external control device directly uses the IO control function of the Python SDK or indirectly sets the configuration information of the Unreal Engine and the intelligent agent simulation model through the Python SDK through the RESTful API, ROS protocol or FastDDS protocol to meet the needs of user interaction configuration.

[0079] In step S203, a first control instruction from an external control device is received, and the control instruction is sent to the Unreal Engine to control the intelligent agent simulation model to perform a corresponding action according to the control instruction.

[0080] The agent instruction unit 1015 in the agent simulation device 101 includes a third Unreal Engine blueprint plug-in and a third blueprint program. According to an embodiment of the present application, the first control instruction includes an action control instruction and parameters for the agent simulation model, so that after the third Unreal Engine blueprint plug-in receives the first control instruction, the third blueprint program parses the first control instruction, and controls the agent simulation model through the Unreal Engine according to the parsed first control instruction to perform a corresponding action according to the control instruction.

[0081] In a specific embodiment, the external control device directly sends the first control instruction through the UDP protocol (UDP client) of the Python SDK.

[0082] In other embodiments, the external control device first transmits the first control instruction to the third blueprint program through the RESTful API transfer method, or through the ROS protocol or FastDDS protocol publish-subscribe transfer method, and then sends the first control instruction to the third blueprint program through the UDP protocol (UDP client) of the Python SDK. By using the UDP protocol for communication control of the intelligent agent simulation, not only can efficient data transmission be achieved, but also communication delay can be reduced to solve the problem of network congestion.

[0083] In step S205, the intelligent agent action execution data sent from the Unreal Engine is received and sent to the external control device, so that the external control device generates a second control instruction according to the intelligent agent action execution data.

[0084] The agent data unit 1017 in the agent simulation device 101 includes a fourth Unreal Engine blueprint plug-in and a fourth blueprint program. After the agent simulation model performs the corresponding action according to the first control instruction, it is also necessary to sort and summarize the current state of the agent simulation model and the execution process log data of the corresponding action through the fourth blueprint program, and then send it to the external control device using the fourth Unreal Engine blueprint plug-in. According to an embodiment of the present application, the Unreal Engine MQTT blueprint plug-in caches the agent data in the MQTT server (e.g., MQTT Broker server) using the MQTT protocol to support multiple MQTT subscribers to subscribe to relevant data from the MQTT service to solve the problem of communication jamming when the data communication volume is large. In order to improve load performance, the MQTT server can be deployed to other computers, and this application does not limit this. At the same time, using the MQTT protocol, the accuracy and integrity of the agent action execution data can be improved.

[0085] According to the embodiment of the present application, after caching the agent action execution data to the MQTT server, the transfer program of the Python SDK uses the MQTT subscription service to transfer the agent action execution data to the MongoDB database. Then, the external control device reads the agent state data from the MongoDB database through the RESTful API transfer method, or through the ROS protocol or FastDDS protocol publish-subscribe transfer method, and generates a second control instruction based on the read agent data (including the state data of the agent simulation model and the sensor detection data, etc.), which is used to control the agent simulation model to perform the action corresponding to the second control instruction.

[0086] In other embodiments, the external control device may also directly send a second control instruction through the Python SDK to read the agent action execution data from the MongoDB database.

[0087] According to the embodiment shown in the figure, the simulation of the intelligent body is achieved by using (for example, Python SDK) and the blueprint program of the Unreal Engine to interact instead of the traditional development method using UEC++, thereby solving the problems of low development efficiency and difficulty in getting started when using the Unreal Engine for intelligent body simulation.

[0088] Figure 3 A schematic diagram of a system interaction process for simulating an intelligent agent using Unreal Engine according to an example embodiment of the present application is shown.

[0089] like Figure 3 The agent algorithm shown corresponds to Figure 2 The external control and algorithm access section shown, Figure 2 The first blueprint program, the second blueprint program, the third blueprint program and the fourth blueprint program in the method shown correspond to Figure 3 Shown are Blueprint Program A, Blueprint Program B, Blueprint Program C, and Blueprint Program D.

[0090] Agent algorithms are a series of steps or methods used by agents to solve problems, usually implemented through programming languages ​​such as Python programs. Agent algorithms cover the core elements required by agent programs to solve problems, such as sensing the environment, understanding information, making decisions, executing actions, and improving their performance through iteration and optimization, all of which are achieved through data interaction. There are many types of agent algorithms, including but not limited to reinforcement learning, genetic algorithms, neural networks, and rule-based algorithms. These algorithms are usually used to train agents to enable them to learn, adapt, and evolve autonomously in a specific environment.

[0091] exist Figure 3 The system shown includes an external control and algorithm access part, an information configuration part, an external program part, and an Unreal Engine part. The external control and algorithm access part provides configuration instructions for the Unreal Engine and the intelligent agent simulation model and control instructions for the intelligent agent simulation model, and uses the Python SDK IO control function through the external program part to store these configuration information in a json file or stores the intelligent agent initialization information in a MongoDB database through a MongoDB client to generate an intelligent agent initialization table. The Unreal Engine part implements the system configuration of the Unreal Engine through a blueprint program A, and implements the initialization configuration of the intelligent agent simulation model through a blueprint program B to generate a corresponding intelligent agent simulation model.

[0092] The virtual engine part uses the blueprint program C to call the intelligent agent action function after receiving the first control instruction sent by the intelligent agent algorithm through the external program part using the UDP protocol, controls the intelligent agent simulation model to perform the corresponding action, and caches the intelligent agent simulation model execution action data into the MQTT protocol using the blueprint program D, so that the MQTT subscriber transfers the intelligent agent simulation model execution action data to the MongoDB database, so that the intelligent agent algorithm directly or indirectly uses the Python SDK through the MongoDB client to read the intelligent agent simulation model execution action data from the MongoDB database, and sends it to the blueprint program C through the Python SDK to control the intelligent agent simulation model to perform the action corresponding to the second control instruction.

[0093] Figure 4 A simulation process flow chart of an intelligent agent according to an exemplary embodiment of the present application is shown as follows: Figure 4 The intelligent agent shown is an unmanned vehicle. Figure 3 ,right Figure 4 The simulation process shown is described in detail.

[0094] In step S401, configuration data of the Unreal Engine and the unmanned vehicle are generated.

[0095] For example, the IO control function is called through the Python SDK to generate a json system configuration file for configuring the Unreal Engine, and the Python SDK function is called to generate complete multi-agent initialization information (for example, the initialization information of multiple unmanned vehicles), and the information is inserted into the MongoDB agent initialization information table through the MongoDB client.

[0096] In step S403, the Unreal Engine and the unmanned vehicle are respectively initialized using the configuration data.

[0097] like Figure 3 As shown, the blueprint program A of the Unreal Engine is used to read the configuration system information to perform systematic configuration of the Unreal Engine, and the blueprint program B is used to read the data agent initialization information table to generate an unmanned vehicle model.

[0098] In step S405, the unmanned vehicle is controlled to perform simulation actions.

[0099] like Figure 3 As shown, the blueprint program C is used to receive UDP control instructions from the intelligent agent algorithm to control the unmanned vehicle to perform model actions such as steering, throttle, lens switching, image exporting, and turning on the GPS sensor, and the unmanned vehicle data (including unmanned vehicle action execution data and unmanned vehicle status data) is published to the MQTT server through the MQTT client using the blueprint plug-in program D.

[0100] In step S407, the unmanned vehicle data is stored in the MongoDB database.

[0101] like Figure 3 As shown, the MQTT subscriber of the external program part is used to transfer the unmanned vehicle data, and the unmanned vehicle data is stored in the unmanned vehicle status data table or operation data table of the MongoDB database.

[0102] S409, read the unmanned vehicle data from the MongoDB database to generate corresponding action instructions.

[0103] like Figure 3 As shown, the unmanned vehicle intelligent agent algorithm can directly or indirectly obtain the unmanned vehicle status or operation data through the Python SDK function, that is, the Python SDK obtains the corresponding unmanned vehicle data through the MongoDB client according to the data request of the unmanned vehicle and transfers it to the unmanned vehicle intelligent agent algorithm. When the unmanned vehicle intelligent agent algorithm obtains the unmanned vehicle data, it calls the Python SDK control function (such as the car steering function) after algorithm perception, decision-making and other related processing. The Python SDK control function automatically processes the control parameter content, synthesizes the control instruction information in json format, and sends the information to the UDP server of the Unreal Engine part through the UDP client. After being parsed by the blueprint program C, the corresponding unmanned vehicle action instructions and action parameters are obtained, such as which specific unmanned vehicle performs what action, what the action parameters are, etc. Then, the specific unmanned vehicle intelligent agent model is controlled through the blueprint program C to perform related actions.

[0104] The loop from S405 to S409 is executed repeatedly in this way, thereby realizing the simulation interaction control between the intelligent agent algorithm and the intelligent agent model in the virtual environment, as well as the perceptual interaction of the virtual simulation environment.

[0105] Figure 5 An electronic device according to an exemplary embodiment of the present application is shown. Figure 5 The electronic device 200 according to this embodiment of the present application is described. Figure 5 The electronic device 200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0106] like Figure 5 As shown, the electronic device 200 is in the form of a general computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including the storage unit 220 and the processing unit 210), a display unit 240, etc.

[0107] The storage unit stores program codes, which can be executed by the processing unit 210, so that the processing unit 210 executes the methods described in this specification according to various exemplary embodiments of the present application. For example, the processing unit 210 can execute the following: Figure 2 The method shown in .

[0108] The storage unit 220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 2201 and / or a cache memory unit 2202 , and may further include a read-only memory unit (ROM) 2203 .

[0109] The storage unit 220 may also include a program / utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0110] Bus 230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0111] The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 250. Furthermore, the electronic device 200 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0112] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. The technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation method of the present application.

[0113] The software product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0114] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0115] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0116] The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the computer-readable medium implements the aforementioned functions.

[0117] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0118] According to an embodiment of the present application, a computer program is provided, including a computer program or an instruction. When the computer program or the instruction is executed by a processor, the method described above can be executed.

[0119] The embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, changes or deformations made by those skilled in the art based on the ideas of the present application, the specific implementation methods and the scope of application of the present application, all belong to the scope of protection of the present application. In summary, the content of this specification should not be construed as a limitation on the present application.

Claims

1. A simulation system based on Unreal Engine blueprint, comprising an intelligent agent simulation device and an external control device, characterized in that: The intelligent agent simulation device comprises: A system initialization unit, configured to read the json configuration file generated by the external control device, and use the json configuration file to initialize the Unreal Engine through the Unreal Engine blueprint; An agent initialization unit, configured to read an agent initialization list generated by the external control device, and use the agent initialization list to initialize multiple agent simulation models through the Unreal Engine blueprint; an agent instruction unit, configured to receive an agent control instruction sent from the external control device, and use the agent control instruction to control the agent simulation model through the Unreal Engine blueprint; and The agent data unit is configured to collect and publish agent data from the agent simulation model so that the external control device transfers and subscribes to the agent data.

2. The simulation system according to claim 1, characterized in that: The external control device includes a Python SDK unit, wherein: The Python SDK unit is configured to generate a json configuration file using the IO control function of the Python SDK.

3. The simulation system according to claim 1, characterized in that: Initializing the Unreal Engine includes initializing the following information: Simulation name, simulation rate, UDP server communication information, MQTT server communication information, MongoDB communication information, and / or data export frequency.

4. The simulation system according to claim 2, characterized in that: The external control device includes a MongoDB client unit and a MongoDB server unit, wherein: The MongoDB client unit is configured to insert initialization information of multiple agents into the MongoDB server unit using an agent initialization control function of the Python SDK; The MongoDB server unit is configured to generate the agent initialization list using the initialization information of the plurality of agents.

5. The simulation system according to claim 4, characterized in that: The initialization information of the multiple agents includes: The name, type, initial position, initial pose, and / or parameter properties of the agent simulation model.

6. The simulation system according to claim 4, characterized in that: The external control device further includes an MQTT server unit and an MQTT subscriber unit, wherein: The agent data unit is further configured to transfer the agent data to the MQTT server unit; The MQTT subscriber unit subscribes to the agent data through the MQTT server unit and transfers it to the MongoDB server unit.

7. The simulation system according to claim 6, characterized in that: The Python SDK unit is further configured to read the agent data from the MongoDB server unit through the MongoDB client unit using the Python SDK according to the agent control instruction sent by the agent algorithm device, and send the read agent data to the agent algorithm device.

8. The simulation system according to claim 7, characterized in that: The agent control instruction is sent directly to the PythonSDK unit by the agent algorithm device; or The intelligent agent algorithm device first transfers the intelligent agent control instruction through the publication and subscription of RESTful API, ROS or FastDDS, and then transmits it to the Python SDK unit, or the intelligent agent algorithm device directly calls the Python SDK unit according to the intelligent agent control instruction.

9. The simulation system according to claim 7, characterized in that: The external control device also includes a UDP client unit, wherein: The UDP client unit is configured to send the agent control instructions sent by the agent algorithm device to the agent instruction unit through the UDP protocol.

10. The simulation system according to claim 9, characterized in that: The agent control instruction is sent directly by the agent algorithm device to the Python SDK unit; or The agent algorithm device first transfers the agent control instruction through the publication and subscription of RESTful API, ROS or FastDDS, and then transmits it to the Python SDK unit; The Python SDK unit is further configured to use the Python SDK to send the agent control instruction to the UDP client unit.

11. The simulation system according to claim 1, characterized in that: The system initialization unit includes a first blueprint plug-in sub-unit and a blueprint initialization sub-unit, wherein: The first blueprint plug-in subunit is configured to read a json configuration file generated by the external control device; The blueprint initialization subunit is configured to parse the json configuration file to obtain parsed data, and initialize the Unreal Engine according to the parsed data.

12. The simulation system according to claim 1, characterized in that: The agent initialization unit includes a second blueprint plug-in sub-unit and a model initialization sub-unit, wherein: The second blueprint plug-in subunit is configured to read an agent initialization list generated by the external control device; The model initialization subunit is configured to utilize the agent initialization list to initialize multiple agent simulation models through the Unreal Engine blueprint.

13. The simulation system according to claim 1, characterized in that: The agent instruction unit includes a third blueprint plug-in subunit and a model control subunit, wherein: The third blueprint plug-in subunit is configured to receive an agent control instruction sent from the external control device; The model control subunit is configured to control the agent simulation model through the Unreal Engine blueprint using the agent control instructions.

14. The simulation system according to claim 1, characterized in that: The agent data unit includes a fourth blueprint plug-in sub-unit and a model data sub-unit, wherein: The model data subunit is configured to aggregate the agent data from the agent simulation model; The fourth blueprint plug-in sub-unit is configured to publish aggregated agent data so that the external control device transfers and subscribes to the agent data.

Images