Simulation implementation method and server based on IOCP network model server

Abstract

The present invention provides a simulation implementation method and server based on an IOCP network model server. The IOCP network model server includes an intermediate component, which includes a middleware communication module and a middleware mounting algorithm model module. The intermediate component of the IOCP network model server of the present invention enables communication between the Unreal Engine and other modules, solves the data interaction problem between the visual perception algorithm model and the Unreal Engine simulation environment, and fully decouples the transmission of simulation data required by the visual perception algorithm model from the underlying environment of the UE itself, thereby improving ease of use and applicability.

Description

Technical Field

[0001] The present invention relates to the field of network communication development, and more particularly to a simulation implementation method and a server based on an IOCP network model server. Background Art

[0002] With the rapid development of hardware and algorithms in the autonomous driving field, virtual simulation technology is becoming increasingly widely used in autonomous driving R&D and testing. Simulation tool models are playing an increasingly important role in mass-produced vehicles. Traditional simulation and debugging solutions require testing existing models and performing a series of coding or adaptation operations. However, middleware, with its cross-platform and easy-to-use features, allows the Unreal Engine's simulation and algorithm modules to focus solely on data input and output, eliminating the need for other tasks. Summary of the Invention

[0003] The present invention aims to solve the technical problems existing in the prior art and provides a simulation implementation method and a server based on an IOCP network model server.

[0004] According to a first aspect of the present invention, a simulation implementation method based on an IOCP network model server is provided, wherein the IOCP network model server includes an intermediate component, wherein the intermediate component includes a middleware communication module and a middleware mounting algorithm model module, and the method includes:

[0005] The Unreal engine obtains the simulation data and encapsulates it, and sends the encapsulated simulation data to the middleware communication module of the IOCP network model server through the network communication module;

[0006] The middleware communication module packages and unpacks the encapsulated simulation data according to the requirements of the visual perception algorithm model, and sends it to the middleware mounting algorithm model module;

[0007] The middleware mounting algorithm model module calls the corresponding visual perception algorithm model to process the simulation data, and returns the feedback data output by the visual perception algorithm model to the middleware communication module of the Unreal engine through the middleware communication module.

[0008] On the basis of the above technical solution, the present invention can also make the following improvements.

[0009] Optionally, the middleware communication module includes a first thread, a second thread and a third thread, the first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

[0010] Optionally, the first thread determines whether the request socket sent by the Unreal engine is a legal connection, and if so, marks it; if not, directly disconnects it; and if so, delivers the encapsulated simulation data to the second thread;

[0011] The second thread unpacks the encapsulated simulation data according to a pre-agreed packet protocol, unpacks the instruction number and specific simulation data, and delivers them to the middleware mounting algorithm model module;

[0012] The third thread packages the feedback data output by the middleware mounting algorithm model module according to a pre-agreed package protocol, and returns the packaged feedback data to the Unreal engine.

[0013] Optionally, connect the Unreal Engine to the IOCP network model server as follows:

[0014] Create and load the visual perception algorithm model on the IOCP network model server, create a server object through the App Manager, create an object for the visual perception algorithm model, and register the agreed instruction set;

[0015] The Unreal engine sets the server's IP, port number, and password through the configuration file to connect to the IOCP network model server.

[0016] Optionally, the middleware mounting algorithm model module includes an APP manager, which is used to manage multiple visual perception algorithm models and encapsulate input and output interfaces for each visual perception algorithm model.

[0017] Optionally, the IOCP network model server includes multiple intermediate components. When receiving simulation data sent by multiple Unreal engines at the same time, a corresponding intermediate component is assigned to each Unreal engine, and the simulation data sent by the Unreal engine is processed based on the assigned intermediate component.

[0018] Optionally, the IOCP network model server includes multiple intermediate components. When receiving large-scale simulation data sent by the Unreal engine, the large-scale simulation data is divided into multiple simulation data blocks, a corresponding intermediate component is assigned to each simulation data block, and the simulation data block is processed based on the assigned intermediate component.

[0019] According to a second aspect of the present invention, there is provided an IOCP server, the IOCP network model server comprising an intermediate component, the intermediate component comprising a middleware communication module and a middleware mounting algorithm model module;

[0020] The middleware communication module is used to package and unpack the encapsulated simulation data according to the requirements of the visual perception algorithm model, and send it to the middleware mounting algorithm model module, wherein the encapsulated simulation data is obtained by encapsulating the Unreal engine, and the encapsulated simulation data is sent to the middleware communication module through the network communication module;

[0021] The middleware mounting algorithm model module is used to call the corresponding visual perception algorithm model to process the simulation data, and return the feedback data output by the visual perception algorithm model to the middleware communication module of the Unreal engine through the middleware communication module.

[0022] Optionally, the middleware communication module includes a first thread, a second thread and a third thread, the first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

[0023] Optionally, the first thread is configured to determine whether the request socket sent by the Unreal engine is a legal connection, mark the connection if it is a legal connection, and directly disconnect the connection if it is illegal; and deliver the encapsulated simulation data to the second thread when the connection is legal.

[0024] The second thread is used to unpack the encapsulated simulation data according to a pre-agreed packet protocol, unpack the instruction number and specific simulation data, and deliver them to the middleware mounting algorithm model module;

[0025] The third thread is used to package the feedback data output by the middleware mounting algorithm model module according to a pre-agreed package protocol, and return the packaged feedback data to the Unreal engine.

[0026] The present invention provides a simulation implementation method and server based on an IOCP network model server. The method uses the IOCP network model server middleware framework to load an existing simulation algorithm. The method encapsulates simulation data required by the Unreal engine, communicates data via the Unreal engine's network communication module and the IOCP network model server middleware, and provides the encapsulated Unreal engine simulation data to the algorithm model, providing feedback. This method solves the data interaction issue between the visual perception algorithm model and the Unreal engine simulation environment, fully decoupling the transmission of the simulation data required by the visual perception algorithm model from the underlying environment of the UE itself, thereby improving ease of use and applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of a simulation implementation method based on an IOCP network model server provided by the present invention;

[0028] Figure 2 This is a diagram of data interaction between the Unreal engine and the IOCP network model server;

[0029] Figure 3 This is a structural diagram of an IOCP network model server provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not restricted by the sequence of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0031] This invention relates to the technical field of developing perception modules for intelligent driving APA algorithms, primarily for processing input data for simulation tests in visual perception algorithm development. Specifically, it involves a method for implementing decoupled integration by loading simulation data and algorithm models using IOCP network model middleware based on the Unreal Engine.

[0032] Figure 1 The present invention proposes a simulation implementation method based on an IOCP network model server, wherein the IOCP network model server includes an intermediate component, and the intermediate component includes a middleware communication module and a middleware mounting algorithm model module. The simulation implementation method includes:

[0033] The Unreal engine obtains simulation data for encapsulation, and sends the encapsulated simulation data to the middleware communication module of the IOCP network model server through the network communication module; the middleware communication module packages and unpacks the encapsulated simulation data according to the requirements of the visual perception algorithm model, and sends it to the middleware mounting algorithm model module; the middleware mounting algorithm model module calls the corresponding visual perception algorithm model to process the simulation data, and returns the feedback data output by the visual perception algorithm model to the middleware communication module of the Unreal engine through the middleware communication module.

[0034] It can be understood that the embodiments of the present invention are mainly used to solve the communication decoupling between the simulation environment and the algorithm unit under test, which facilitates the work independence and efficiency between developers and testers. It mainly involves two modules, corresponding to the algorithm unit under test and the simulation environment platform respectively. Usually, the engine provides the simulation environment platform, and the algorithm model under test is mounted on the server. The embodiments of the present invention realize the communication and data interaction between the Unreal engine and the algorithm model under test.

[0035] Based on this, an embodiment of the present invention develops an intermediate component within an IOCP network model server. The intermediate component primarily includes a middleware communication module and a middleware mounting algorithm model module. The middleware communication module is primarily responsible for communication with the Unreal engine, while the middleware mounting algorithm model module is primarily responsible for mounting various visual perception algorithm models. The Unreal engine sends simulation data to the middleware communication module of the IOCP network model server, which in turn sends the simulation data to the middleware mounting algorithm model module. The middleware mounting algorithm model module then invokes the corresponding visual perception algorithm model to process the simulation data and returns the processed return data to the Unreal engine via the middleware communication module.

[0036] Among them, the middleware mounting algorithm model module includes an APP manager, and the APP manager is used to manage multiple visual perception algorithm models and encapsulate input and output interfaces for each visual perception algorithm model.

[0037] Specifically, the middleware mounting algorithm model module encapsulates the existing algorithm model and only needs to expose the input and output interfaces to serve as the middleware APP module. Create an APP manager in the middleware mounting algorithm model module to manage the APP module and distinguish it by the instruction number. Figure 2 , respectively call the corresponding algorithm model and return data. The data can be returned to the Unreal engine through the middleware communication module, or directly call other APP modules for operation.

[0038] As an embodiment, the middleware communication module includes a first thread, a second thread and a third thread. The first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

[0039] It is understandable that the middleware communication module includes multiple threads for processing different tasks. In an embodiment of the present invention, the middleware communication module mainly includes a thread specifically for processing connection delivery, a thread specifically for processing received data, and a thread specifically for processing sent data, which are referred to as the first thread, the second thread, and the third thread. Among them, the first thread judges the request socket sent by the Unreal engine to determine whether it is a legal connection. If it is a legal connection, it marks it; if it is not legal, it directly disconnects the connection; and when it is legally connected, the encapsulated simulation data is delivered to the second thread; the second thread unpacks the encapsulated simulation data according to a pre-agreed packet protocol, unpacks the instruction number and specific simulation data, and delivers it to the middleware mounting algorithm model module; the third thread packages the feedback data output by the middleware mounting algorithm model module according to a pre-agreed packet protocol, and returns the packaged feedback data to the Unreal engine.

[0040] The connection between the Unreal Engine and the IOCP network model server is established as follows: a visual perception algorithm model is created and loaded on the IOCP network model server. A server object is created through the App Manager, and the visual perception algorithm model object is created and registered with the agreed instruction set. The Unreal Engine then connects to the IOCP network model server by setting the server's IP address, port number, and password in a configuration file. Once the connection is successful, data exchange between the Unreal Engine and the IOCP network model server is established.

[0041] The above technical solution can theoretically not only solve the problem of simulation test decoupling of visual perception algorithm models, but other simulation tests of the same type can also directly refer to this module, and only the corresponding key communication content needs to be updated.

[0042] At the same time, this solution can also support parallel operation, facilitating the simultaneous execution of multiple development tasks and improving development efficiency. As an embodiment, the IOCP network model server includes multiple intermediate components. When receiving simulation data sent simultaneously by multiple Unreal engines, a corresponding intermediate component is assigned to each Unreal engine. Based on the assigned intermediate component, the simulation data sent by the Unreal engine is processed.

[0043] It is understandable that multiple middlewares are developed in the IOCP network model server. If multiple Unreal engines send simulation data to the IOCP network model server at the same time, one or more middlewares are assigned to each Unreal engine, and the simulation data is processed based on the assigned middleware.

[0044] As another embodiment, the IOCP network model server includes multiple intermediate components. When receiving large-scale simulation data sent by the Unreal engine, the large-scale simulation data is divided into multiple simulation data blocks, a corresponding intermediate component is assigned to each simulation data block, and the simulation data block is processed based on the assigned intermediate component.

[0045] It is understandable that when the simulation data sent by the Unreal engine is large-scale simulation data, the large-scale simulation data can be divided into multiple simulation data blocks, and corresponding middleware can be assigned to each simulation data block. Based on the assigned middleware, the corresponding simulation data block can be processed.

[0046] See also Figure 3 , provides an IOCP network model server, the IOCP network model server includes an intermediate component 10, the intermediate component includes a middleware communication module 101 and a middleware mounting algorithm model module 102.

[0047] The middleware communication module 101 is used to package and unpack the encapsulated simulation data according to the requirements of the visual perception algorithm model, and send it to the middleware mounting algorithm model module 102, wherein the encapsulated simulation data is the simulation data obtained by the Unreal engine 20 encapsulated, and the encapsulated simulation data is sent to the middleware communication module 101 through the network communication module 201; the middleware mounting algorithm model module 102 is used to call the corresponding visual perception algorithm model to process the simulation data, and return the feedback data output by the visual perception algorithm model to the network communication module 201 of the Unreal engine 20 through the middleware communication module 101.

[0048] The middleware communication module 101 includes a first thread, a second thread and a third thread. The first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

[0049] The first thread is used to judge the request socket sent by the Unreal engine to determine whether it is a legal connection. If it is a legal connection, it is marked; if it is not legal, the connection is directly disconnected; and when it is a legal connection, the encapsulated simulation data is delivered to the second thread; the second thread is used to unpack the encapsulated simulation data according to a pre-agreed packet protocol, unpack the instruction number and specific simulation data, and deliver it to the middleware mounting algorithm model module; the third thread is used to package the feedback data output by the middleware mounting algorithm model module according to the pre-agreed packet protocol, and return the packaged feedback data to the Unreal engine.

[0050] It can be understood that the IOCP network model server provided by the present invention corresponds to the simulation implementation method based on the IOCP network model server provided in the aforementioned embodiments. The relevant technical features of the IOCP network model server can refer to the relevant technical features of the simulation implementation method based on the IOCP network model server, which will not be repeated here.

[0051] Embodiments of the present invention provide a simulation implementation method and server based on an IOCP network model server. By developing middleware within the IOCP network model server, the middleware can be used to decouple the Unreal Engine (data producer) from the algorithm model (data consumer), allowing each to focus solely on its own needs. This eliminates concerns about data communication, cross-platform compatibility, and compatibility between the two. Furthermore, the server supports multithreading and distribution, significantly improving development efficiency while increasing convenience.

[0052] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0053] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0054] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0055] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0057] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0058] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A simulation implementation method based on an IOCP network model server, characterized in that: The IOCP network model server includes an intermediate component, which includes a middleware communication module and a middleware mounting algorithm model module. The method includes: The Unreal engine obtains the simulation data and encapsulates it, and sends the encapsulated simulation data to the middleware communication module of the IOCP network model server through the network communication module; The middleware communication module packages and unpacks the encapsulated simulation data according to the requirements of the visual perception algorithm model, and sends it to the middleware mounting algorithm model module; The middleware mounting algorithm model module calls the corresponding visual perception algorithm model to process the simulation data, and returns the feedback data output by the visual perception algorithm model to the network communication module of the Unreal engine through the middleware communication module.

2. The simulation implementation method according to claim 1, characterized in that: The middleware communication module includes a first thread, a second thread and a third thread. The first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

3. The simulation implementation method according to claim 2, characterized in that: The first thread determines whether the request socket sent by the Unreal engine is a valid connection, and if so, marks the connection; If it is illegal, the connection is directly disconnected; and when the connection is legal, the encapsulated simulation data is delivered to the second thread; The second thread unpacks the encapsulated simulation data according to a pre-agreed packet protocol, unpacks the instruction number and specific simulation data, and delivers them to the middleware mounting algorithm model module; The third thread packages the feedback data output by the middleware mounting algorithm model module according to a pre-agreed package protocol, and returns the packaged feedback data to the Unreal engine.

4. The simulation implementation method according to claim 3, characterized in that: The connection between the Unreal Engine and the IOCP network model server is achieved as follows: Create and load the visual perception algorithm model on the IOCP network model server, create a server object through the App Manager, create an object for the visual perception algorithm model, and register the agreed instruction set; The Unreal engine sets the server's IP, port number, and password through the configuration file to connect to the IOCP network model server.

5. The simulation implementation method according to claim 4, characterized in that: The middleware mounting algorithm model module includes an APP manager, which is used to manage multiple visual perception algorithm models and encapsulate input and output interfaces for each visual perception algorithm model.

6. The simulation implementation method according to claim 1, characterized in that: The IOCP network model server includes multiple intermediate components. When receiving simulation data sent by multiple Unreal engines at the same time, it allocates a corresponding intermediate component to each Unreal engine and processes the simulation data sent by the Unreal engine based on the allocated intermediate component.

7. The simulation implementation method according to claim 1, characterized in that: The IOCP network model server includes multiple intermediate components. When receiving large-scale simulation data sent by the Unreal engine, it divides the large-scale simulation data into multiple simulation data blocks, assigns a corresponding intermediate component to each simulation data block, and processes the simulation data block based on the assigned intermediate component.

8. An IOCP network model server, characterized in that: The IOCP network model server includes an intermediate component, and the intermediate component includes a middleware communication module and a middleware mounting algorithm model module; The middleware communication module is used to package and unpack the encapsulated simulation data according to the requirements of the visual perception algorithm model, and send it to the middleware mounting algorithm model module, wherein the encapsulated simulation data is obtained by encapsulating the Unreal engine, and the encapsulated simulation data is sent to the middleware communication module through the network communication module; The middleware mounting algorithm model module is used to call the corresponding visual perception algorithm model to process the simulation data, and return the feedback data output by the visual perception algorithm model to the network communication module of the Unreal engine through the middleware communication module.

9. The IOCP network model server according to claim 8, characterized in that: The middleware communication module includes a first thread, a second thread and a third thread. The first thread is a thread specifically processing connection delivery, the second thread is a thread specifically processing receiving data, and the third thread is a thread specifically processing sending data.

10. The IOCP network model server according to claim 9, characterized in that: The first thread is used to determine whether the request socket sent by the Unreal engine is a valid connection, and if so, mark the connection; If it is illegal, the connection is directly disconnected; and when the connection is legal, the encapsulated simulation data is delivered to the second thread; The second thread is used to unpack the encapsulated simulation data according to a pre-agreed packet protocol, unpack the instruction number and specific simulation data, and deliver them to the middleware mounting algorithm model module; The third thread is used to package the feedback data output by the middleware mounting algorithm model module according to a pre-agreed package protocol, and return the packaged feedback data to the Unreal engine.

Images