Data resource pool applied to simulation system

Abstract

The invention provides a data resource pool applied to a simulation system, and relates to the technical field of data resource pools. The data resource pool comprises a data resource pool body, wherein the data resource pool body comprises a simulation system connection platform, a data scheduling platform and a resource pool construction platform; the simulation system connection platform provides a basis for multi-machine control and data sharing for abstract packaging, unified scheduling and resource management of public resources or common services; the data scheduling platform provides a task model to directly execute the task cached in the layer, and transmits the task to a task backlog layer to obtain a buffer layer of the task backlog layer; and the resource pool construction platform provides data management, data service and data retrieval capabilities for the application of the simulation system.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data resource pools, and particularly to a data resource pool applied to a simulation system. Background Art

[0002] With the rapid development of technology and the explosive growth of data volume, data management and storage have become one of the core challenges in modern engineering and scientific research fields. Especially in complex simulation systems, data management and processing are crucial because these systems usually involve a large amount of inputs, outputs, models, and algorithms, all of which highly rely on data. Against this background, the concept and application of data resource pools have gradually attracted wide attention. With the progress of various technologies and applications, simulation technology has gradually become a core research tool in many fields, especially in those fields where the experimental cost is high, the risk is large, or the experimental conditions are difficult. Whether it is climate change simulation, space exploration, or biomedical research, simulation plays a crucial role. However, the successful implementation of complex simulation not only depends on efficient algorithms and powerful computing capabilities, but also on data management and processing.

[0003] However, traditional data resource pools have the following disadvantages: Traditional data resource pools are the information center and intermediate agents for data interaction among all functional components in the control system. When a complex simulation system operates, it needs to access resources from the data resource pool, which is prone to excessive resource access requests and concurrent conflicts between access requests. Summary of the Invention

[0004] In view of this, the purpose of the present disclosure is to propose a data resource pool applied to a simulation system, which can specifically solve existing problems.

[0005] Based on the above purpose, the present disclosure proposes a data resource pool applied to a simulation system, including: a data resource pool body, which includes a simulation system connection platform, a data scheduling platform, and a resource pool construction platform; the simulation system connection platform performs abstract encapsulation, unified scheduling, and resource management of public resources or common services, and provides a basis for multi-machine control and data sharing; the data scheduling platform provides direct execution of the task model for tasks cached in this layer, and transmits tasks to the task backlog layer to obtain a buffer layer of the task backlog layer; the resource pool construction platform provides data management, data service, and data retrieval capabilities for applications of the simulation system.

[0006] Generally speaking, the present disclosure has at least the following beneficial effects: By setting up a resource pool construction platform, the construction technology of a high-performance simulation basic data resource pool is discussed. Using the overall architecture of the resource pool, the metadata management module, and the data publishing interface module, it realizes high efficiency and reliability, supports complex simulation applications, and avoids the phenomenon of concurrent conflicts between access requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In the drawings, unless otherwise specified, the same reference numerals throughout the several views denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0008] Figure 1 Structural schematic diagram of the present invention;

[0009] Figure 2 Architectural schematic diagram of the simulation system connection platform of the present invention;

[0010] Figure 3 Data communication protocol diagram of the present invention;

[0011] Figure 4 Data distribution mechanism diagram of the present invention;

[0012] Figure 5 Message packet sending flowchart of the present invention;

[0013] Figure 6 Data reception and processing mechanism diagram of the present invention;

[0014] Figure 7 Architectural schematic diagram of the data scheduling platform of the present invention;

[0015] Figure 8 Queue model schematic diagram of the present invention;

[0016] Figure 9 Architectural schematic diagram of the resource pool construction platform of the present invention;

[0017] Figure 10 Overall architecture diagram of the basic data resource pool of the present invention;

[0018] Figure 11 Data form design and create index flowchart of the present invention;

[0019] Figure 12 Data interface example diagram of the present invention;

[0020] Figure 13 Query interface example diagram of the present invention;

[0021] Figure 14 This is a diagram of the data acquisition interface module of the present invention;

[0022] Figure 15 This is the working principle diagram of the data set resource pool of the present invention. Specific embodiments

[0023] The following further elaborates on the present disclosure in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and do not limit the invention. Additionally, it should be noted that for ease of description, only the parts related to the relevant invention are shown in the drawings.

[0024] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The following will detail the present disclosure with reference to the accompanying drawings and embodiments.

[0025] Figure 1 Shows the data resource pool applied to the simulation system of the present disclosure.

[0026] The data resource pool includes a data resource pool body, and the data resource pool body includes a simulation system connection platform, a data scheduling platform, and a resource pool construction platform; the simulation system connection platform performs abstract encapsulation, unified scheduling, and resource management of public resources or common services, providing a basis for multi-machine control and data sharing; the data scheduling platform provides for directly executing the task model cached in this layer and transmitting the task to the task backlog layer to obtain the buffer layer of the task backlog layer; the resource pool construction platform provides data management, data services, and data retrieval capabilities for the applications of the simulation system.

[0027] In this embodiment, the simulation system is used to simulate the collaborative operation between multiple devices, and the simulation system includes multiple simulation models. The simulation models are used to simulate the confrontation scenarios participated by the multiple devices in the simulation system.

[0028] Among them, the multiple devices can be various confrontation devices. For example, radar, aircraft (such as drones). Each device can correspond to an application in the application model layer. Through this simulation system, the operation collaboration relationships between different devices can be obtained, such as the startup and operation of different devices, and the relationships between the startup and operation of each device. In the present disclosure, the confrontation can be various activities, such as competitions, battles, etc.

[0029] This embodiment can discuss the construction technology of the high-performance simulation basic data resource pool by setting up the resource pool construction platform, and use the overall architecture of the resource pool, the metadata management module, and the data publishing interface module to achieve high efficiency, reliability, support complex simulation applications, and avoid the phenomenon of concurrent conflicts between access requests.

[0030] In some alternative implementation manners of any embodiment of the present disclosure, the simulation system connection platform includes an interactive communication architecture module, a data distribution mechanism module, a data reception and processing mechanism module, and a data analysis and caching module; the interactive communication architecture module conducts information interaction among various simulation models and subsystems in the simulation system; the data distribution mechanism module sends its own interactive instructions or status data to a ground station or other devices; the data reception and processing mechanism module receives control instructions from the ground station, as well as control instructions or status data interacted by the other devices; the data analysis and caching module updates data resources in real time in the memory and periodically stores them in a disk database.

[0031] In some alternative implementation manners of any embodiment of the present disclosure, the interactive communication architecture module includes a communication sending subsystem and a receiving subsystem; the communication sending subsystem uniformly processes data sending requests within the system and completes message sending; the receiving subsystem receives external data and performs parsing, processing, and shunting on the data.

[0032] In some alternative implementation manners of any embodiment of the present disclosure, the data analysis and caching module includes a first-level cache, a second-level cache, and a third-level cache; the first-level cache is a distributed memory cache that independently caches frequently used data in core functional components or key task functional subsystems and provides real-time data acquisition services.

[0033] In some alternative implementation manners of any embodiment of the present disclosure, the second-level cache is a centralized memory cache that caches necessary data and current data in a central data resource pool, and caches estimated data if necessary conditions are met; the third-level cache is a disk cache that performs persistent storage in a file or database and provides a disk cache read / write channel.

[0034] In some alternative implementation manners of any embodiment of the present disclosure, the data scheduling platform includes an abstract model module and an algorithm scheduling module; the abstract model module transmits data streams to corresponding auxiliary models, and the algorithm scheduling module performs scheduling calculations according to the data in real time.

[0035] In some alternative implementation manners of any embodiment of the present disclosure, the following formula is used for the scheduling calculation to evaluate the stability and performance of the simulation system: where Qi(t) and Zi(t) respectively represent the states of each task in the first task queue and the second task queue, N is the tasks in each task queue, Qi(t) represents the state of the i-th task in the first task queue at time t, and Zi(t) represents the state of the i-th task in the second task queue at time t.

[0036] In some alternative implementation manners of any embodiment of the present disclosure, the abstract model module includes a task model, a queue model, and an energy consumption model; the task model generates a model according to the task requirements of data; the queue model generates a model according to the data queue manner; the energy consumption model scheduler offloads the data stream task to an appropriate model through D2D (Device-to-Device) transmission.

[0037] In some alternative implementation manners of any embodiment of the present disclosure, the resource pool construction platform includes an overall architecture module, a data management module, a publishing interface module, and a collection interface module; the overall architecture module sets up an overall architecture based on simulation basic data; the data management module has a data type management function and a metadata configuration function, the publishing interface module provides a lightweight external interface for querying various data structures, and the collection interface module provides a data storage function for the top-level function of the platform.

[0038] In some alternative implementation manners of any embodiment of the present disclosure, if the communication sending subsystem receives a custom message packet sent by other working components from a specified data channel, it serializes the custom message packet into a binary stream, fills the binary stream into the payload field of a unified message packet, serializes the unified message packet into a binary stream to obtain a binary message packet, and sends the binary message packet into the cluster communication network.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] By setting up a data scheduling platform, the scheduling technology of real-time data in the data resource pool, including the establishment of an abstract model, a data stream scheduling algorithm based on Lyapunov, and the specific implementation of the data stream scheduling algorithm, improves data processing efficiency and system performance;

[0041] By setting up a resource pool construction platform, the construction technology of a high-performance simulation basic data resource pool is discussed, including the overall architecture of the resource pool, a metadata management module, a data publishing interface module, and a data collection interface module. The data resource pool system is efficient and reliable, supports complex simulation applications, and avoids the phenomenon of concurrent conflicts between access requests.

[0042] The present disclosure also discloses a data resource pool applied to a simulation system. The data resource pool applied to the simulation system includes: a simulation system connection platform, a data scheduling platform, and a resource pool construction platform;

[0043] The simulation system connection platform provides a technical basis for multi-machine control and data sharing through the abstract encapsulation, unified scheduling, and resource management of public resources or common services;

[0044] The data scheduling platform provides the task model to directly execute the tasks cached in this layer, and transfers tasks of appropriate sizes to the task backlog layer to act as a buffer layer for the backlog layer.

[0045] The resource pool construction platform provides flexible configurable data management, stable clustered data services, and efficient data retrieval capabilities for the top-level applications of the simulation system.

[0046] The simulation system connection platform includes an interactive communication architecture module, a data distribution mechanism module, a data reception and processing mechanism module, and a data analysis and caching module. The interactive communication architecture module conducts information interaction among various simulation models or subsystems. The data distribution mechanism module is responsible for sending its own interactive instructions or status data to the ground station or other devices. The data reception and processing mechanism module is responsible for receiving control instructions from the ground station, as well as control instructions or status data interacted with other devices. The data analysis and caching module updates data resources in real time in the memory and periodically stores them in the disk database.

[0047] The interactive communication architecture module includes a communication sending subsystem and a receiving subsystem. The communication sending subsystem uniformly processes data sending requests within the system and completes message sending. The receiving subsystem uniformly receives external data and conducts parsing, processing, and shunting.

[0048] The data analysis and caching module includes a first-level cache, a second-level cache, and a third-level cache. The first-level cache is a distributed memory cache that independently caches frequently used data in core functional components or key task functional subsystems to provide real-time data acquisition services. The second-level cache is a centralized memory cache that caches necessary data and current data in the central data resource pool and adds estimated data when necessary. The third-level cache is a disk cache that realizes persistent storage in files or databases and provides a disk cache read / write channel.

[0049] The data scheduling platform includes an abstract model module and an algorithm scheduling module. The abstract model module transmits data streams to appropriate auxiliary models. The algorithm scheduling module conducts scheduling calculations in real time according to the data. The scheduling formula is specifically as follows: First, it is defined that I(t) = {Q(t), Z(t)} is the set of current system states, where the queue of SAIL is Z(t) = {Zi(t)|i ∈ N}, the queue of TBL is Q(t) = {Qi(t)|i ∈ N}, and the Lyapunov equation is defined as:

[0050] The abstract model module includes a task model, a queue model, and an energy consumption model. The task model generates a model according to the task requirements of the data. The queue model generates a model according to the data queue method. The energy consumption model scheduler controls the offloading of data stream tasks to appropriate models through D2D transmission.

[0051] The resource pool construction platform includes a general architecture module, a data management module, a publishing interface module, and a collection interface module. The general architecture module sets up the general architecture based on the simulation basic data. The data management module includes a data type management function and a metadata configuration function. The publishing interface module provides a complete set of lightweight external interfaces suitable for querying various data structures. The collection interface module provides an efficient and reliable data storage function for the top-level functions of the platform.

[0052] In the present invention, in the information interaction of each simulation model or subsystem, in order to reduce the interference between the sending and receiving of instructions, the interactive communication architecture module is divided into a communication sending subsystem and a receiving subsystem. The former is responsible for uniformly processing the data sending requests within the system and completing the message sending. The latter is responsible for uniformly receiving the incoming data and performing parsing, processing, and distribution. First, a unified data communication protocol is designed for this distributed simulation task, combined with the attached Figure 3 , and the communication protocol is used to serialize and standardize the data to obtain a unified data format, enabling the data to be transmitted between the simulation nodes without obstacles. The meanings of the fields of the protocol are as shown in the following table:

[0053]

[0054]

[0055] The data distribution mechanism module is mainly responsible for sending its own interactive instructions or status data to the ground station or other unmanned aerial vehicles. At this time, the data distribution mechanism module exposes ports to the outside as a server and sends the messages into the communication domain, combined with the attached instructions Figure 4 , first, the sending subsystem needs to continuously monitor the data sending ports of all other functional components on the software bus in real time. When a data sending request is received, the corresponding callback processing method is executed, the data is encapsulated into a defined message packet, and then sent into the network. At the same time, in order to take into account the data processing capabilities of different message receivers, different message sending frequencies also need to be processed, combined with the attached instructions Figure 5, when the communication sending subsystem receives a message sending request, it encapsulates and serializes the message, and then sends the message packet to the ground station or the specified UAV algorithm process, receives custom message packets sent by other working components from the specified data channel; serializes the custom message packets into binary streams; fills the binary sequences into the payload fields of the unified message packets; fills the header part of the unified message packets, including message type, message ID, topic ID, uses the local ID as the source machine ID at the same time, and fills the ID of the target UAV or ground station to which the message will be sent, as well as other header fields; serializes the unified message packets into binary streams; sends the binary message packets into the cluster communication network. Among them, by serializing the custom message packets, the unified processing of various different types of messages is realized, so that only a single type of message packet flows in the cluster, simplifying the communication work of the cluster. By filling the cluster ID, source machine ID and destination machine ID, the transmission and sharing of messages between specified UAVs in the cluster are realized.

[0056] Combined with the attached drawings of the specification Figure 6 , the data receiving and processing mechanism module is responsible for receiving control instructions from the ground station, as well as control instructions or status data for interaction with other UAVs. Therefore, the data receiving and processing mechanism module needs to connect to multiple different servers, the ground station or other UAVs, as a client to form a one-to-many wireless communication link. At the same time, in order to avoid data interference and channel load congestion between different communication links, it is necessary to open up independent running threads for each link. After receiving the message, it is also necessary to complete the parsing and shunting of the message.

[0057] At the beginning of time slot t of the task model, model i randomly generates a random data stream task, and bi(t) (CPU revolutions) is the amount of resources required to process this data stream. At the beginning of each time slot, the tasks arriving in the model are independent of the queue backlog of SAIL, and bi(t) has an uncertain size change in each time slot. bmax is the upper limit of the amount of resources required for the data stream task. For TBL, it is responsible for caching the data stream task and sending the data stream to the resource model.

[0058] In the t time slot, this layer receives the data stream task from SAIL, and through the control of the scheduler, transmits the data stream task of appropriate size to the resource model. The data stream task in the TBL is represented as a triple <Si(t), Ai(t), pi(t)>. When the scheduler selects to schedule the data stream task cached in the TBL of model i, the simulation system will collect the data stream data required by the task and transmit it to resource model j. Kij(t) is the transmission rate from task model i to resource model j, and Si(t) (bits / s) is the data collection rate of model i in the t time slot. According to the parallelism and real-time nature of data stream processing, the more resources required by the task to be scheduled in the TBL, the greater the amount of data collected by the simulation system, that is, Kij(t) ≥ Si(t), and Ai(t) is the corresponding processing density (CPU revolutions / bit) in the t time slot. Previous work has confirmed that different tasks for a unit amount of data will require different CPU revolutions. pi(t) is the ratio of the output data volume to the input data volume of the data stream task. When the data stream task is executed on any resource model, a certain amount of output data stream will be generated, and its size is the input data volume of the data stream task multiplied by pi(t), and 0 < pi(t) < 1.

[0059] Figure 7 Schematic diagram of the architecture of the data scheduling platform of the present invention.

[0060] Combined with the specification appendix Figure 8 , at the beginning of the t time slot, task model i will first process the data stream task with the ability of ci*. Due to physical capacity limitations, when the ability of the model itself is insufficient to process the data stream task, that is, ci* < bi(t), the amount of resources required by the simulation system is bi(t) - ci*. Model i caches the part of the data stream task that needs assistance in SAIL. When model i does not need the assistance of other models, that is, ci* > bi(t), and the simulation system will process the data stream cached in SAIL, with a size of ci* - bi(t). At the beginning of the t time slot, the backlog of the cache queue of SAIL is recorded as Zi(t), and its size is the amount of resources required for the data stream cached in SAIL. In the t time slot, SAIL transmits part of the data stream task to the TBL, and the amount of resources required for the transmitted task is r(t). Therefore, the queue backlog Z(t) follows the following formula:

[0061]

[0062] For the stability of the entire system, the scheduler control unloads the data stream task to an appropriate model through D2D transmission. According to the network state, in the t time slot, the scheduler establishes a scheduling channel from the task model to the resource model and transmits the data stream data. The total energy consumption P(t) generated by the transmission process from task model i is expressed as: P i,sum (t) = ∑j∈Δi (x ij (t)P ij (t)+x ij (t)P ji (t)), where P(t) is the transmission energy consumption from the task model to the resource model, and Pij(t) is the transmission energy consumption from the resource model j to the task model i. Cellular D2D communication is used. Due to its large coverage area and ability to adapt to more model conditions, scheduling calculations are performed in real time according to the data.

[0063] The scheduling formula is specifically as follows: First, I(t) = {Q(t), Z(t)} is defined as the set of current system states, where the queue of SAIL is Z(t) = {Zi(t)|i ∈ N}, and the queue of TBL is Q(t) = {Qi(t)|i ∈ N}. And the Lyapunov equation is defined as: The data scheduling algorithm is as follows:

[0064]

[0065] Combined with the attached Figure 10 The application layer provides interactive processing function modules for users, including a metadata management module, a data management module, a data publishing interface module, and a data collection interface module; the basic layer is the core of the system design, providing key functions and basic tools for the functions of the presentation layer, including the relational data engine Mysql, the object data retrieval engine ElasticSearch, the data collection Logstach, and the message queue Kafka. Mysql is used to store and manage metadata information, ElasticSearchCluster provides technical support for data management, data publishing, and retrieval, Logstach and Kafka are combined to achieve the data collection function, and the main function of the metadata information management module is to view all data category information managed by the system. The number of data categories indicates how many types of data the system manages.

[0066] The main function of the metadata configuration function is to view, create, and edit the specific data structure of a certain data type. The data structure design is completed through a Web form designer. The form designer automatically decomposes the output designed by the user into a form, a form draft, and a data structure description. The form is an HTML fragment containing data editing controls and layout information, used to provide data interaction display for the human-computer interface. The form draft is an HTML fragment with the same structure layout as the form, and at the same time contains more design-time tags, used to provide the function of further metadata changes based on the existing form. The output after the change is the next version of the data of this type. The data structure description is a description document that extracts the data structure in a tree form based on the current form HTML fragment. Here, the JSON format is used. The nodes of the JSON document contain data types such as int, double, string, datetime, node ID, node name, and interface control type, providing essential basic information for data management.

[0067] To enable the system to have the ability to efficiently store and retrieve data, the Mysql database is only used to store information related to the metadata management module and does not store the data itself. To ensure the correspondence between metadata and data, the metadata management module has developed extensions based on the Setting and Mapping APIs of ElasticSearch to implement the management function of the library index and the setting function component of the data structure, ensuring the consistency and synchronization of the data structure definitions between the two libraries. From the data form design to the system combination instructions for creating the Elastic-Search index attached Figure 11, the data form design result generates two output contents: one is an HTML fragment used to describe the content display layout; the other is a data structure tree JSON document. Some information in the JSON document, such as name, value-Type, and format, is used to generate the JSON document required to define the creation of the ElasticSearch index. The JSON related to this index is directly used to create the ElasticSearch index and set the data structure. The JSON related to the index mainly includes two parts of information, Setting and Mapping. The Setting contains the number of index shards and index replicas. The Mapping content is the description information of each attribute field of the data, including the field name, type, and format. The setting of the Mapping is beneficial to ensuring the correctness and execution efficiency of the ElasticSearch retrieval function. Since the platform requires the data structure stored in the data resource pool to be flexible and configurable, to meet the query and retrieval requirements, the system must provide a complete set of lightweight external interfaces suitable for querying various data structures to facilitate the access of multiple external functions and data query; After comprehensive consideration, taking advantage of the advantages of ElasticSearch in retrieving data, its retrieval API is encapsulated and developed, and the SearchAPI is exposed externally in the form of a RESTful interface. The encapsulated data service provides two data query and retrieval interface usage modes for the top-level application of the platform. Taking simulation telemetry data as an example, it is a typical JSON document, containing keys such as title, satId, and info, combined with the attached Figure 12 and the attached Figure 13 , the first query method is to construct the query condition through URL parameters and submit the query request to the server via the GET method. The second query method uses a JSON document to define the query rule and sends the JSON document to the server based on the Body. Obviously, the second method of sending the query rule request based on the Body provides more abundant query methods. When the external system queries data, the first URISearch query method is used for simple queries, and the RequestBodySearch query method is required for complex rule queries. The data acquisition interface module mainly provides an efficient and reliable data storage function for the top-level function of the platform. The data generated by the simulation system has the characteristics of a large overall data volume, intensive storage requests, high real-time performance, and horizontal scalability.

[0068] To adapt to this feature, two tools need to be introduced based on ElasticSearch. One is Kafka, and the other is Logstash. Apache Kafka is a distributed message queue tool implemented in Scala, which uses ZooKeeper for cluster management. Kafka adopts a decoupled design concept and is based on non-primitive publish-subscribe logic. Instead of the producer being responsible for generating data and directly pushing it to the consumer, a persistent layer - broker is added in the middle. The producer stores the data in the broker, and the consumer retrieves the data from the broker. This brings several benefits: decoupling the load of the producer and the consumer; the consumer reads data according to its own capabilities; the consumer can customize the quantity of consumption.

[0069] In addition, since Kafka adopts the idea of partitioning based on topics, the order within a certain partition is guaranteed to be orderly. In this way, different producers and consumers, taking partitions as units, agree on the positions to read data. Logstash is a high-performance data processing pipeline that can collect data from multiple sources simultaneously, perform customized transformation processing, and then send the data to the specified target location. Of course, the target location defined here is ElasticSearch. The complete data processing process of the data collection interface module is combined with the attached Figure 14 As shown in the figure, first, the data output by each entity in the simulation is cached in Kafka according to the topics (themes) partitioning of Kafka. Logstash obtains data of different types from multiple topics partitions of multiple Kafka according to the agreement of the input nodes in its own configuration file. After conversion and filtering processing, data in the agreed format is formed. The output outputs according to the type configuration and stores it in different indexes of ElasticSearch for query and analysis. Combined with the attached Figure 15 , the working principle of the resource pool is based on the concept of a "buffer pool" that temporarily stores data in memory. This is an effective performance optimization strategy. In the traditional database interaction mode, the server must establish a database connection for each user request, then read the required information, and close the connection after the operation is completed. This process not only takes time but also consumes the server's computing resources. Especially in a high-concurrency environment, frequent database operations will significantly reduce the server's response speed and processing ability. The design of the resource pool aims to relieve the pressure of frequent interaction between the server and the database. By preloading and storing commonly used or expected-to-be-frequently accessed database information in the buffer pool in memory, when the user requests data, the server directly retrieves the information from this memory pool instead of querying the database every time. Since the reading speed of memory is much faster than that of the disk, this method greatly reduces the data retrieval time.

[0070] In addition, the resource pool not only improves the speed of data access but also reduces the load on the database. Under the resource pool strategy, the server will establish a connection with the database to read data only when the required information cannot be found in the buffer pool. This means that the number of database connections will be significantly reduced, thereby reducing the load on the database and its impact on performance. At the same time, this also reduces the network I / O pressure on the server because it reduces the dependence on external data sources. The management of the resource pool is dynamic and it needs to update its content regularly to ensure the real-time nature and accuracy of the data; this is usually achieved by setting cache policies, such as setting the expiration time of the data and using the least recently used (LRU) algorithm to replace old data methods.

[0071] In addition, to prevent the data in the buffer pool from becoming outdated, some advanced techniques such as data subscription or listening to database change events are used to update the data in memory in a timely manner. In the software design of the resource pool mode, Resource (resource object), ResourcePool (resource pool object), and Client (client object) constitute three core components. Each component undertakes different responsibilities and jointly realizes the effective management and reuse of resources, improving the performance and resource utilization rate of the Web information system. Resource (resource object): Resource refers to the object that is reused in the Web information management system; specifically in database applications, each database record is regarded as a Resource object, and each field in the record corresponds to the attribute of the Resource class. This design allows the system to encapsulate the complexity of database operations inside the object, making it easy to manage and maintain; ResourcePool (resource pool object): ResourcePool is a collection that manages resource objects. Its core responsibility is to maintain, allocate, and recycle these Resource objects; in terms of design, ResourcePool usually adopts the Singleton design pattern to ensure that there is only one resource pool instance throughout the application; such a design guarantees the unified allocation and recycling strategy of resources and reduces the complexity of state synchronization of the resource pool itself. The Resource objects in the resource pool usually come from the same database table, which makes it possible to use the same management strategy; Client (client object): Client represents the client application that uses the resource pool and resource objects; the client requests resource objects through ResourcePool, uses the resource objects to complete specific tasks, such as database query or update operations, and then returns the resource objects to the resource pool. This design pattern allows the client to focus on the implementation of business logic without having to worry about the life cycle and reuse issues of resource objects.

[0072] The software design of the resource pool pattern is effective because it is based on a core premise: the objects to be managed are reusable; in a Web information system, the read and write operations of database records are frequent and repetitive. Therefore, materializing these records into reusable resource objects not only meets the requirements of the resource pool pattern but also satisfies the need for system performance optimization. Through the resource pool pattern, direct calls to the database are reduced, thereby alleviating the burden on the database and improving the system's response speed. In the detailed design, the resource pool needs to implement several key functions: (1) Initialization of resources: Pre-create resource objects and load them into the pool as needed; (2) Allocation and recycling of resources: Be able to allocate resources according to client requests and recycle the resources for reuse after use; (3) Resource status management: Ensure that the resources are in a clean and consistent state before being reused; (4) Resource monitoring: Track the usage of resources, including the number of allocations and recycling times, for performance tuning and troubleshooting; (5) Resource synchronization: If the system runs in a multi-threaded or distributed environment, the resource pool also needs to ensure the thread safety and synchronization mechanism of resource objects.

[0073] The initialization of resources is a crucial part of the resource pool management strategy. It involves the pre-creation and pre-configuration of resource objects so that they can quickly respond to client requests during actual use. The following is a detailed elaboration of the implementation plan for resource initialization: Pre-create resource objects: During the initialization phase of the resource pool, the system pre-creates resource objects based on the estimated maximum concurrency and resource usage frequency. This number should not be too large to avoid wasting memory, nor too small to be unable to handle peak demands. Implement an initialization function that is called when the application starts and creates resource objects according to the number specified in the configuration file or startup parameters. These resource objects will be stored in the data structure of the resource pool, such as a queue or a pool; Configuration management: The configuration information of resource objects usually comes from configuration files or environment variables; During the initialization process, the resource pool needs to read these configurations and apply them to each resource object. The configuration of resource objects includes database connection strings, credentials, and initialization SQL statements; Status check and health monitoring: When resource objects are created, they need to pass a health check to ensure that they are in an available state. For example, if the resource object is a database connection, a connection test needs to be performed during the initialization process to confirm a normal connection to the database; Exception handling and retry mechanism: Resource creation failures may occur during the initialization process. For example, the database cannot be connected. An exception handling mechanism needs to be implemented to retry or log when the initialization fails, ensuring the reliable initialization of the resource pool; Factory pattern for resource objects: Use the factory pattern to create resource objects, separating the creation logic of resource objects from the management logic of the resource pool to make the code clearer; The factory class is responsible for generating various resource objects according to the configuration, while the resource pool is responsible for using the objects provided by the factory class; Dynamic expansion: When initializing the resource pool, also consider implementing a dynamic expansion mechanism; That is, when resource usage approaches saturation, more resource objects can be dynamically created to handle the increasing demand; Pre-creating resource objects significantly improves the system's response speed to client requests. When a client requests a resource, the resource pool immediately allocates the already initialized resources without waiting for the creation of resources; Pre-creating resource objects during the initialization phase avoids the load of dynamically creating resources during the client request processing, thus reducing the pressure on the system during runtime. The pre-created resource objects are shared and used by multiple clients when not needed, thus improving the utilization rate of resources. Performing resource health checks and configuration management during the initialization phase can detect and solve potential configuration errors or connection problems in advance, improving the overall reliability of the system. Since the resource objects are already initialized and ready, the client code is more concise, focusing on the implementation of business logic rather than the creation and management of resources.

[0074] By implementing a dynamic expansion mechanism, the resource pool automatically adjusts the number of resource objects according to actual needs, making the system more flexible and capable of adapting to different load changes. The resource pool provides a unified platform to manage all resource objects, which helps maintain and monitor resource usage, and also facilitates resource optimization and adjustment. In the initialization phase, combined with the actual usage pattern of the system, a more intelligent resource allocation strategy is implemented, such as dynamically adjusting the initialized resource quantity based on a prediction algorithm. In the above implementation scheme, the initialization of resources not only provides fast and reliable resource access for the client, but also provides strong support for resource management, making the entire system more robust and efficient.

[0075] Initialization is an essential part of resource pool management and is crucial for building a high-performance Web information system. The resource allocation and recycling mechanism is the core function of the resource pool, ensuring the efficient recycling of resources and the stability of the system; this mechanism allows the client to obtain resources on demand and return them to the resource pool after resource usage for other clients to use.

[0076] The following is a detailed description of the resource allocation and recycling process: The resource allocation process is as follows: Request processing: When a client requests a resource, the resource pool first checks the free resource list; if there are available resources, the resource pool will select one and allocate it to the requester; the selection algorithms are simple queue (FIFO), stack (LIFO), or more complex strategies based on priority or least recently used (such as LRU); Resource initialization: Before allocation, the resource pool ensures that the resource is in its initial state. For example, if the resource is a database connection, the resource pool will reset all session information of the connection to ensure that the client gets a "clean" connection; Resource status update: Once the resource is allocated, the resource pool updates its internal state, marks the resource as occupied, and updates some monitoring information, such as the usage counter of the resource; Exception handling: If all resources are in use, the resource pool chooses to wait for free resources, create new resources (if the configuration allows), or throw an exception to inform the client that there is a shortage of resources. The resource recycling process is as follows: Resource return: After the client finishes using the resource, it will call the recycling interface provided by the resource pool to return the resource; the returned resource needs to be checked to confirm that its status meets the recycling conditions; Resource cleaning: Necessary cleaning operations need to be carried out before the resource is returned to the resource pool; for example, if the resource is a database connection, any open transactions need to be closed and temporary data cleared; Resource status restoration: After cleaning, the resource pool resets the resource to its initial state so that the resource can be safely allocated to the next requester; Resource monitoring update: When the resource is recycled, the resource pool updates the monitoring information, records the number of resource recycling times and usage statistics, for performance analysis and optimization. Through the allocation and recycling of resources, fast resource allocation reduces the client waiting time, effectively improves the response efficiency of the entire system and the user experience. Through the recycling and reallocation mechanism, resources are reused to the greatest extent, reducing the overhead of creating and destroying resources, avoiding excessive requests for system resources under high concurrency conditions, reducing the load on the database and other backend services. At the same time, orderly resource management reduces the risk of resource leakage, enhancing the stability and reliability of the system. On the other hand, the resource pool dynamically adjusts the number of resources according to the current system load and resource usage, enhancing the adaptability and scalability of the system.

[0077] In summary, the allocation and recycling of resources are key parts of resource pool management, which ensure the effective management and optimized use of resources and are an indispensable part of an efficient Web information system; through carefully designed allocation and recycling strategies, the system performance is significantly improved, resource waste is reduced, and user satisfaction is increased. Resource synchronization is the key mechanism to ensure the normal operation of the resource pool in a multi-threaded or distributed environment.

[0078] When multiple threads or service instances access and modify resources in the resource pool simultaneously, a synchronization mechanism must be adopted to avoid conflicts and data inconsistency issues. Regarding the synchronization mechanism, the main aspects are as follows: Locking mechanism: In a multi-threaded environment, locking is a common method to ensure thread safety; the resource pool implements locking mechanisms with different granularities for resource objects, such as fine-grained object-level locking or coarse-grained global locking, to control concurrent access to resource objects.

[0079] The resource pool adopts concurrent data structures: Modern programming languages usually provide thread-safe concurrent data structures, such as ConcurrentHashMap in Java. These data structures internally implement locking mechanisms to store resource objects, thereby reducing the need for explicit synchronization.

[0080] The transaction memory of the resource pool is as follows: Transaction memory is a synchronization method that allows developers to execute a series of operations in the form of a transaction. These operations either all succeed or all roll back, which helps simplify complex synchronization logic; Distributed lock: In a distributed system, the resource pool needs to synchronize resource states across multiple service instances; this is usually achieved through distributed locks, such as using Redis or Zookeeper to ensure the atomicity and consistency of cross-service operations.

[0081] The resource pool adopts a message queue: Using a message queue for resource state synchronization is also an option, especially in a distributed system. By queuing to serialize access requests to resources, concurrent conflicts can be avoided. Therefore, based on the above synchronization mechanism, to implement resource synchronization, the lock implementation is carried out first. Locks are implemented on each resource object. When a thread needs to access a resource, it must first obtain the lock; after use, the thread releases the lock so that other threads can access the resource. Then, a read-write lock needs to be configured. If the resource read operations are much more than the write operations, a read-write lock is used to optimize performance; the read-write lock allows multiple read operations to be executed concurrently, but write operations are exclusive. At the same time, optimistic locks and version control need to be introduced. For scenarios where conflicts rarely occur, an optimistic locking strategy is used, and updates are controlled by version numbers. If a version inconsistency is found during the update process, the operation is rolled back. In summary, resource synchronization is an important part of the resource pool design in multi-threaded and distributed environments. By using appropriate synchronization mechanisms, the resource pool can manage resources safely and effectively, ensuring the correctness and stability of the system during concurrent access.

[0082] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A data resource pool applied to a simulation system, characterized in that: It includes a data resource pool body, which includes a simulation system connection platform, a data scheduling platform and a resource pool construction platform; The simulation system connection platform provides the basis for abstract encapsulation, unified scheduling and resource management of public resources or common services, and multi-machine control and data sharing; The data scheduling platform provides a task model to directly execute the tasks cached in this layer, and transmits the tasks to the task backlog layer to obtain the buffer layer of the task backlog layer; The resource pool construction platform provides data management, data service and data retrieval capabilities for the application of the simulation system.

2. The data resource pool according to claim 1, characterized in that: The simulation system connection platform includes an interactive communication architecture module, a data distribution mechanism module, a data receiving and processing mechanism module, and a data analysis and caching module; The interactive communication architecture module performs information exchange among various simulation models and subsystems in the simulation system; The data distribution mechanism module sends its own interaction instructions or status data to the ground station or other equipment; The data receiving and processing mechanism module receives control instructions from the ground station, and control instructions or status data of interaction with other devices; The data analysis cache module updates data resources in real time in the memory and stores them to the disk database regularly.

3. The data resource pool according to claim 2, characterized in that: The interactive communication architecture module includes a communication sending subsystem and a receiving subsystem; The communication sending subsystem uniformly processes data sending requests within the system and completes message sending; The receiving subsystem receives external data, and performs analysis, processing and diversion on the data.

4. The data resource pool according to claim 2, characterized in that: The data analysis cache module includes a first-level cache, a second-level cache and a third-level cache; The first-level cache is a distributed memory cache that independently caches commonly used data in core functional components or key mission functional subsystems to provide real-time data acquisition services.

5. The data resource pool according to claim 4, characterized in that: The secondary cache centralizes the memory cache, caches necessary data and current data in a central data resource pool, and caches estimated data if necessary conditions are met; The third-level cache is a disk cache, which performs persistent storage in a file or database and provides a disk cache read and write channel.

6. The data resource pool according to claim 1, characterized in that: The data scheduling platform includes an abstract model module and an algorithm scheduling module; The abstract model module transmits the data stream to the corresponding auxiliary model, and the algorithm scheduling module performs scheduling calculations in real time according to the data.

7. The data resource pool according to claim 6, characterized in that: The scheduling calculation uses the following formula to evaluate the stability and performance of the simulation system: Among them, Qi(t) and Zi(t) represent the status of each task in the first task queue and the second task queue respectively, N is the tasks in each task queue, Qi(t) represents the status of the i-th task in the first task queue at time t, and Zi(t) represents the status of the i-th task in the second task queue at time t.

8. The data resource pool according to claim 6, characterized in that: The abstract model module includes a task model, a queue model and an energy consumption model; The task model generates a model according to the task requirements of the data; The queue model generates a model according to the data queue mode; The energy consumption model scheduler offloads data flow tasks to appropriate models via D2D transmission.

9. The data resource pool according to claim 1, characterized in that: The resource pool construction platform includes an overall architecture module, a data management module, a publishing interface module and a collection interface module; The overall architecture module establishes the overall architecture based on the simulation basic data; The data management module has data type management functions and metadata configuration functions, the publishing interface module provides a lightweight external interface for querying various data structures, and the acquisition interface module provides data storage functions for the platform top-level functions.

10. The data resource pool according to claim 3, characterized in that: If the communication sending subsystem receives a custom message packet sent by other working components from a designated data channel, it serializes the custom message packet into a binary stream, fills the binary stream into the payload field of a unified message packet, serializes the unified message packet into a binary stream, obtains a binary message packet, and sends the binary message packet to the cluster communication network.

Images