Acquisition system supporting online configuration

Abstract

The invention discloses an acquisition system supporting online configuration. The acquisition system comprises an event bus, a thread pool, a database connection pool, an equipment instance and an execution unit, loading dependencies required by the system according to the dependency configuration file, wherein the dependencies comprise a dynamic library, global dependencies and device container dependencies; when the system is initialized, loading the dynamic library into the system according to the dynamic library file address provided by the load tag; obtaining a full path name provided by the label according to the app label, searching a corresponding type in a program, instantiating the type by using a dependency injection factory, and sequentially loading equipment instances according to the docks label; after the event is put into the event bus, the event receiver receives the event and distributes the event to the corresponding execution unit according to the name of the event; the system comprises a plurality of execution units, and each execution unit comprises business logic loaded into an independent Domain; each execution unit has the name of a group of executable events, and the event executor can be reloaded during operation.

Description

Technical Field

[0001] The invention belongs to the technical field of data acquisition, and in particular relates to an acquisition system supporting online configuration. Background Art

[0002] In existing data collection systems, the following problems usually exist: the coupling between system components is high, which makes maintenance and expansion difficult; the dependency relationship is unclear, making it difficult to flexibly load and manage the dependencies of different devices; business logic is difficult to update, and each modification of business logic requires restarting the program, causing the collection program to be interrupted; log management and database connection management are not efficient enough, affecting system performance and stability; device connection and data collection are not flexible enough to adapt to multiple protocols and complex network environments; device heartbeat monitoring is not smart enough to accurately determine the online status of the device. Summary of the Invention

[0003] The present invention aims to address the aforementioned issues by providing a data collection method that supports online configuration. Through modular design, dependency injection, an event bus, a log bus, a thread pool, a database connection pool, and other technical means, this method enables flexible system configuration, efficient operation, and stable management. Specifically, it dynamically reloads execution units when business logic changes, enabling online business configuration.

[0004] To achieve the above object, the technical solution adopted by the present invention is: a collection system supporting online configuration, including an event bus, a thread pool, a database connection pool, a device instance, and an execution unit; Load dependencies according to the dependency configuration file. There are load tags, app tags, and docks tags in the root directory of the dependency configuration file. The load tag describes the dynamic library that needs to be loaded when the system loads; The app tag describes the system's global dependencies; The docks tag describes the dependencies of each device instance; During system initialization, the dynamic library is loaded into the system according to the dynamic library file address provided by the load tag; Get the full path name provided by the app tag according to the tag, find the corresponding type in the program, use the dependency injection factory to instantiate the type, and load the device instance in sequence according to the docks tag; When an event is placed into the event bus, it is received by the event receiver and distributed to the corresponding execution unit according to the event name; The system includes several execution units, each of which includes business logic loaded into an independent domain; each execution unit has a set of executable event names, and the event executor can be reloaded at runtime.

[0005] Furthermore, there are multiple device instances in the system. The method of loading device instances in sequence is: Match the corresponding dependency description tag dock under the docks tag according to the device instance category; there are multiple device dependency description tags dock under the docks tag, and each dependency description tag dock provides a name attribute as the device category; Read the class attribute value of the dependency description tag dock as the full path name of the device container type. The full path name is used to find the type of the device container. Instantiate the device container type through dependency injection factory; Read the dependent tags in the dependency description tag dock in sequence and load the dependencies.

[0006] Furthermore, dependency injection factories include global factories and device-level factories. There is only one global factory, and the number of device-level factories corresponds to the number of device instances. The device-level factories are within the global factory. The dependency injection factory can instantiate objects according to the type and register the objects with the dependency injection factory; When injecting dependencies, the global factory will only look for dependencies of the current factory. When injecting dependencies, the device-level factory will look for dependencies in the global factory if it fails to find dependencies of the current factory. Whether it is a global factory or a device-level factory, when it cannot find a corresponding dependency, it tries to instantiate a dependency. If this dependency is not of a certain type but an interface or an abstract class, the dependency injection factory reports an exception.

[0007] Furthermore, the process of executing unit loading is as follows: Create a new domain; Load the dll containing the business logic; Create a dependency injection factory in the Domain; Find all methods in the dll that are marked with the Event attribute; Store the methods annotated with the Event attribute in a list and name them with the Name provided in Event; The execution unit receives the event, matches the corresponding method according to the event name, and executes it.

[0008] Furthermore, the reloading process of the execution unit is as follows: The execution unit will be marked as Hold state and stop receiving new events; At the same time, new execution units are loaded and events are received; The executor in the Hold state is uninstalled after the current business logic is executed.

[0009] Furthermore, the system also includes a log bus and a log module; the system includes three log interfaces: Debug\Info\Error, corresponding to three levels of logs respectively; Through the log interface, the system puts the log into the log bus; The log module puts the logs pushed by the log bus into the buffer; Save the buffer contents to the hard disk in different levels at regular intervals; When the log file is full of 4M, it will be compressed and saved.

[0010] Furthermore, the system's database connection pool is compatible with a variety of database interfaces that implement ado.net; the database connection pool sets the initial number of connections, the upper limit of connections, and the timeout period; the database connection pool supports dynamic scaling, which is implemented as follows: During initialization, the database connection is pushed into an Active stack and recorded as the current number of connections + 1; When acquiring a connection, it is obtained from the top of the stack, following the first-in, last-out principle of the stack. When the database connection is closed, the link channel is not closed, but the database connection is put back into the active stack; When a connection needs to be obtained and there is no database connection in the stack, if the current number of connections is less than the maximum number of connections, the database connection pool opens a new connection and returns it. Otherwise, the database connection pool waits for the previous connection to be put back on the stack. If no connection is put back on the stack within a given time, the database connection pool reports an exception. The database connection pool periodically scans the connections in the Active stack. If a connection remains in the Active stack for longer than a set threshold, the connection is closed.

[0011] Furthermore, the device instance features: each device instance has a data driver, which is an abstraction of the device connection. It can be implemented with multiple protocols such as OPC protocol and MQ, providing a unified read and write interface.

[0012] Furthermore, the device instance also has scheduled task triggers and data change triggers. The scheduled event trigger is used to register periodic tasks with an accuracy of 100ms. The scheduled event trigger regularly pushes events to the event bus according to the registered event period. The data change trigger registers the data name that needs to be detected, and detects the data in real time based on the registered data name. When the data changes, the trigger event is pushed to the event bus.

[0013] Furthermore, the device instance includes a heartbeat module, and the monitoring of the heartbeat module includes timed monitoring and pulse monitoring; in timed monitoring, the heartbeat module reads the heartbeat data regularly, and if the reading fails more than a set number of times, it is regarded as disconnected, otherwise it is online; in pulse monitoring, the heartbeat module detects changes in pulse data, and when the pulse data does not change for more than a set time, it is regarded as disconnected, otherwise it is online; when it is determined to be disconnected, the timed events and trigger events will stop triggering events; when the heartbeat module determines to be online, the event resumes triggering.

[0014] The present invention has the following beneficial effects: When the system of the present invention is implemented, it first loads the system's required dependencies, including dynamic libraries, global dependencies, and device container dependencies, according to the dependency configuration file "depend.xml." It then performs type instantiation and dependency injection through a dependency injection factory, initializing each system module. Then, the modules achieve decoupling and information transfer between system components via an event bus and a log bus. The log bus resolves the coupling between the log module and other modules, while the event bus resolves the coupling between other modules. This improves the system's flexibility and maintainability. Thread pools and database connection pools are utilized to improve the system's concurrent processing capabilities and resource utilization. Similar to connection pools, thread pools are initialized by creating a corresponding number of threads based on the configuration file and entering a wait state. When a task needs to be executed, the thread pool randomly wakes up a thread to execute the task. After the task is completed, the thread returns to a wait state, thereby achieving thread resource reuse. This avoids frequent thread creation and exit, conserving system resources and improving concurrency (the ability to execute multiple tasks simultaneously).

[0015] For device instances, data-driven methods are used to achieve connection and data interaction between devices with different protocols. Scheduled task triggers and data change triggers are used to achieve periodic and real-time collection of device data. The heartbeat module is used to monitor the online status of the device to ensure the stability and reliability of data collection.

[0016] The execution unit loads business logic through an independent domain and can be dynamically updated at runtime, improving the flexibility and maintainability of the system.

[0017] The log module uses a hierarchical log interface and buffer mechanism to achieve efficient log recording and storage, facilitating system debugging and operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is an architecture diagram of an acquisition system supporting online configuration according to an embodiment of the present invention.

[0019] Figure 2 This is a lifecycle diagram of an execution unit of an acquisition system supporting online configuration according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to embodiments and accompanying drawings.

[0021] The embodiment of the collection system supporting online configuration includes an event bus, a log bus, a thread pool, a database connection pool, a device instance (device heartbeat, data driver, timed task trigger, data change trigger), an execution unit, and a log module. Figure 1 shown.

[0022] Load dependencies according to the dependency configuration file "depend.xml". The dependency configuration list is in XML format and has the load tag, app tag, and docks tag in the root directory.

[0023] in: The load tag describes the dll that needs to be loaded when the system loads; The app tag describes the system's global dependencies; The docks tag describes the dependencies of each device instance.

[0024] During system initialization, perform the following operations: 1) Load the dynamic library into the system according to the dynamic library file address provided by the load tag in the dependency configuration file.

[0025] 2) Read the "dependent" subtags under the app tag in sequence, obtain the full path name provided by the tag, and find the corresponding type in the program.

[0026] 3) Use a "Dependency Injection Factory" to instantiate the type.

[0027] 4) Load the device instance. There are multiple device instances in an acquisition system. Load the device instances in sequence: Match the corresponding dependency description tag dock under the docks tag according to the device instance category; there are multiple device dependency description tags dock under the docks tag, and each dependency description tag dock provides a name attribute as the device category; Read the class attribute value of the dependency description tag dock as the full path name of the device container type. The full path name is used to find the type of the device container. Instantiate the device container type through dependency injection factory; Read the dependent tags in the dependency description tag dock in sequence and load the dependencies.

[0028] The dependency injection factory is characterized by being divided into two levels: global and device. There is only one global factory, and there are as many device-level factories as there are device instances. The device-level factories are included in the global factory.

[0029] The dependency injection factory can instantiate objects according to the type and register the objects with the factory. After the factory instantiates the type, it searches for registered objects according to the type of the property marked with the [AutoWrire] attribute and assigns them to the instantiated objects. This process is called dependency injection.

[0030] When injecting dependencies, the global-level factory will only look for dependencies of the current factory. When injecting dependencies, the device-level factory will look for dependencies in the global factory if it fails to find dependencies of the current factory.

[0031] Whether it is a global factory or a device-level factory, when it cannot find a corresponding dependency, it tries to instantiate a dependency. If this dependency is not of a certain type but an interface or an abstract class, the dependency injection factory reports an exception.

[0032] The system includes an event bus. When an event is placed on the event bus, it is received by an event receiver and distributed to the corresponding execution unit based on the event name. This method effectively decouples the event trigger source and the execution unit, so that when an exception occurs in the execution unit, it will not affect the entire system.

[0033] The system includes execution units. Multiple execution units can exist in the system at the same time. The executor contains specific business logic and is loaded into an independent domain. Each execution unit has a set of executable event names, and the event executor can be reloaded at runtime.

[0034] like Figure 2 As shown, the process of executing unit loading is as follows: Create a new domain; Load the dll containing the business logic; Create a dependency injection factory in the Domain; Find all methods in the dll that are marked with the Event attribute; Store the methods annotated with the Event attribute in a list and name them with the Name provided in Event; The execution unit receives the event, matches the corresponding method according to the event name, and executes it.

[0035] The reloading process of the execution unit is as follows: The execution unit will be marked as Hold state and stop receiving new events; At the same time, new execution units are loaded and events are received; The executor in the Hold state is uninstalled after the current business logic is executed.

[0036] The system includes a log bus and a log module. There are three log interfaces in the system: Debug, Info, and Error, corresponding to three levels of logs. Through the log interface, the system puts the log into the log bus; The log module puts the logs pushed by the log bus into the buffer; Save the buffer contents to the hard disk in different levels at regular intervals; When the log file reaches 4MB, it will be compressed and saved. The bus setting changes the original situation where module A calls module B to a situation where module A does not directly call module B. Instead, module A pushes data to the bus, and module B listens to the bus and performs corresponding processing after receiving the bus data, thereby eliminating the coupling between modules A and B.

[0037] The system's database connection pool is compatible with a variety of database interfaces that implement ado.net. The database connection pool can set the initial number of connections, the upper limit of connections, and the timeout period; the database connection pool supports dynamic scaling, which is implemented as follows: During initialization, the database connection is pushed into an Active stack and recorded as the current number of connections + 1; When acquiring a connection, it is obtained from the top of the stack, following the first-in, last-out principle of the stack. When the database connection is closed, the link channel is not closed, but the database connection is put back into the active stack; When a connection is needed and there is no database connection in the stack, if the current number of connections is less than the maximum number of connections, the connection pool will open a new connection and return it. Otherwise, the connection pool waits for the previous connection to be put back on the stack. If no connection is put back on the stack within the given time, the database connection pool throws an exception.

[0038] The database connection pool periodically scans the connections in the Active stack. If a connection remains in the stack for longer than a set threshold, the connection is closed.

[0039] Each device instance has a data driver, which is an abstraction of the device connection. The data driver is implemented by the OPC protocol, MQ, or other protocols, providing a unified read and write interface.

[0040] Device instances have scheduled task triggers and data change triggers. Scheduled event triggers can register periodic tasks with an accuracy of 100ms. Scheduled event triggers regularly push events to the event bus according to the registered event period. Data change triggers can register the name of the data to be detected. Data change triggers can detect data in real time based on the registered data name, and trigger events to be pushed to the event bus when the data changes.

[0041] The device instance has a heartbeat module. Heartbeat monitoring includes two modes: timed monitoring and pulse monitoring. The heartbeat module monitors the heartbeat data regularly. If the reading fails multiple times, the connection will be considered disconnected. Otherwise, the connection will be considered online. Pulse monitoring: The heartbeat module detects changes in pulse data. If the pulse data does not change for a long time, it is considered disconnected; otherwise, it is online. When the connection is determined to be disconnected, the timed events and trigger events will stop triggering; when the heartbeat module determines that the event is online, the event will resume triggering.

[0042] The above embodiments are only for illustrating the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A collection system supporting online configuration, characterized by: Including event bus, thread pool, database connection pool, device instance, and execution unit; Load dependencies according to the dependency configuration file. There are load tags, app tags, and docks tags in the root directory of the dependency configuration file. The load tag describes the dynamic library that needs to be loaded when the system loads; The app tag describes the system's global dependencies; The docks tag describes the dependencies of each device instance; During system initialization, the dynamic library is loaded into the system according to the dynamic library file address provided by the load tag; Get the full path name provided by the app tag according to the tag, find the corresponding type in the program, use the dependency injection factory to instantiate the type, and load the device instance in sequence according to the docks tag; When an event is placed into the event bus, it is received by the event receiver and distributed to the corresponding execution unit according to the event name; The system includes several execution units, each of which includes business logic loaded into an independent domain; each execution unit has a set of executable event names, and the event executor can be reloaded at runtime.

2. The acquisition system supporting online configuration according to claim 1, characterized in that: There are multiple device instances in the system. The method to load device instances in sequence is: Match the corresponding dependency description tag dock under the docks tag according to the device instance category; there are multiple device dependency description tags dock under the docks tag, and each dependency description tag dock provides a name attribute as the device category; Read the class attribute value of the dependency description tag dock as the full path name of the device container type. The full path name is used to find the type of the device container. Instantiate the device container type through dependency injection factory; Read the dependent tags in the dependency description tag dock in sequence and load the dependencies.

3. The acquisition system supporting online configuration according to claim 1, characterized in that: The dependency injection factory includes a global factory and a device-level factory. There is only one global factory, and the number of device-level factories corresponds to the number of device instances. The device-level factory is within the global factory. The dependency injection factory can instantiate objects according to the type and register the objects with the dependency injection factory; When injecting dependencies, the global factory will only look for dependencies of the current factory; When injecting dependencies, the device-level factory searches for dependencies in the global factory if no results are found in the current factory. Whether it is a global factory or a device-level factory, when it cannot find a corresponding dependency, it tries to instantiate a dependency. If this dependency is not of a certain type but an interface or an abstract class, the dependency injection factory reports an exception.

4. The acquisition system supporting online configuration according to claim 1, characterized in that: The process of executing unit loading is as follows: Create a new domain; Load the dll containing the business logic; Create a dependency injection factory in the Domain; Find all methods in the dll that are marked with the Event attribute; Store the methods annotated with the Event attribute in a list and name them with the Name provided in Event; The execution unit receives the event, matches the corresponding method according to the event name, and executes it.

5. The acquisition system supporting online configuration according to claim 1, characterized in that: The reloading process of the execution unit is as follows: The execution unit will be marked as Hold state and stop receiving new events; At the same time, new execution units are loaded and events are received; The executor in the Hold state is uninstalled after the current business logic is executed.

6. The acquisition system supporting online configuration according to claim 1, characterized in that: The system also includes a log bus and a log module. The system includes three log interfaces: Debug, Info, and Error, corresponding to three levels of logs. Through the log interface, the system puts the log into the log bus; The log module puts the logs pushed by the log bus into the buffer; Save the buffer contents to the hard disk in different levels at regular intervals; When the log file is full of 4M, the file will be compressed and saved.

7. The acquisition system supporting online configuration according to claim 1, characterized in that: The system's database connection pool is compatible with a variety of database interfaces that implement ado.net. The database connection pool sets the initial number of connections, the upper limit of connections, and the timeout period. The database connection pool supports dynamic scaling, which is implemented as follows: During initialization, the database connection is pushed into an Active stack and recorded as the current number of connections + 1; When acquiring a connection, it is obtained from the top of the stack, following the first-in, last-out principle of the stack. When the database connection is closed, the link channel is not closed, but the database connection is put back into the active stack; When a connection needs to be obtained and there is no database connection in the stack, if the current number of connections is less than the maximum number of connections, the database connection pool opens a new connection and returns it. Otherwise, the database connection pool waits for the previous connection to be put back on the stack. If no connection is put back on the stack within a given time, the database connection pool reports an exception. The database connection pool periodically scans the connections in the Active stack. If a connection remains in the Active stack for longer than a set threshold, the connection is closed.

8. The acquisition system supporting online configuration according to claim 1, characterized in that: The device instance includes a data driver, which provides a unified read and write interface.

9. The acquisition system supporting online configuration according to claim 1, characterized in that: Device instances include scheduled task triggers and data change triggers; The timed event trigger is used to register periodic tasks with an accuracy of 100ms. The timed event trigger regularly pushes events to the event bus according to the registered event period. The data change trigger registers the data name that needs to be detected, and detects the data in real time based on the registered data name. When the data changes, the trigger event is pushed to the event bus.

10. The acquisition system supporting online configuration according to claim 1, characterized in that: The device instance includes a heartbeat module, and the monitoring of the heartbeat module includes timed monitoring and pulse monitoring; Timed monitoring: the heartbeat module reads heartbeat data regularly. If the reading fails for more than the set number of times, the connection is considered disconnected, otherwise it is online. Pulse monitoring: The heartbeat module detects changes in pulse data. If the pulse data does not change for a set time, it is considered disconnected, otherwise it is online. When it is determined to be disconnected, the timed events and triggered events will stop triggering events; The event recovery is triggered when the heartbeat module determines that it is online.

Images