A method and system for managing stateful plug-ins with built-in stream processing in a time series database

Abstract

The present invention relates to the technical field of time series databases, and in particular to a method and system for managing stateful plug-ins with built-in stream processing in a time series database; the method and system are designed with waiting state, dormant state, ready state and running state for three types of plug-ins, namely, extraction plug-ins, processing plug-ins and delivery plug-ins; through the setting of a message module, a data module, a management module and a monitoring module, state conversion can be realized in the process of stream processing, and a more flexible custom programming design strategy is given to the three types of plug-ins; by switching the state of adding a plug-in instance to a plug-in instance during the process of stream processing from loading a plug-in instance to destroying a plug-in instance, the efficient processing capability of a time series database for stream processing tasks in a complex network environment can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series databases, and in particular to a method and system for managing a stateful plug-in for built-in stream processing in a time series database. Background Art

[0002] With the rapid development of IoT technology, various industries have begun to generate a large amount of data with timestamp characteristics, including sensor data, log data, equipment monitoring data, etc. These data not only show a high degree of real-time performance, but also reflect significant time series correlation. In order to effectively store and retrieve data with time series characteristics, time series databases, as a professional time series data storage solution, have gradually been widely used.

[0003] Time series databases usually use built-in stream processing to process data in real time and push it outward, including three stages: data extraction, data processing, and data delivery. Each stage is implemented by three types of plug-ins: extraction plug-in, processing plug-in, and delivery plug-in. Among them, the extraction plug-in is responsible for obtaining data from the storage engine, the processing plug-in is responsible for processing the data, and the delivery plug-in is responsible for sending the data to the external third-party system. All three types of plug-ins support user-defined programming logic, and the execution code of the plug-in can be flexibly written according to actual needs, thereby realizing the highly flexible processing capabilities of the time series database and adapting to diverse data processing needs.

[0004] For the plug-ins that have been loaded into the time series database and configured for execution, the stream processing engine of the time series database will dynamically generate a series of stream processing tasks in a multi-threaded manner. The stream processing tasks will strictly follow the established extraction, processing, and delivery order, and execute the corresponding instances of the three types of plug-ins in sequence. The plug-in instances have no state during the execution process and do not involve state transitions.

[0005] When time series databases are currently applied in various industries, the network environment is diverse and complex due to the wide geographical distribution of IoT edge devices. In the process of transmitting device data to the time series database, delays, losses, instability and other problems often occur, and the integrity and timeliness of the data cannot be guaranteed. As a result, the existing stateless plug-ins of the time series database are difficult to meet the actual stream processing needs, and have the following deficiencies and limitations:

[0006] (1) During the data extraction phase, if the extraction plug-in fails to successfully obtain complete data within the required time period, it will have a significant impact on the execution logic of the subsequent processing plug-in and delivery plug-in, which may lead to interruption of the stream processing process or inaccurate results. Existing stateless extraction plug-ins, even user-defined programming, cannot achieve key functions such as waiting for data to arrive and filling in missing data.

[0007] (2) In the data processing stage, more and more application scenarios require processing plug-ins to perform specific processing on time series data (such as correlation analysis of data from multiple devices). Existing stateless processing plug-ins are often unable to effectively execute specific processing functions programmed by users when faced with out-of-order and delayed data arrival. Summary of the invention

[0008] In view of this, the present invention proposes a method and system for managing stateful plug-ins with built-in stream processing in a time series database. As an internal integrated component of the time series database, four states of waiting, sleeping, preparing and running are designed for the three types of plug-ins for extraction, processing and delivery, and state transitions are realized during the stream processing process. The three types of plug-ins are given more flexible custom programming design strategies, thereby improving the efficient processing capabilities of the time series database for stream processing tasks when dealing with complex network environments.

[0009] In a first aspect of the present invention, a time series database built-in stream processing stateful plug-in management system of the present invention comprises: a message module, a data module, a management module and a monitoring module; wherein,

[0010] The message module is responsible for maintaining the life cycle status information of the plug-in instance and coordinating the communication between other modules; the plug-in instance includes extracting plug-in instances, processing plug-in instances and delivering plug-in instances; the life cycle status information includes waiting state, dormant state, ready state and running state;

[0011] The data module is responsible for receiving, storing and collating the required data transmitted between plug-in instances; the required data includes data of a specific time period and data of a specific device;

[0012] The management module is responsible for loading the plug-in instance and placing the loaded plug-in instance in a waiting state, saving the filtering conditions of the data required by the plug-in instance in the dormant state, providing the filtering conditions of the data required by the plug-in instance in the ready state, and removing the filtering conditions of the data required by the plug-in instance in the running state;

[0013] The monitoring module is responsible for monitoring the usage of server resources and processing the sleep duration of the plug-in instance in the sleep state, and dynamically adjusting the system resources according to the monitoring results; and converting the plug-in instance from the sleep state to the ready state.

[0014] In a second aspect of the present invention, the present invention further provides a method for managing a stateful plug-in for built-in stream processing in a time series database, the method comprising:

[0015] Loading a plug-in instance, the plug-in instance enters a waiting state; the plug-in instance includes an extraction plug-in instance, a processing plug-in instance and a delivery plug-in instance;

[0016] If the plug-in instance reaches the dormancy condition, the plug-in instance enters the dormant state, saves the filtering conditions of the data required by the plug-in instance in the dormant state, and monitors the dormancy time of the plug-in instance;

[0017] If the plug-in instance does not reach the dormancy condition, the plug-in instance enters a ready state, and the plug-in instance transmits required data in the ready state; the required data includes data in a specific time period and data of a specific device;

[0018] If the plug-in instance is released from the dormant state, the plug-in instance enters the ready state, and the plug-in instance transmits the required data in the ready state;

[0019] Filtering required data transmitted between plug-in instances based on the saved filtering conditions, the plug-in instances entering a running state;

[0020] If the delivery target of the delivery plug-in instance is the address of the current database, then after the processing plug-in instance is executed, the extraction plug-in instance, the processing plug-in instance and the delivery plug-in instance are destroyed;

[0021] If the delivery target of the delivery plug-in instance is other addresses, after the delivery plug-in instance is executed, the extraction plug-in instance, the processing plug-in instance and the delivery plug-in instance are destroyed.

[0022] In the embodiment of the present invention, four states, namely, waiting state, sleeping state, ready state and running state, are set correspondingly for extracting plug-in instances, processing plug-in instances and delivering plug-in instances, so as to realize state transition in the process of stream processing. By maintaining the state switching of plug-in instances, the management of plug-in instances is realized, thereby improving the efficient processing capability of the time series database for stream processing tasks when dealing with complex network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of the structure of a time series database according to an embodiment of the present invention;

[0024] Figure 2 is an example state transition diagram of an embodiment of the present invention;

[0025] Figure 3 is a message module diagram of an embodiment of the present invention;

[0026] Figure 4 is a data module diagram of an embodiment of the present invention;

[0027] Figure 5 is a process diagram of capturing and distributing data by a data sorting unit according to an embodiment of the present invention;

[0028] Figure 6 This is a processing diagram after the collection capacity of an embodiment of the present invention reaches the upper limit;

[0029] Figure 7 is a management module diagram of an embodiment of the present invention;

[0030] Figure 8 is a monitoring module diagram of an embodiment of the present invention;

[0031] Fig. 9 is a loading example diagram of an embodiment of the present invention;

[0032] Fig.10 is a plug-in instance sleep diagram of an embodiment of the present invention;

[0033] Fig.11 This is a normal sleep release diagram of an example of an embodiment of the present invention;

[0034] Fig.12 is an example abnormal sleep release diagram of an embodiment of the present invention;

[0035] Fig.13 The flowchart of stopping a stream processing thread according to an embodiment of the present invention is shown in FIG. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] like Figure 1 As shown, the existing time series database structure includes a stream processing engine and a storage engine, which can interact and communicate with an external third-party system; the stream processing engine is responsible for processing real-time or near real-time data streams, and can be implemented by three types of plug-ins: extraction plug-ins, processing plug-ins, and delivery plug-ins. The extraction plug-in instance is used to extract data from the time series database; the processing plug-in instance is used to process the extracted data; the delivery plug-in instance is used to send the processed data to an external third-party system. The storage engine is responsible for persistently storing the processed time series data on disk or in memory. On this basis, the present invention embeds a built-in stream processing stateful plug-in management system, and the stream processing stateful plug-in management system of the present invention interacts with the stream processing engine.

[0038] A time series database built-in stream processing stateful plug-in management system according to an embodiment of the present invention includes four modules: a message module, a data module, a management module and a monitoring module.

[0039] The message module is responsible for maintaining the life cycle status information of the plug-in instance and coordinating the communication between other modules; the plug-in instance includes extracting plug-in instances, processing plug-in instances and delivering plug-in instances; the life cycle status information includes waiting state, dormant state, ready state and running state;

[0040] The data module is responsible for receiving, storing and collating the required data transmitted between plug-in instances; the required data includes data of a specific time period and data of a specific device;

[0041] The management module is responsible for loading the plug-in instance and placing the loaded plug-in instance in a waiting state, saving the filtering conditions of the data required by the plug-in instance in the dormant state, providing the filtering conditions of the data required by the plug-in instance in the ready state, and removing the filtering conditions of the data required by the plug-in instance in the running state;

[0042] The monitoring module is responsible for monitoring the usage of server resources and processing the sleep duration of the plug-in instance in the sleep state, and dynamically adjusting the system resources according to the monitoring results; and converting the plug-in instance from the sleep state to the ready state.

[0043] Through the design of the above modules, the plug-in instance can switch between four states: waiting, sleeping, preparing and running. The state transition diagram is as follows: Figure 2 The four states and their corresponding operations are as follows:

[0044] Waiting state: The plug-in instance enters the waiting state. In this state, the plug-in instance will verify its own configuration parameters, input and output data formats, etc. to ensure that they meet the system requirements. At the same time, the plug-in instance will also detect the current system environment, including computing resources, network connectivity, etc., to ensure that it can run smoothly. Finally, the plug-in instance will verify the connection with the message module, data module, management module and monitoring module. Only after all verifications are passed, the plug-in instance will move to the next state.

[0045] Sleep state: When the plug-in instance enters the sleep state, the monitoring module will continue to monitor its sleep duration. Once the threshold is exceeded, the plug-in instance will be forced to wake up. During the sleep period, the management module will save the filter conditions for the data required by the plug-in instance, the message module will update the status information of the plug-in instance, and the data module will continue to receive the required data of the plug-in instance from other plug-in instances.

[0046] Ready state: The plug-in instance is in the process of filtering data, obtaining filtering conditions from the management module and filtering its own data.

[0047] Running state: The plug-in instance enters the normal running state and starts processing data obtained from the data module or the extraction plug-in instance. At this time, the management module will remove the filter conditions of the plug-in instance, and the message module will also update the status information to the running state during this period.

[0048] The four modules, message module, data module, management module and monitoring module, are introduced as follows.

[0049] The message module is responsible for saving the life cycle status information of the plug-in instance and coordinating the communication between other modules and the plug-in instance. It can provide other modules with query services for the life cycle status information of the plug-in instance, such as Figure 3 As shown, it includes: an information storage unit and a message distribution unit.

[0050] The information storage unit is responsible for storing the life cycle status information of all plug-in instances, including plug-in instance type (such as "extraction plug-in"), current state (such as dormant state), etc. Other modules can obtain all information of the plug-in instance through this unit.

[0051] The message distribution unit is responsible for providing a decoupled communication mechanism between modules, so that other modules do not need to worry about how to interact with the plug-in instance, but only need to focus on the implementation of functions within their own responsibilities. This unit is responsible for receiving requests from other modules and forwarding the requests to the corresponding plug-in instance.

[0052] Through the above design, the message module can manage the life cycle status information of the plug-in instance and support collaboration between modules.

[0053] The data module is used to receive, store and organize the data transmitted between the plug-in instances, and the required data includes data of a specific time period and data of a specific device; Figure 4 As shown, it includes: a data receiving unit, a data sorting unit, and a data recovery unit.

[0054] The data receiving unit is responsible for receiving two types of data shared by the plug-in instance, one is data in a specific time period (such as receiving data within "2024-06-025T16:26:00~2024-06-025T17:26:00"), and the other is data of a specific device (such as receiving data of device "root.l n.wf01.wt01").

[0055] The data sorting unit is responsible for monitoring the data receiving unit, receiving the required data of all plug-in instances in a dormant state, classifying and storing the required data according to the plug-in instance type, and sending the classified and stored plug-in instances and the related information of the required data to the message module or the data recovery unit. Specifically, when the plug-in instance is dormant, a collection will be created for the plug-in instance in this unit to receive the data required by the plug-in instance, and the size of the collection is equal to the amount of data required by the plug-in instance. Figure 5 As shown in , after the data receiving unit receives the data, the data sorting unit will capture the data, and determine which plug-in instances the data belongs to according to the filtering conditions, and store the data in the corresponding plug-in instance set, taking n data sets as an example. When a plug-in instance set reaches the upper limit of capacity, such as Figure 6As shown, the data sorting unit will check the current state of the plug-in instance. If the current state of the plug-in instance can receive data, the data sorting unit will package the plug-in instance name and the corresponding data and send them to the message distribution unit of the message module, which will forward them to the target plug-in instance. If the plug-in instance cannot currently receive data, the data sorting unit will send the data to the data recovery unit for subsequent processing.

[0056] The data recovery unit is responsible for processing the plug-in instance that is not sent to the message module and the related information of the data required by it, that is, the data that cannot be sent to the plug-in instance. When the unit receives data that cannot be sent to the plug-in instance from the data sorting unit, it will first identify the type of data. If the data type is data for a specific time period, the data recovery unit will create a delivery plug-in instance, and the delivery plug-in instance will send the data to an external third-party system to ensure that the data is not lost. If the data type is data for a specific device, since this type of data is usually copied and filtered from other plug-in instances, the data recovery unit will directly discard it to avoid repeated sending and occupying system resources.

[0057] Through the above design, the data module can effectively receive, organize and process various types of data transmitted between plug-in instances to ensure that the data can be fully utilized.

[0058] The management module is responsible for managing the loading of plug-in instances and the filtering conditions of the data required by the plug-in instances in a dormant state, ensuring that the system is initialized correctly and can correctly filter the data required by the dormant plug-in instances. Figure 7 As shown, it includes: an instance loading unit and a filtering condition unit.

[0059] The instance loading unit is responsible for loading the plug-in instance according to the user configuration information, which includes: maximum sleep time (such as "2s"), maximum data generation time misalignment tolerance (such as "20%"), maximum data missing tolerance (such as "10%"), etc. After loading is completed, the plug-in instance information is registered to the information storage unit of the message module for query by other modules.

[0060] The filtering condition unit is responsible for storing the filtering conditions of the data required by the plug-in instance in the dormant state. The filtering condition format has two types: data of a specific time period, such as {created_time_from:2024-06-025T16:26:00, created_time_to:2024-06-025T17:26:00}; data of a specific device, such as {device_name:root.ln.wf01.wt01}. The specific process is as follows:

[0061] 1. When the plug-in instance enters the dormant state, the filtering condition unit obtains the filtering conditions of the data required by the plug-in instance through the message distribution unit of the message module and saves them.

[0062] 2. When the plug-in instance resumes operation, the filter condition unit will filter the required data according to the previously saved filter conditions to ensure that the plug-in instance can obtain complete data.

[0063] 3. When the plug-in instance completely exits the dormant state, the filter condition unit receives a request to remove the filter condition of the plug-in instance through the message distribution unit of the message module, and then deletes the corresponding filter condition from the filter condition unit.

[0064] Through the above design, the management module can ensure that the system correctly loads the plug-in instance and provides filtering support for the required data for the plug-in instance in the dormant state.

[0065] The monitoring module is responsible for monitoring the usage of system resources and the sleep duration of the plug-in instances in the sleep state, and dynamically adjusting according to the monitoring results. Figure 8 As shown, it includes: a sleep monitoring unit and a system monitoring unit.

[0066] The sleep monitoring unit is responsible for monitoring the sleep duration of the plug-in instance in sleep mode. Once it is found that the sleep time of a plug-in instance exceeds the maximum limit set by the user (such as "2s"), the sleep monitoring unit will force the plug-in instance to wake up. After waking up, the unit will notify the information storage unit of the message module to update the status of the plug-in instance, and at the same time obtain the data filtered during the sleep state of the plug-in instance from the data sorting unit of the data module, and remove the filter condition of the plug-in instance from the filter condition unit in the management module to ensure that the plug-in instance can resume operation smoothly.

[0067] The system monitoring unit is responsible for monitoring the average usage of the system's RAM and CPU. Once the average usage is found to be too high or too low, the system monitoring unit will take corresponding dynamic adjustment measures:

[0068] 1. When the system monitoring unit finds that the average RAM usage is too high or the average CPU usage reaches the preset maximum threshold (such as 90%), in order to relieve resource pressure, the system monitoring unit will randomly select some running stream processing task threads and send a stop instruction through the message distribution unit of the message module. If there are plug-in instances in a dormant state at this time, the system monitoring unit will also give priority to waking up these plug-in instances, and then execute the operation of stopping the stream processing task thread.

[0069] 2. When the system monitoring unit finds that the average RAM usage is too low or the average CPU usage drops to the preset minimum threshold (such as set to 20%), in order to improve resource utilization efficiency, the system monitoring unit will notify the instance loading unit of the management module through the message distribution unit of the message module to add a new stream processing task thread to meet the system's processing requirements.

[0070] The calculation frequency parameters required to calculate the average system RAM and CPU usage are configured by default as follows and can be customized by the user, for example, the calculation is performed every 60 seconds by default. The calculation formula is:

[0071]

[0072]

[0073] Where T a is the average load, T(RAM) and T(CPU) are the usage rates of RAM and CPU at a certain moment, t is the calculation frequency, and n is the calculation duration.

[0074] Through the above dynamic monitoring and adjustment mechanism, the system monitoring unit can ensure the reasonable allocation and efficient use of system resources, and avoid performance problems caused by excess or insufficient resources.

[0075] Through the functional collaboration of the above modules, the life cycle status of the plug-in instance can be effectively scheduled and controlled, thereby supporting the core management processes of stateful plug-ins such as plug-in instance loading, plug-in instance dormancy, plug-in instance normal wake-up, plug-in instance abnormal wake-up, and stream processing task thread stop.

[0076] Plugin instance loading: Fig. 9 As shown, the instance loading unit in the management module completes the loading of the plug-in instance according to the information pre-configured by the user (such as maximum sleep time, data alignment tolerance, etc.). After loading is completed, the plug-in instance will enter a waiting state to verify whether the current environment affects its own operation. Only when the current system environment can guarantee the operation, the instance loading unit will send the detailed information of the plug-in instance, including the plug-in instance name, data requirement type, startup parameters, etc., to the information storage unit of the message module. After receiving the plug-in instance information, the information storage unit will save it in RAM for subsequent queries by other modules.

[0077] Plugin instance sleep: Fig.10 As shown in the figure, when the plug-in instance enters the dormant state, the system will execute the following process:

[0078] 1. The plug-in instance first sends a sleep message to the message distribution unit of the message module.

[0079] 2. After receiving the sleep request, the message distribution unit notifies the information storage unit of the message module to update the state of the plug-in instance from the running state to the sleep state.

[0080] 3. The message distribution unit sends the type information of the data required by the plug-in instance and the plug-in instance name to the filter condition unit of the management module. The filter condition unit adds corresponding data filter conditions based on the received information for subsequent data filtering of the plug-in instance in operation.

[0081] 4. The data receiving unit of the data module will add the plug-in instance to the data receiving list managed by itself, so as to subsequently capture and cache the data required by the plug-in instance.

[0082] 5. The message distribution unit will notify the sleep monitoring unit of the monitoring module, which will start monitoring the sleep duration of the plug-in instance.

[0083] The plugin instance wakes up normally: Fig.11 As shown in the figure, when the data sorting unit in the data module completes the collection of data required by a plug-in instance, the system will execute the following process of normal wake-up of the plug-in instance:

[0084] 1. The data sorting unit first checks whether the current state of the plug-in instance is a dormant state.

[0085] 2. If the data sorting unit finds that the plug-in instance is in a dormant state, the data sorting unit will notify the filter condition unit of the management module through the message distribution unit of the message module to remove the data filter condition previously saved by the plug-in instance. At the same time, it will notify the dormancy monitoring unit of the monitoring module to cancel the dormancy duration monitoring of the plug-in instance.

[0086] 3. The dormant monitoring unit will wake up the plug-in instance, and then the message distribution unit will send the complete data collected by the data sorting unit to the plug-in instance for processing.

[0087] 4. The data sorting unit notifies the information storage unit of the message module to update the state of the plug-in instance to the running state.

[0088] 5. If the data collation unit finds that the plug-in instance is not in a dormant state, it sends the data to the data recovery unit of the data module and performs different processing according to the data type:

[0089] 6. For data in a specific time period, the recycling unit creates a delivery plug-in instance and sends the data to an external third-party system through the delivery plug-in instance.

[0090] 7. For data of specific devices, since such data is usually copied and filtered from other plug-in instances, the data recovery unit will directly discard it to avoid occupying too many system resources.

[0091] The plugin instance wakes up abnormally: Fig.12 As shown in the figure, when the sleep monitoring unit in the monitoring module detects that the sleep duration of a plug-in instance has exceeded the preset maximum threshold, the system will execute the following wake-up process:

[0092] 1. The sleep monitoring unit will first end the sleep duration monitoring of the plug-in instance;

[0093] 2. The sleep monitoring unit sends the end monitoring information to the message distribution unit of the message module as an abnormal wake-up request.

[0094] 3. After receiving the abnormal wake-up request, the message distribution unit will notify the filter condition unit of the management module to remove the data filter condition previously saved by the plug-in instance.

[0095] 4. At the same time, the data sorting unit of the data module will forward all the data accumulated during the dormancy period of the plug-in instance to the plug-in instance through the message distribution unit.

[0096] 5. After receiving the data, the message distribution unit will immediately send it to the target plug-in instance and force the plug-in instance to wake up and put it into operation.

[0097] 6. The message distribution unit notifies the information storage unit of the message module to update the status of the plug-in instance in the system to running.

[0098] The stream processing task thread stops: Fig.13 As shown in the figure, when the system needs to reduce the number of threads of a stream processing task, that is, to adjust the number of plug-in instances running the stream processing task, the following process of stopping the stream processing thread is executed:

[0099] 1. First, the system checks if there are any dormant plugin instances in the thread.

[0100] 2. If no dormant plug-in instance is found in the thread, the system will wait for the task in the current thread to complete and then destroy the thread directly. Finally, the plug-in instance information associated with the thread is removed from the information storage unit of the message module.

[0101] 3. If a dormant plug-in instance is found in the thread, the system will first collect all the data cached for the plug-in instance by the data collation unit in the data module.

[0102] 4. Next, the dormant monitoring unit of the monitoring module will forcefully wake up the plug-in instance, and send the collected data to the plug-in instance for processing through the message distribution unit. When the plug-in instance completes this processing task, the system will destroy the corresponding running thread.

[0103] 5. Finally, as in the previous case, the system will remove the plug-in instance information associated with the thread from the information storage unit of the message module.

[0104] If the RAM or CPU usage is too high, the number of stream processing task threads will be automatically reduced: When the system detects that the current RAM or CPU usage is too high, one or more threads will be selected randomly or sequentially to execute the operation of stopping a thread of the stream processing task.

[0105] If the RAM or CPU usage is too low, the number of stream processing task threads will be automatically increased: When the system detects that the current RAM or CPU usage is too low, the number of execution threads of the stream processing task will be increased by performing the load plug-in instance operation.

[0106] Through the above-mentioned collaboration mechanism, even if there is a misalignment in the data generation time, the system can ensure that all types of plug-in instances obtain all the required data to the greatest extent possible to meet business processing requirements. In addition, apart from the above-mentioned situation, if there is data dependency or transmission requirements between plug-in instances, they can be solved in a similar way to give full play to the synergy of various modules of the system.

[0107] Based on the above embodiment, the present invention further provides a method for managing a stateful plug-in for built-in stream processing in a time series database, the method comprising:

[0108] Loading a plug-in instance, the plug-in instance enters a waiting state; the plug-in instance includes an extraction plug-in instance, a processing plug-in instance and a delivery plug-in instance;

[0109] If the plug-in instance reaches the dormancy condition, the plug-in instance enters the dormant state, saves the filtering conditions of the data required by the plug-in instance in the dormant state, and monitors the dormancy time of the plug-in instance;

[0110] If the plug-in instance does not reach the dormancy condition, the plug-in instance enters a ready state, and the plug-in instance transmits required data in the ready state; the required data includes data in a specific time period and data of a specific device;

[0111] If the plug-in instance is released from the dormant state, the plug-in instance enters the ready state, and the plug-in instance transmits the required data in the ready state;

[0112] Filtering required data transmitted between plug-in instances based on the saved filtering conditions, the plug-in instances entering a running state;

[0113] If the delivery target of the delivery plug-in instance is the address of the current database, then after the processing plug-in instance is executed, the extraction plug-in instance, the processing plug-in instance and the delivery plug-in instance are destroyed;

[0114] If the delivery target of the delivery plug-in instance is other addresses, after the delivery plug-in instance is executed, the extraction plug-in instance, the processing plug-in instance and the delivery plug-in instance are destroyed.

[0115] It can be understood that the method and system for managing a stateful plug-in with built-in stream processing in a time series database of the present invention are the same concept of the present invention, and their corresponding features can be referenced to each other, and the present invention will not elaborate on them one by one.

[0116] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

[0117] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0118] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A time series database built-in stream processing stateful plug-in management system, characterized in that: The system includes: a message module, a data module, a management module and a monitoring module; The message module is responsible for maintaining the life cycle status information of the plug-in instance and coordinating the communication between other modules; the plug-in instance includes extracting plug-in instances, processing plug-in instances and delivering plug-in instances; the life cycle status information includes waiting state, dormant state, ready state and running state; The data module is responsible for receiving, storing and collating the required data transmitted between plug-in instances; the required data includes data of a specific time period and data of a specific device; The management module is responsible for loading the plug-in instance and placing the loaded plug-in instance in a waiting state, saving the filtering conditions of the data required by the plug-in instance in the dormant state, providing the filtering conditions of the data required by the plug-in instance in the ready state, and removing the filtering conditions of the data required by the plug-in instance in the running state; The monitoring module is responsible for monitoring the usage of server resources and processing the sleep duration of the plug-in instance in the sleep state, and dynamically adjusting the system resources according to the monitoring results; and converting the plug-in instance from the sleep state to the ready state.

2. According to claim 1, a time series database built-in stream processing stateful plug-in management system is characterized in that: The message module includes: an information storage unit and a message distribution unit; The information storage unit is responsible for storing the life cycle status information of the plug-in instance and providing query services for the life cycle status information of the plug-in instance for other modules; The message distribution unit is responsible for receiving requests from other modules and forwarding the requests to the corresponding plug-in instances.

3. According to claim 1, a time series database built-in stream processing stateful plug-in management system is characterized in that: The data module includes: a data receiving unit, a data sorting unit and a data recovery unit; The data receiving unit is responsible for receiving the required data shared by the plug-in instance; The data sorting unit is responsible for monitoring the data receiving unit, receiving the required data of all plug-in instances in a dormant state, classifying and storing the required data according to the plug-in instance type, and sending the classified and stored plug-in instances and the related information of the required data to the message module or the data recovery unit; The data recovery unit is responsible for processing the plug-in instances that are not sent to the message module and the related information of the required data. For the data in a specific time period, it is sent to an external third-party system; for the data of a specific device, it is directly discarded.

4. According to claim 1, a time series database built-in stream processing stateful plug-in management system is characterized in that: The management module includes: an instance loading unit and a filtering condition unit; The instance loading unit is responsible for loading the plug-in instance according to the user configuration information, and after loading is completed, registering the life cycle status information of the plug-in instance to the message module; The filtering condition unit is responsible for saving the filtering conditions of the data required by the plug-in instance in the dormant state, providing the filtering conditions of the data required by the plug-in instance in the ready state, and removing the filtering conditions of the data required by the plug-in instance in the running state; the filtering conditions include filtering conditions for a specific time period and filtering conditions for a specific device.

5. According to claim 1, a time series database built-in stream processing stateful plug-in management system is characterized in that: The monitoring module includes: a sleep monitoring unit and a system monitoring unit; The sleep monitoring unit is responsible for monitoring the sleep duration of the plug-in instance in the sleep state. If it is found that the sleep duration of a plug-in instance in the sleep state exceeds the maximum limit, the plug-in instance will be forcibly awakened; after awakening, the message module is notified to update the life cycle status information of the plug-in instance, and the required data filtered by the plug-in instance in the sleep state is obtained from the data module, and the filtering conditions of the plug-in instance in the running state are removed from the management module; The system monitoring unit is responsible for monitoring the average utilization rate of the system. If it is found that the average utilization rate exceeds a preset range, the number of plug-in instances running the stream processing task will be dynamically adjusted.

Images