Data management and control method and system for monitoring system

Abstract

The invention discloses a data management and control method and system for a monitoring system. The method comprises the following steps: performing parallel processing on a plurality of data combinations based on a parallel processing mode; in the parallel processing process of the multiple data combinations, elimination of the to-be-eliminated data is triggered based on the cache data and the cache space of the multiple data combinations, primary optimization of the multiple data combinations is achieved, and data redundancy in the data combinations is avoided; furthermore, based on a data set output by the stream processing engine and a corresponding data change diagram matched by the decision engine, a data decision result is defined according to the data change diagram, a data processing mode of the stream processing engine and decision parameters of the decision engine, and dynamic early warning is triggered according to the data decision result. The data processing mode of the data change graph and the stream processing engine and the decision-making parameters of the decision-making engine are compatible, the accuracy of the data decision-making result and the corresponding dynamic early warning are ensured, the data management and control effect of the monitoring system is improved, and single-dimensional data management and control are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of stream processing engines, and in particular to a data control method and system for a monitoring system. Background Art

[0002] With the development of science and technology, monitoring systems are gradually applied to people's lives. The monitoring system interacts with multiple data devices and introduces multiple real-time data during the interaction process. In the existing technology, the amount of real-time data is sometimes large and mixed with some data that need to be eliminated. The existing technology processes the real-time data as a whole and is compatible with the processing of data that needs to be eliminated, which affects the data processing effect. As a result, the existing technology controls data along a single dimension and cannot guarantee the accuracy of data decision results. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides a data management and control method and system for a monitoring system. The monitoring system is traversed, and a stream processing engine of the monitoring system receives real-time data; multiple data combinations are formed based on the data allocation of the stream processing engine to the real-time data; a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and multiple data combinations are processed in parallel based on the parallel processing mode; in the parallel processing process of multiple data combinations, the elimination of data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations, so as to introduce the data to be eliminated, be compatible with the parallel processing of multiple data combinations, realize the primary optimization of multiple data combinations, and avoid data redundancy in the data combinations;

[0004] Furthermore, a cache space is constructed according to the data volume of the cache data and the memory space of the monitoring system; multiple cache sub-data are formed based on the division of the cache space into multiple cache sub-data; the cache space is dynamically balanced based on the multiple cache sub-data and the buffer processing method; based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine matching, the data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, which is compatible with the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and realizes multi-dimensional control of the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, thereby ensuring the accuracy of the data decision results and the corresponding dynamic warnings, improving the data management and control effect of the monitoring system, and avoiding single-dimensional data management.

[0005] The embodiment of the present invention provides a data control method for a monitoring system, which is applied to a data control scenario of a monitoring system;

[0006] The data control method of the monitoring system includes:

[0007] Traverse the monitoring system and monitor the stream processing engine of the monitoring system to receive real-time data;

[0008] Forming multiple data combinations based on data distribution of real-time data by the stream processing engine;

[0009] Selecting a parallel processing mode according to multiple data combinations and a stream processing engine, and performing parallel processing on the multiple data combinations based on the parallel processing mode;

[0010] In the parallel processing of multiple data combinations, the elimination of data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations;

[0011] Build a cache space based on the amount of cache data and the memory space of the monitoring system; divide the cache data based on the cache space to form multiple cache sub-data; dynamically balance the cache space based on multiple cache sub-data and buffer processing methods;

[0012] Match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision result according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger dynamic warning according to the data decision result.

[0013] Optionally, the traversal monitoring system, wherein the stream processing engine of the monitoring system receives real-time data, includes:

[0014] Collect the location of the monitoring system and trigger the positioning detection of the monitoring system according to the location of the monitoring system;

[0015] Based on the positioning detection of the monitoring system, multiple data devices that interact with the monitoring system are defined, and data of the monitoring system and multiple data devices are controlled;

[0016] The monitoring system and multiple data devices are fixed, and corresponding interaction parameters are defined based on the dynamic interaction between the monitoring system and multiple data devices. At the same time, the monitoring system and multiple data devices are functionally matched, and corresponding matching parameters are defined;

[0017] Associating interaction parameters and matching parameters, and matching the corresponding stream processing engine according to the interaction parameters, matching parameters and working modules of the monitoring system;

[0018] Trigger the online work of the stream processing engine, and the stream processing engine of the monitoring system receives real-time data.

[0019] Optionally, the data allocation of the real-time data by the stream processing engine to form multiple data combinations includes:

[0020] Freeze real-time data;

[0021] Defining corresponding incomplete data based on the integrity detection of real-time data;

[0022] Trigger the data integrity logic of the stream processing engine based on incomplete data and the nodes where the incomplete data is located;

[0023] Define data tracing and supplementation based on the data integrity logic of the stream processing engine, real-time data, and nodes where incomplete data is located, and ensure the integrity of real-time data;

[0024] Defining multiple allocation intervals based on data allocation of real-time data by a stream processing engine;

[0025] A plurality of data combinations are formed according to the matching of the plurality of allocation intervals and the real-time data.

[0026] Optionally, defining a parallel processing mode according to the multiple data combinations and the stream processing engine, and performing parallel processing on the multiple data combinations based on the parallel processing mode, includes:

[0027] Freeze multiple data combinations;

[0028] Associating multiple data sets and stream processing engines;

[0029] Based on multiple data combinations and stream processing engines, a parallel coefficient of each data combination is defined;

[0030] Construct parallel processing efficiency based on the parallel coefficient of each data combination and the function type corresponding to the data combination;

[0031] Define the corresponding parallel processing mode based on the parallel processing efficiency and the load level of the stream processing engine;

[0032] Based on the parallel processing mode, multiple data combinations are processed in parallel. At this time, multiple data combinations are processed synchronously, and the data processing progress of multiple data combinations is controlled based on the stream processing engine.

[0033] Optionally, in the parallel processing of the multiple data combinations, triggering the elimination of the data to be eliminated based on the cache data and cache space of the multiple data combinations includes:

[0034] In the parallel processing of multiple data combinations, multiple data combinations are screened;

[0035] Defining cache data of multiple data combinations based on screening of multiple data combinations;

[0036] The cache data of the plurality of data combinations are allocated to corresponding cache spaces, and at this time, the cache spaces have a plurality of cache spaces;

[0037] Filter the trigger data scenarios for multiple cache spaces and define the corresponding eliminated data;

[0038] Collect multiple elimination data, and build corresponding elimination data aggregation according to the multiple elimination data;

[0039] Based on the aggregation of eliminated data and the matching of the types of eliminated data, the elimination of the data to be eliminated is triggered according to the data to be eliminated and the corresponding cache space, so as to perform primary optimization on the buffered data.

[0040] Optionally, the step of constructing a cache space according to the amount of cache data and the memory space of the monitoring system; forming a plurality of cache sub-data based on the division of the cache data in the cache space; and dynamically balancing the cache space based on the plurality of cache sub-data and the buffer processing mode includes:

[0041] Freeze cached data and mark the amount of cached data;

[0042] The amount of associated cached data and the memory space of the monitoring system;

[0043] Define the corresponding space matching coefficient according to the amount of cached data and the memory space of the monitoring system;

[0044] The corresponding cache space is constructed based on the space matching coefficient, the amount of cached data and the working module of the monitoring system.

[0045] Optionally, the step of constructing a cache space according to the amount of cache data and the memory space of the monitoring system; forming a plurality of cache sub-data based on the division of the cache data in the cache space; and dynamically balancing the cache space based on the plurality of cache sub-data and the buffer processing mode further includes:

[0046] Forming multiple cache sub-data based on the division of cache data by cache space and dynamic interaction of multiple cache data;

[0047] Associating multiple cache sub-data, the load level of the cache space, and the buffer processing method;

[0048] Dynamically balance the cache space based on multiple cache sub-data, cache space load level, and buffer processing method.

[0049] Optionally, the data set outputted by the stream processing engine and the data change graph corresponding to the match of the decision engine are defined based on the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered based on the data decision result, including:

[0050] Based on the monitoring system fixed-frame stream processing engine and decision engine;

[0051] Associated stream processing engine and decision engine;

[0052] Based on the data set output by the stream processing engine and the corresponding data change graph matched by the decision engine.

[0053] Optionally, the data set outputted by the stream processing engine and the data change graph corresponding to the decision engine are matched, a data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, further comprising:

[0054] Define data decision results based on the data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine;

[0055] Correlate data decision results and result warning coefficients;

[0056] Dynamic warnings are triggered based on data decision results and result warning coefficients to dynamically optimize various data of the monitoring system based on the stream processing engine and decision engine.

[0057] In addition, an embodiment of the present invention further provides a data control system for a monitoring system, the data control system for the monitoring system comprising:

[0058] The traversal module is used to traverse the monitoring system and monitor the stream processing engine of the monitoring system to receive real-time data;

[0059] A data combination module, used to form multiple data combinations based on data allocation of real-time data by the stream processing engine;

[0060] A parallel module is used to select a parallel processing mode according to multiple data combinations and a stream processing engine, and perform parallel processing on multiple data combinations based on the parallel processing mode;

[0061] An elimination module is used to trigger elimination of data to be eliminated based on cache data and cache space of multiple data combinations during the parallel processing of multiple data combinations;

[0062] A buffer module is used to construct a buffer space according to the amount of buffer data and the memory space of the monitoring system; to form multiple buffer sub-data based on the division of buffer data in the buffer space; and to dynamically balance the buffer space based on multiple buffer sub-data and buffer processing methods;

[0063] The decision module is used to match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision results according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger dynamic warnings based on the data decision results.

[0064] In an embodiment of the present invention, through the method in the embodiment of the present invention, the monitoring system is traversed, and the stream processing engine of the monitoring system receives real-time data; multiple data combinations are formed based on the data allocation of the stream processing engine to the real-time data; a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and the multiple data combinations are processed in parallel based on the parallel processing mode; in the parallel processing process of the multiple data combinations, the elimination of the data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations, so as to introduce the data to be eliminated, be compatible with the parallel processing of the multiple data combinations, realize the primary optimization of the multiple data combinations, and avoid data redundancy in the data combinations;

[0065] Furthermore, a cache space is constructed according to the data volume of the cache data and the memory space of the monitoring system; multiple cache sub-data are formed based on the division of the cache space into multiple cache sub-data; the cache space is dynamically balanced based on the multiple cache sub-data and the buffer processing method; based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine matching, the data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, which is compatible with the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and realizes multi-dimensional control of the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, thereby ensuring the accuracy of the data decision results and the corresponding dynamic warnings, improving the data management and control effect of the monitoring system, and avoiding single-dimensional data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 is a flow chart of a data control method of a monitoring system in an embodiment of the present invention;

[0068] Figure 2 is a flow chart of S11 in the data control method of the monitoring system in an embodiment of the present invention;

[0069] Figure 3 is a flow chart of S12 in the data control method of the monitoring system in the embodiment of the present invention;

[0070] Figure 4 is a flow chart of S13 in the data control method of the monitoring system in the embodiment of the present invention;

[0071] Figure 5 is a flow chart of S14 in the data control method of the monitoring system in the embodiment of the present invention;

[0072] Figure 6 is a flow chart of S15 in the data control method of the monitoring system in the embodiment of the present invention;

[0073] Figure 7 is a flow chart of S16 in the data control method of the monitoring system in the embodiment of the present invention;

[0074] Figure 8 Schematic diagram of the structure of the data control system of the monitoring system in the embodiment of the present invention;

[0075] Fig. 9 The figure is a hardware diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] See also Figures 1 to 9 , a data control method for a monitoring system, applied to a data control scenario of a monitoring system; the data control method for a monitoring system includes:

[0078] Step S11: traverse the monitoring system to monitor the stream processing engine of the monitoring system to receive real-time data;

[0079] Step S12: forming a plurality of data combinations based on data allocation of the real-time data by the stream processing engine;

[0080] Step S13: selecting a parallel processing mode according to the multiple data combinations and the stream processing engine, and performing parallel processing on the multiple data combinations based on the parallel processing mode;

[0081] Step S14: in the parallel processing of the multiple data combinations, triggering elimination of the data to be eliminated based on the cache data and cache space of the multiple data combinations;

[0082] Step S15: constructing a cache space according to the amount of cache data and the memory space of the monitoring system; forming a plurality of cache sub-data based on the division of the cache data in the cache space; dynamically balancing the cache space based on the plurality of cache sub-data and the buffer processing mode;

[0083] Step S16: Match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision result according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger a dynamic warning based on the data decision result.

[0084] In an embodiment of the present invention, through the method in the embodiment of the present invention, the monitoring system is traversed, and the stream processing engine of the monitoring system receives real-time data; multiple data combinations are formed based on the data allocation of the stream processing engine to the real-time data; a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and the multiple data combinations are processed in parallel based on the parallel processing mode; in the parallel processing process of the multiple data combinations, the elimination of the data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations, so as to introduce the data to be eliminated, be compatible with the parallel processing of the multiple data combinations, realize the primary optimization of the multiple data combinations, and avoid data redundancy in the data combinations;

[0085] Furthermore, a cache space is constructed according to the data volume of the cache data and the memory space of the monitoring system; multiple cache sub-data are formed based on the division of the cache space into multiple cache sub-data; the cache space is dynamically balanced based on the multiple cache sub-data and the buffer processing method; based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine matching, the data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, which is compatible with the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and realizes multi-dimensional control of the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, thereby ensuring the accuracy of the data decision results and the corresponding dynamic warnings, improving the data management and control effect of the monitoring system, and avoiding single-dimensional data management.

[0086] refer to Figure 2 ,In step S11, the monitoring system is traversed, and the stream processing engine of the monitoring system receives real-time data;

[0087] In the specific implementation process of the present invention, the specific steps may be:

[0088] S111: collecting the location of the monitoring system, and triggering positioning detection of the monitoring system according to the location of the monitoring system;

[0089] S112: defining multiple data devices that interact with the monitoring system based on the positioning detection of the monitoring system, and performing data management and control on the monitoring system and the multiple data devices;

[0090] S113: freeze the monitoring system and the multiple data devices, define corresponding interaction parameters based on the dynamic interaction between the monitoring system and the multiple data devices, and at the same time, perform functional matching on the monitoring system and the multiple data devices, and define corresponding matching parameters;

[0091] S114: Associating the interaction parameters and the matching parameters, and matching the corresponding stream processing engine according to the interaction parameters, the matching parameters and the working module of the monitoring system;

[0092] S115: triggering the online work of the stream processing engine, and the stream processing engine of the monitoring system receives real-time data.

[0093] In an embodiment of the present application, the location of the monitoring system is collected, and the positioning detection of the monitoring system is triggered according to the location of the monitoring system. The monitoring system is further managed and controlled, and multiple data devices that interact with the monitoring system are defined based on the positioning detection of the monitoring system. Data of the monitoring system and the multiple data devices are managed and controlled, thereby realizing online management and control of the monitoring system and the multiple data devices to facilitate further processing of the data.

[0094] At this time, the monitoring system and multiple data devices are frozen, and corresponding interaction parameters are defined based on the dynamic interaction of the monitoring system and multiple data devices. At the same time, functional matching is performed on the monitoring system and multiple data devices, and corresponding matching parameters are defined to facilitate matching of the monitoring system and multiple data devices. The matching degree between the monitoring system and multiple data devices is thereby ensured by matching the parameters, thereby ensuring further interaction between the monitoring system and multiple data devices.

[0095] Therefore, the interaction parameters and matching parameters are associated, and the corresponding stream processing engine is matched according to the interaction parameters, matching parameters and the working modules of the monitoring system. The interaction parameters, matching parameters and the working modules of the monitoring system are introduced, and multi-dimensional control is performed on the interaction parameters, matching parameters and the working modules of the monitoring system to facilitate matching the corresponding stream processing engine, thereby realizing further management and control of the stream processing engine. At the same time, the online work of the stream processing engine is triggered, and the stream processing engine of the monitoring system receives real-time data, so that the stream processing engine can further process the real-time data.

[0096] refer to Figure 3 , in step S12, a plurality of data combinations are formed based on data allocation of the real-time data by the stream processing engine;

[0097] In the specific implementation process of the present invention, the specific steps may be:

[0098] S121: freeze real-time data;

[0099] S122: defining corresponding incomplete data based on integrity detection of real-time data;

[0100] S123: triggering the data integrity logic of the stream processing engine according to the incomplete data and the node where the incomplete data is located;

[0101] S124: Define data tracing and supplementation according to the data integrity logic of the stream processing engine, the real-time data, and the nodes where the incomplete data is located, and ensure the integrity of the real-time data;

[0102] S125: defining a plurality of allocation intervals based on data allocation of the real-time data by the stream processing engine;

[0103] S126: forming a plurality of data combinations according to the matching of the plurality of allocation intervals and the real-time data.

[0104] In an embodiment of the present application, real-time data is frozen and introduced, which is not intermittent data. At this time, corresponding incomplete data is defined based on the integrity check of the real-time data, which triggers the integrity check of the real-time data. The integrity check is performed on the real-time data, and the completeness and incompleteness of the real-time data are defined by the content of the real-time data, the data sequence number or the readable logic of the data, thereby realizing the investigation of incomplete data.

[0105] At this time, the data integrity logic of the stream processing engine is triggered according to the incomplete data and the nodes where the incomplete data is located, so as to facilitate overall management of the incomplete data and the nodes where the incomplete data is located, thereby triggering the data integrity logic of the stream processing engine, and further managing the data integrity logic of the stream processing engine.

[0106] Therefore, according to the data integrity logic of the stream processing engine, the node definitions of real-time data and incomplete data, the data tracing and supplementation are introduced. Based on the data integrity logic of the stream processing engine, the node definitions of real-time data and incomplete data, multi-dimensional management and control are carried out to achieve data tracing and supplement, so as to ensure the integrity of real-time data, and then ensure the subsequent processing of real-time data.

[0107] Furthermore, multiple allocation intervals are defined based on the data allocation of real-time data by the stream processing engine, and data allocation is performed on the real-time data to facilitate reasonable data management of the real-time data, thereby introducing multiple allocation spaces, and managing the multiple allocation spaces to form multiple data combinations based on the matching of multiple allocation intervals and real-time data, thereby realizing the output of multiple data combinations and ensuring further management of multiple data combinations.

[0108] refer to Figure 4, in step S13, a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and the multiple data combinations are processed in parallel based on the parallel processing mode;

[0109] In the specific implementation process of the present invention, the specific steps may be:

[0110] S131: freeze multiple data combinations;

[0111] S132: Associating multiple data combinations and a stream processing engine;

[0112] S133: defining a parallel coefficient of each data combination according to the multiple data combinations and the stream processing engine;

[0113] S134: constructing parallel processing efficiency based on the parallel coefficient of each data combination and the function type corresponding to the data combination;

[0114] S135: defining a corresponding parallel processing mode based on the parallel processing efficiency and the load level of the stream processing engine;

[0115] S136: Perform parallel processing on multiple data combinations based on the parallel processing mode. At this time, multiple data combinations are processed synchronously, and the data processing progress of the multiple data combinations is managed and controlled based on the stream processing engine.

[0116] In an embodiment of the present application, multiple data combinations are frozen, multiple data combinations and stream processing engines are introduced, thereby associating multiple data combinations and stream processing engines, and further controlling multiple data combinations and stream processing engines, so as to define the parallel coefficient of each data combination according to the multiple data combinations and stream processing engines, and perform multi-dimensional control on multiple data combinations and stream processing engines to ensure the accuracy of the parallel coefficient of each data combination, and present the degree of parallelism of each data combination through the parallel coefficient of each data combination.

[0117] Furthermore, parallel processing efficiency is constructed based on the parallel coefficient of each data combination and the functional type corresponding to the data combination, so as to introduce parallel processing efficiency, and then the corresponding parallel processing mode is defined based on the parallel processing efficiency and the load degree of the stream processing engine, thereby realizing multi-dimensional control of the parallel processing efficiency and the load degree of the stream processing engine, and ensuring the accuracy of the parallel processing mode.

[0118] Therefore, multiple data combinations are processed in parallel based on the parallel processing mode. At this time, multiple data combinations are processed synchronously, and the data processing progress of multiple data combinations is controlled based on the stream processing engine, thereby realizing progress control of multiple data combinations and ensuring the synchronization of data processing of multiple data combinations.

[0119] refer to Figure 5, S14: in the parallel processing of multiple data combinations, triggering elimination of the data to be eliminated based on the cache data and cache space of the multiple data combinations;

[0120] In the specific implementation process of the present invention, the specific steps may be:

[0121] S141: In the parallel processing process of multiple data combinations, screening multiple data combinations;

[0122] S142: defining cache data of multiple data combinations based on screening of multiple data combinations;

[0123] S143: Allocating cache data of the plurality of data combinations to corresponding cache spaces, where there are a plurality of cache spaces;

[0124] S144: screening the data scenarios triggered by the multiple cache spaces, and defining the corresponding eliminated data;

[0125] S145: Collect multiple elimination data, and construct corresponding elimination data aggregation according to the multiple elimination data;

[0126] S146: Based on the aggregation of eliminated data and the matching of the types of the preset eliminated data, the elimination of the data to be eliminated is triggered according to the data to be eliminated and the corresponding cache space, so as to perform primary optimization on the buffered data.

[0127] In an embodiment of the present application, a monitoring system is traversed, and a stream processing engine of the monitoring system receives real-time data; multiple data combinations are formed based on data allocation of the stream processing engine to the real-time data; a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and the multiple data combinations are processed in parallel based on the parallel processing mode; in the parallel processing of the multiple data combinations, the elimination of the data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations, so as to introduce the data to be eliminated, make the parallel processing of the multiple data combinations compatible, realize the primary optimization of the multiple data combinations, and avoid data redundancy in the data combinations.

[0128] At this time, in the parallel processing of multiple data combinations, the multiple data combinations are screened, so as to perform a preliminary screening on the multiple data combinations, and then the cache data of the multiple data combinations are defined based on the screening of the multiple data combinations, and the cache data of the multiple data combinations are introduced to facilitate further control of the cache data of the multiple data combinations and ensure the subsequent processing of the cache data of the multiple data combinations.

[0129] Furthermore, cache data according to multiple data combinations are allocated to corresponding cache spaces. At this time, there are multiple cache spaces, thereby triggering the screening of data scenarios for multiple cache spaces, and defining corresponding elimination data, performing corresponding screening on multiple cache spaces, and introducing corresponding elimination data, thereby achieving optimization of data in multiple cache spaces, so as to facilitate further management and control of eliminated data.

[0130] Therefore, multiple eliminated data are collected, and corresponding eliminated data aggregation is constructed based on the multiple eliminated data; based on the eliminated data aggregation and the matching of the types of eliminated data, the elimination of the data to be eliminated is triggered according to the data to be eliminated and the corresponding cache space, so as to perform primary optimization on the buffered data, thereby ensuring further regulation of the buffered data and further completing the optimization of the data.

[0131] refer to Figure 6 , S15: constructing a cache space according to the amount of cache data and the memory space of the monitoring system; forming a plurality of cache sub-data based on the division of the cache data in the cache space; dynamically balancing the cache space based on the plurality of cache sub-data and the buffer processing mode;

[0132] In the specific implementation process of the present invention, the specific steps may be:

[0133] S151: freeze the cached data and mark the amount of the cached data;

[0134] S152: Associating the amount of cached data with the memory space of the monitoring system;

[0135] S153: defining a corresponding space matching coefficient according to the amount of cached data and the memory space of the monitoring system;

[0136] S154: constructing a corresponding cache space based on the space matching coefficient, the amount of cached data, and the working module of the monitoring system;

[0137] S155: forming a plurality of cache sub-data based on the division of the cache data by the cache space and the dynamic interaction of the plurality of cache data;

[0138] S156: Associating multiple cache sub-data, the load level of the cache space, and the buffer processing method; dynamically balancing the cache space based on the multiple cache sub-data, the load level of the cache space, and the buffer processing method.

[0139] In an embodiment of the present application, cached data is frozen and the data volume of the cached data is marked, the data volume of the cached data is introduced, and the data volume of the cached data is further controlled. At the same time, the data volume of the cached data and the memory space of the monitoring system are associated, so that the corresponding spatial matching coefficient is defined according to the data volume of the cached data and the memory space of the monitoring system, and multi-dimensional control is performed through the data volume of the cached data and the memory space of the monitoring system, so that the corresponding matching degree can be presented through the spatial matching coefficient.

[0140] At this time, the corresponding cache space is constructed based on the spatial matching coefficient, the data volume of the cached data, and the working module of the monitoring system. The spatial matching coefficient, the data volume of the cached data, and the working module of the monitoring system are introduced, and multi-dimensional control is performed on the spatial matching coefficient, the data volume of the cached data, and the working module of the monitoring system, thereby ensuring the rationality and accuracy of the cache space, and being compatible with the overall consideration of the spatial matching coefficient, the data volume of the cached data, and the working module of the monitoring system.

[0141] Therefore, based on the division of cache data by cache space and the dynamic interaction of multiple buffer data, multiple cache sub-data are formed, thereby introducing the division of cache data by cache space and the dynamic interaction of multiple buffer data. The division of cache data by cache space and the dynamic interaction of multiple buffer data are synchronously managed and controlled, thereby realizing multi-dimensional processing of cache data.

[0142] Furthermore, multiple cache sub-data, the load levels of the cache space, and the buffer processing methods are associated; the cache space is dynamically balanced based on the multiple cache sub-data, the load levels of the cache space, and the buffer processing methods, and the multiple cache sub-data, the load levels of the cache space, and the buffer processing methods are considered as a whole, and the load levels of the cache space and the buffer processing methods are fully considered, so as to reasonably interact with the load levels of the cache space and the buffer processing methods, and then the multiple cache sub-data are managed and controlled in multiple dimensions, thereby ensuring the dynamic balance of the cache space in the working state.

[0143] refer to Figure 7 , S16: Based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine, define the data decision result according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger a dynamic warning according to the data decision result;

[0144] In the specific implementation process of the present invention, the specific steps may be:

[0145] S161: based on the monitoring system fixed-frame stream processing engine and decision engine;

[0146] S162: Associating a stream processing engine and a decision engine;

[0147] S163: Based on the data set output by the stream processing engine and the data change graph corresponding to the matching by the decision engine;

[0148] S164: defining a data decision result according to the data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine;

[0149] S165: correlation data decision results and result warning coefficients;

[0150] S166: triggering a dynamic warning according to the data decision result and the result warning coefficient, so as to dynamically optimize various data of the monitoring system based on the stream processing engine and the decision engine.

[0151] In the specific implementation process of the present invention, a cache space is constructed according to the data volume of the cache data and the memory space of the monitoring system; a plurality of cache sub-data are formed based on the division of the cache space into multiple cache sub-data; the cache space is dynamically balanced based on the multiple cache sub-data and the buffer processing method; based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine matching, the data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, which is compatible with the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and realizes multi-dimensional control of the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, thereby ensuring the accuracy of the data decision results and the corresponding dynamic warnings, improving the data management and control effect of the monitoring system, and avoiding single-dimensional data management.

[0152] At this time, based on the monitoring system, the stream processing engine and the decision engine are introduced to facilitate the association of the stream processing engine and the decision engine, so as to further control the stream processing engine and the decision engine.

[0153] Furthermore, the corresponding data change graph is matched based on the data set output by the stream processing engine and the decision engine to facilitate the management of the data set output by the stream processing engine and the decision engine. The corresponding data change graph is output based on the management of the data set, and the decision factors of the decision engine are fully considered to ensure the accuracy of the data change graph.

[0154] Therefore, the data decision results are defined according to the data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine. The data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine are introduced. Multi-dimensional control is performed on the data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine to ensure the accuracy of the data decision results and be compatible with the overall consideration of the data processing mode of the stream processing engine and the decision parameters of the decision engine.

[0155] Furthermore, the data decision results and the result warning coefficients are associated; dynamic warnings are triggered according to the data decision results and the result warning coefficients, which ensures the accuracy of the data decision results and the corresponding dynamic warnings, improves the data management and control effect of the monitoring system, avoids single-dimensional data management, and at the same time, dynamically optimizes various data of the monitoring system based on the stream processing engine and the decision engine.

[0156] Specific:

[0157] With Flink as the core stream processing engine, these frameworks have excellent performance and scalability, and can cope with the challenges of real-time processing of large-scale data.

[0158] In terms of configuration, the resource allocation of the stream processing engine (such as memory, number of CPU cores, etc.) is fine-tuned according to the expected load and data processing requirements of the system to achieve the best performance. At the same time, for different types of data processing tasks (such as data filtering, aggregation, association, etc.), appropriate operators and processing strategies are configured to ensure that data can be accurately processed in a very short time. The following optimizations are made to the stream processing engine.

[0159] 1. Adjust stream processing engine resources based on expected load and data processing requirements:

[0160] Monitor system load: Use performance monitoring tools to monitor system CPU usage, memory usage and other indicators in real time.

[0161] Analyze data volume and processing speed: Estimate the required resources based on the input data volume and processing time requirements of the data acquisition module.

[0162] Adjust memory allocation: If the data volume is large and the processing is complex, increase the memory allocation to improve the data cache capacity. You can gradually increase the memory, observe the changes in system performance, and find the best configuration.

[0163] Adjust the number of CPU cores: When there are many concurrent processing tasks, increasing the number of CPU cores can increase the processing speed. You can determine the optimal configuration by testing the processing time under different numbers of cores.

[0164] 2. Configure operators and processing strategies according to different types of data processing tasks:

[0165] Data filtering:

[0166] Operator: Use filter operators, such as conditional filtering (filtering based on the value of a specific field).

[0167] Processing strategy: Determine the filtering conditions and select appropriate fields and values ​​for filtering based on business needs. For example, select only data within a specific time period.

[0168] Data Aggregation:

[0169] Operator: Aggregation operators, such as sum, average, count, etc.

[0170] Processing strategy: Determine the dimensions and indicators for aggregation. For example, aggregate the data by time dimension and calculate the average value per hour.

[0171] Data association:

[0172] Operator: association operator, such as inner join, outer join, etc.

[0173] Processing strategy: Determine the associated fields and the association method. For example, associate data from two different data sources based on user ID.

[0174] The caching mechanism is as follows:

[0175] Cache object determination: Determine which data and intermediate results need to be cached. Frequently used data, such as configuration information, static dictionary data, etc., and intermediate results in the processing process, such as aggregation results, association results, etc., can be cached.

[0176] Cache strategy: Use appropriate cache replacement algorithms, such as the least recently used (LRU) algorithm, the first-in-first-out (FIFO) algorithm, etc. When the cache space is full, select the data to be eliminated according to the cache replacement algorithm.

[0177] Cache management: Set the cache size and validity period. For data with a short validity period, you can regularly clear the expired cache to free up memory space. At the same time, dynamically adjust the cache size according to the system load to ensure the efficiency of cache use.

[0178] The method used for dynamic adjustment is as follows:

[0179] Data feature analysis: Analyze the need for parallel processing based on the characteristics of the data, such as data volume, data distribution, data complexity, etc. For example, for data with large volume and uneven distribution, the degree of parallelism can be increased to improve processing efficiency.

[0180] Processing logic analysis: Consider the complexity and computational effort of the processing logic. If the processing logic is complex and requires more computing resources, you can increase the degree of parallelism. For example, for data processing tasks that require a large number of associated operations, you can increase the degree of parallelism to speed up processing.

[0181] Performance indicator monitoring: Real-time monitoring of system performance indicators, such as processing time, CPU usage, memory usage, etc. If the processing time is too long or the resource utilization is too high, you can increase the degree of parallelism; if the resource utilization is too low, you can reduce the degree of parallelism.

[0182] Dynamic adjustment algorithm: Adopt an adaptive algorithm to automatically adjust the parallelism according to the changes in performance indicators. For example, you can set a threshold. When the performance indicator exceeds the threshold, the parallelism is automatically increased; when the performance indicator is lower than the threshold, the parallelism is automatically reduced.

[0183] The decision engine is the core decision-making component of the real-time monitoring system, which is responsible for defining and executing a series of complex monitoring rules and thresholds.

[0184] In terms of rule definition, using Grafana+Prometheus as a decision engine provides an intuitive and flexible interface that allows users to define various monitoring rules using graphical tools or specific rule description languages. These rules can be set based on a variety of factors such as the data's numerical range, change trend, and combination conditions, and can be dynamically adjusted and updated according to business needs. The monitoring rules are given below as an example:

[0185] Rules based on numeric ranges

[0186] Use Prometheus to collect the CPU usage metrics of the server, create a dashboard in Grafana and set rules to trigger an alert when the CPU usage exceeds 80%. For example, in a server monitoring scenario, you can promptly detect excessive server load so that you can take measures to optimize or expand capacity.

[0187] Prometheus is used to collect the number of database connections, and rules are set in Grafana to issue an alert when the number of database connections exceeds a certain threshold (such as 100). This can help database administrators detect possible performance problems or connection leaks in a timely manner.

[0188] Rules based on changing trends

[0189] Prometheus is used to collect website visit indicators, and the visit trend is analyzed in Grafana. If the visit growth rate is less than 10% for three consecutive days, an early warning is triggered, prompting the marketing department to evaluate promotion activities or optimize the website.

[0190] Use Prometheus to monitor the memory usage of the container. Set a rule in Grafana to check for memory leaks or abnormal application behavior if the container memory usage continues to grow by more than 50% within an hour.

[0191] Rules based on combined conditions

[0192] Combined with the server load indicators and network delay indicators collected by Prometheus, set rules in Grafana to issue an early warning and notify the operation and maintenance team to conduct a comprehensive investigation when the server load is higher than 70% and the network delay exceeds 100ms.

[0193] Prometheus is used to collect the error rate and response time indicators of the application. In Grafana, rules are set to trigger an alert when the error rate exceeds 5% and the response time exceeds 2 seconds, prompting the development team to perform performance optimization and error troubleshooting.

[0194] The setting of thresholds fully considers the characteristics of the business and the distribution of historical data. Through the analysis and statistics of a large amount of historical data, combined with expert experience and machine learning algorithms, a reasonable threshold range is determined to ensure that warnings can be triggered in time when abnormal situations occur.

[0195] In order to ensure the accuracy and effectiveness of the rules, the decision engine also has automatic verification and testing functions. After the new rules or thresholds are modified, playback tests can be performed through simulated data or historical data to verify the correctness and rationality of the rules and avoid false positives and false negatives. The specific verification steps are as follows:

[0196] Select simulated data or historical data. Simulated data is randomly generated data that is simulated based on the data characteristics of the monitored system. For example, if you are monitoring the CPU usage of a server, you can generate a series of random values ​​in different ranges to simulate CPU usage data. Historical data is recent historical data that covers data in different business scenarios and system states, including data during normal operation and data when problems have occurred.

[0197] Run the decision engine. Input the selected simulation data or historical data into the decision engine and let the decision engine process it according to the new rules or modified rules.

[0198] Compare results. For simulated data, set the expected results in advance, and then compare the output of the decision engine with the expected results. For example, if the simulated data is a series of known CPU usage values, and according to the rules, an alert should be triggered when the usage exceeds a certain threshold, then check whether the decision engine triggered the alert at the correct value. For historical data, compare the output of the decision engine with what actually happened at the time. For example, if a server failed during a certain period of time in the historical data, and the monitoring system at the time should have triggered an alert but did not, then in the playback test, check whether the new rule can correctly identify the failure and trigger the alert.

[0199] In an embodiment of the present invention, through the method in the embodiment of the present invention, the monitoring system is traversed, and the stream processing engine of the monitoring system receives real-time data; multiple data combinations are formed based on the data allocation of the stream processing engine to the real-time data; a parallel processing mode is defined according to the multiple data combinations and the stream processing engine, and the multiple data combinations are processed in parallel based on the parallel processing mode; in the parallel processing process of the multiple data combinations, the elimination of the data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations, so as to introduce the data to be eliminated, be compatible with the parallel processing of the multiple data combinations, realize the primary optimization of the multiple data combinations, and avoid data redundancy in the data combinations;

[0200] Furthermore, a cache space is constructed according to the data volume of the cache data and the memory space of the monitoring system; multiple cache sub-data are formed based on the division of the cache space into multiple cache sub-data; the cache space is dynamically balanced based on the multiple cache sub-data and the buffer processing method; based on the data set output by the stream processing engine and the data change graph corresponding to the decision engine matching, the data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, which is compatible with the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and realizes multi-dimensional control of the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, thereby ensuring the accuracy of the data decision results and the corresponding dynamic warnings, improving the data management and control effect of the monitoring system, and avoiding single-dimensional data management.

[0201] See also Figure 8 , Figure 8 It is a schematic diagram of the structural composition of the data control system of the monitoring system in an embodiment of the present invention.

[0202] like Figure 8 As shown, a data control system of a monitoring system, the data control system of the monitoring system comprises:

[0203] A traversal module 21 is used to traverse the monitoring system and monitor the stream processing engine of the monitoring system to receive real-time data;

[0204] A data combination module 22, configured to form a plurality of data combinations based on data allocation of the real-time data by the stream processing engine;

[0205] A parallel module 23, used to select a parallel processing mode according to multiple data combinations and a stream processing engine, and perform parallel processing on the multiple data combinations based on the parallel processing mode;

[0206] An elimination module 24 is used to trigger elimination of data to be eliminated based on cache data and cache space of the multiple data combinations during the parallel processing of the multiple data combinations;

[0207] The buffer module 25 is used to construct a buffer space according to the amount of buffer data and the memory space of the monitoring system; to form a plurality of buffer sub-data based on the partition of the buffer space for the buffer data; and to dynamically balance the buffer space based on the plurality of buffer sub-data and the buffer processing mode;

[0208] The decision module 26 is used to match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision result according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger dynamic warning according to the data decision result.

[0209] See also Fig. 9 , refer to the following Fig. 9 The electronic intelligent device 40 according to this embodiment of the present invention is described. Fig. 9 The electronic intelligent device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0210] like Fig. 9 As shown, the electronic intelligent device 40 is in the form of a general computing intelligent device. The components of the electronic intelligent device 40 may include but are not limited to: at least one processing unit 41, at least one storage unit 42, and a bus 43 connecting different system components (including the storage unit 42 and the processing unit 41).

[0211] The storage unit stores program codes, which can be executed by the processing unit 41, so that the processing unit 41 executes the steps according to various exemplary embodiments of the present invention described in the above “Embodiment Method” section of this specification.

[0212] The storage unit 42 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 421 and / or a cache memory unit 422 , and may further include a read-only memory unit (ROM) 423 .

[0213] The storage unit 42 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0214] Bus 43 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0215] The electronic smart device 40 may also communicate with one or more external smart devices (e.g., keyboards, pointing smart devices, Bluetooth smart devices, etc.), one or more smart devices that enable a user to interact with the electronic smart device 40, and / or any smart device that enables the electronic smart device 40 to communicate with one or more other computing smart devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 44. Furthermore, the electronic smart device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 45. Fig. 9 As shown, the network adapter 45 communicates with other modules of the electronic intelligent device 40 via the bus 43. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with the electronic intelligent device 40, including but not limited to: microcode, intelligent device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup planning systems.

[0216] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing intelligent device (which can be a personal computer, a server, a terminal device, or a network intelligent device, etc.) to execute the method according to the implementation of the present disclosure.

[0217] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. In addition, it stores computer program instructions, and when the computer program instructions are executed by a computer, the computer executes the above method.

[0218] In addition, the data control method and system of the monitoring system provided by the embodiment of the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data control method for a monitoring system, characterized in that: The steps include: Traverse the monitoring system and monitor the stream processing engine of the monitoring system to receive real-time data; Forming multiple data combinations based on data distribution of real-time data by the stream processing engine; Selecting a parallel processing mode according to multiple data combinations and a stream processing engine, and performing parallel processing on the multiple data combinations based on the parallel processing mode; In the parallel processing of multiple data combinations, the elimination of data to be eliminated is triggered based on the cache data and cache space of the multiple data combinations; Build a cache space based on the amount of cached data and the memory space of the monitoring system; The cache data is divided into multiple cache sub-data based on the cache space; Dynamically balance cache space based on multiple cache sub-data and buffer processing methods; Match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision result according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger dynamic warning according to the data decision result.

2. The data control method of the monitoring system according to claim 1, characterized in that: The traversal monitoring system, wherein the stream processing engine of the monitoring system receives real-time data, includes: Collect the location of the monitoring system and trigger the positioning detection of the monitoring system according to the location of the monitoring system; Based on the positioning detection of the monitoring system, multiple data devices that interact with the monitoring system are defined, and data of the monitoring system and multiple data devices are controlled; The monitoring system and multiple data devices are fixed, and corresponding interaction parameters are defined based on the dynamic interaction between the monitoring system and multiple data devices. At the same time, the monitoring system and multiple data devices are functionally matched, and corresponding matching parameters are defined; Associating interaction parameters and matching parameters, and matching the corresponding stream processing engine according to the interaction parameters, matching parameters and working modules of the monitoring system; Trigger the online work of the stream processing engine, and the stream processing engine of the monitoring system receives real-time data.

3. The data control method of the monitoring system according to claim 2, characterized in that: The data allocation of the real-time data by the stream processing engine to form multiple data combinations includes: Freeze real-time data; Defining corresponding incomplete data based on the integrity detection of real-time data; Trigger the data integrity logic of the stream processing engine based on incomplete data and the nodes where the incomplete data is located; Define data tracing and supplementation based on the data integrity logic of the stream processing engine, real-time data, and nodes where incomplete data is located, and ensure the integrity of real-time data; Defining multiple allocation intervals based on data allocation of real-time data by a stream processing engine; A plurality of data combinations are formed according to the matching of the plurality of allocation intervals and the real-time data.

4. The data control method of the monitoring system according to claim 3, characterized in that: Defining a parallel processing mode according to the plurality of data combinations and the stream processing engine, and performing parallel processing on the plurality of data combinations based on the parallel processing mode, includes: Freeze multiple data combinations; Associating multiple data sets and stream processing engines; Based on multiple data combinations and stream processing engines, a parallel coefficient of each data combination is defined; Construct parallel processing efficiency based on the parallel coefficient of each data combination and the function type corresponding to the data combination; Define the corresponding parallel processing mode based on the parallel processing efficiency and the load level of the stream processing engine; Based on the parallel processing mode, multiple data combinations are processed in parallel. At this time, multiple data combinations are processed synchronously, and the data processing progress of multiple data combinations is controlled based on the stream processing engine.

5. The data control method of the monitoring system according to claim 4, characterized in that: In the parallel processing of the multiple data combinations, triggering the elimination of the data to be eliminated based on the cache data and cache space of the multiple data combinations includes: In the parallel processing of multiple data combinations, multiple data combinations are screened; Defining cache data of multiple data combinations based on screening of multiple data combinations; The cache data of the plurality of data combinations are allocated to corresponding cache spaces, and at this time, the cache spaces have a plurality of cache spaces; Filter the trigger data scenarios for multiple cache spaces and define the corresponding eliminated data; Collect multiple elimination data, and build corresponding elimination data aggregation according to the multiple elimination data; Based on the aggregation of eliminated data and the matching of the types of eliminated data, the elimination of the data to be eliminated is triggered according to the data to be eliminated and the corresponding cache space, so as to perform primary optimization on the buffered data.

6. The data control method of the monitoring system according to claim 5, characterized in that: The cache space is constructed according to the amount of cached data and the memory space of the monitoring system; The cache data is divided into multiple cache sub-data based on the cache space; Dynamically balance cache space based on multiple cache sub-data and buffer processing methods, including: Freeze cached data and mark the amount of cached data; The amount of associated cached data and the memory space of the monitoring system; Define the corresponding space matching coefficient according to the amount of cached data and the memory space of the monitoring system; The corresponding cache space is constructed based on the space matching coefficient, the amount of cached data and the working module of the monitoring system.

7. The data control method of the monitoring system according to claim 6, characterized in that: The cache space is constructed according to the amount of cached data and the memory space of the monitoring system; The cache data is divided into multiple cache sub-data based on the cache space; Dynamically balance cache space based on multiple cache sub-data and buffer processing methods, including: Forming multiple cache sub-data based on the division of cache data by cache space and dynamic interaction of multiple cache data; Associating multiple cache sub-data, the load level of the cache space, and the buffer processing method; dynamically balancing the cache space based on the multiple cache sub-data, the load level of the cache space, and the buffer processing method.

8. The data control method of the monitoring system according to claim 7, characterized in that: The data set outputted by the stream processing engine and the data change graph corresponding to the decision engine are matched, a data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, including: Based on the monitoring system fixed-frame stream processing engine and decision engine; Associated stream processing engine and decision engine; Based on the data set output by the stream processing engine and the corresponding data change graph matched by the decision engine.

9. The data control method of the monitoring system according to claim 8, characterized in that: The data set outputted by the stream processing engine and the data change graph corresponding to the decision engine are matched, a data decision result is defined according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and a dynamic warning is triggered according to the data decision result, and further includes: Define data decision results based on the data change graph, the data processing mode of the stream processing engine, and the decision parameters of the decision engine; Correlate data decision results and result warning coefficients; Dynamic warnings are triggered based on data decision results and result warning coefficients to dynamically optimize various data of the monitoring system based on the stream processing engine and decision engine.

10. A data control system for a monitoring system, characterized in that: The data control system of the monitoring system is applied to the data control method of the monitoring system according to any one of claims 1 to 9, and the data control system of the monitoring system includes: The traversal module is used to traverse the monitoring system and monitor the stream processing engine of the monitoring system to receive real-time data; A data combination module, used to form multiple data combinations based on data allocation of real-time data by the stream processing engine; A parallel module is used to select a parallel processing mode according to multiple data combinations and a stream processing engine, and perform parallel processing on multiple data combinations based on the parallel processing mode; An elimination module is used to trigger elimination of data to be eliminated based on cache data and cache space of multiple data combinations during the parallel processing of multiple data combinations; A buffer module is used to construct a buffer space according to the amount of buffer data and the memory space of the monitoring system; to form multiple buffer sub-data based on the division of buffer data in the buffer space; and to dynamically balance the buffer space based on multiple buffer sub-data and buffer processing methods; The decision module is used to match the corresponding data change graph based on the data set output by the stream processing engine and the decision engine, define the data decision results according to the data change graph, the data processing mode of the stream processing engine and the decision parameters of the decision engine, and trigger dynamic warnings based on the data decision results.

Images