Safety protection capability evaluation method based on Flink streaming processing technology

Abstract

The invention relates to the field of electric digital data processing, in particular to a safety protection capability evaluation method based on an Flink streaming processing technology. The invention discloses a safety protection capability evaluation method based on an Flink streaming processing technology. The method comprises the following steps: S1, collecting IoT device data; s2, predicting the operation condition of the equipment; s3, analyzing a scene, and evaluating a clock synchronization demand; s4, monitoring and adjusting the load of the NTP server; s5, evaluating the global time synchronization efficiency; and S6, judging the safety protection level. According to the invention, for the problem of time asynchronization caused by sending a large amount of data by the IoT equipment in the mountainous area under a special event, through regional response and tracking of an internal transmission path, the problem of time asynchronization caused by the load of an NTP server when the Flink streaming processing technology is applied to process real-time streaming data is solved; the invention provides a novel safety protection capability evaluation method.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a safety protection capability evaluation method based on Flink streaming processing technology. Background Art

[0002] Apache Flink is a high-performance stream processing and batch processing framework. Its advantages lie in its low-latency, high-throughput data processing capabilities, as well as its excellent support for event time processing and state management, making it suitable for large-scale data stream applications. In the data transmission of IoT devices in remote mountainous areas, Flink effectively ensures the timeliness and reliability of data with its low latency characteristics and real-time data processing capabilities under unstable connection conditions, demonstrating its important value. Mountain fire monitoring is highly dependent on IoT devices. Due to the limitations of mountain power facilities, these devices often use solar energy, wind energy or low-power designs to extend their battery life. In the face of fire-prone seasons or fire scenes, low-power IoT devices need to transmit data intensively. At this time, the NTP server they rely on will cause time synchronization errors due to a surge in concurrent requests, and time asynchrony will seriously affect the analysis and judgment of the current fire situation and the accuracy of response strategies. Summary of the invention

[0003] In order to overcome the disadvantage of time asynchrony of IoT devices in remote mountainous areas, the present invention provides a security protection capability evaluation method based on Flink streaming processing technology.

[0004] The technical implementation scheme of the present invention is: a security protection capability evaluation method based on Flink streaming processing technology, comprising the following steps:

[0005] S1: Collect IoT device data;

[0006] S2: predict equipment operating status;

[0007] S3: Analyze the scenario and evaluate the clock synchronization requirements;

[0008] S4: Monitor and adjust NTP server load;

[0009] S5: Evaluate global time synchronization performance;

[0010] S6: Determine the safety protection level.

[0011] Preferably, the IoT device data is collected, including: ensuring the accuracy of IoT device data collection by building a decision tree model; the specific method is: extracting N sub-datasets from the original data set using the Bootstrap method, each sub-dataset is used to train a decision tree; in each decision tree construction process, randomly selecting less than N / 2 features from all features as candidates; for each sub-dataset, recursively building a decision tree; at each node, based on the candidate features after feature selection, selecting the best split point to split the data; the decision tree grows until a preset depth, number of nodes or purity threshold is reached; repeating the above process, building multiple decision trees, each tree is trained independently, and finally forming a forest; for new data points, let each tree in the forest make predictions separately, and then decide the final output according to the majority voting principle; wherein, information entropy and Gini impurity are used to measure the degree of confusion of the data set and help select the best splitting attribute; information entropy formula:

[0012]

[0013] , where p i is the proportion of category i in data set D; Gini impurity formula:

[0014] .

[0016] Preferably, the predicting of the equipment operation status includes: using Flink's DataStream API to clean the data according to the collected data to remove invalid and abnormal data; using Flink's stateful transformations to maintain and update the status information; sending the preprocessed data to the Transformer model for training; deploying the trained model in the Flink environment for real-time prediction; using the trained model to perform real-time prediction on new data to identify changes or abnormalities in the equipment operation status; feeding back the prediction results to the equipment management system; monitoring and predicting equipment energy consumption, and predicting the equipment operation status according to the equipment energy consumption; in the feature engineering stage, the formula used is, mean:

[0017]

[0018] ,variance:

[0019]

[0020] , autocorrelation:

[0021]

[0022] , in the Transformer model, the key formula of the self-attention mechanism is:

[0023]

[0024] , where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0025] Preferably, the analysis scenario and evaluation of clock synchronization requirements include: analyzing the device application scenarios according to the device operation status, and classifying the application scenarios according to the time synchronization requirements, specifically, the first category, N IoT devices need to perform time synchronization at the same time, and the second category, N IoT devices perform time synchronization at off-peak times or less than N / 2 IoT devices need to perform time synchronization at the same time; according to the application scenario classification results, combined with the IoT device data transmission mode, determine the processing method of the Flink streaming processing framework for the real-time data stream generated by the IoT device; specifically, according to the IoT device energy-saving mode, determine the intermittent data transmission mode of the IoT device, use the LSTM model to predict data disorder, and evaluate the effect of Flink's Watermark mechanism on event time processing based on the prediction results; based on the processing effect, determine the processing method of the Flink streaming processing framework for the real-time data stream generated by the IoT device; based on the processing method, evaluate the clock synchronization requirements.

[0026] Among them, the data transmission mode of IoT devices is divided into sleep state and wake-up state; the time synchronization situation under simple conditions, that is, ideal power consumption and signal conditions, is recorded to the time synchronization situation under complex conditions, that is, the actual power consumption and signal conditions of IoT devices in mountainous areas, and the path from simple conditions to complex conditions is tracked in real time. By using the adjusted power consumption and signal conditions, the time asynchrony problem in the wake-up state is gradually approached, the trend of time asynchrony is observed, and the data of time asynchrony is found; the wake-up state devices in mountainous areas are marked; the specific marking method is: (when it is monitored that the device changes from sleep state to wake-up state, the wake-up state device is marked as a wake-up device; the location of the wake-up device in the mountainous area is obtained; the wake-up device area is marked as the wake-up area; if the wake-up area changes back to sleep state, the wake-up device is reset to a sleep device, and the area where the sleep device is located is the sleep area;) the wake-up area of ​​the wake-up state device is set as the first wake-up area; the interval area of ​​the first wake-up area is set as the second wake-up area; the interval area of ​​the second wake-up area is set as the third wake-up area; when there is the next or next batch of devices After being awakened, obtain the devices in the interval area around the next or next batch of awakened devices, and mark the response time of the awakening area based on the location information of the devices in the interval area; divide the area according to the response time of the awakening area devices and the marked response time of the awakening area, and plan the area path according to the area division result; (obtain the location information of the awakening device and the sleep device in the area, record the response time of the awakening device, sort according to the response time, and determine the first path according to the sorting result; divide the number of awakening devices, sort according to the division result, and determine the second path according to the sorting result; analyze according to the first path and the second path, and determine the subsequent paths of the first path and the second path, that is, the subsequent path of the first path and the subsequent path of the second path according to the analysis result; obtain the third path according to the subsequent path of the first path and the subsequent path of the second path); according to the area planning path, actively wake up the area on the planned path, that is, adjust the devices in the area on the planned path from the sleep state to the awakened state; adjust the data transmission mode based on the awakened state devices combined with the marked awakened state devices and the actively awakened state devices;

[0027] Among them, the real-time path also includes: data trajectory initialization, using Flink's state preservation mechanism, the device sensor data in remote mountainous areas is initially filtered and preprocessed by the edge gateway and then forwarded to the Flink cluster; at this stage, the "first tracking time" is recorded as the latest processing timestamp of the data, and the "second tracking time" is introduced to store detailed metadata including the location, size, modification time and displacement information of the data flow; in addition, the "third tracking time" is added to mark each key jump time and location information of the data packet from the source to the processing node to form a network trajectory diagram of the data packet; continuous processing and trajectory update, the Flink cluster uses the first tracking time and the second tracking time to maintain the continuity of data processing, and updates the third tracking time every time the data packet passes through an important network node or processing stage to dynamically depict the data flow path; fault recovery and trajectory restoration, in the face of processing interruption, Flink uses the checkpoint recovery mechanism, and by reading the second tracking time and the third tracking time information, accurately locates the state and data packet trajectory before the interruption, ensuring that the processing after recovery is both continuous and accurately reflects the actual transmission path; adaptive task generation and trajectory connection, based on the displacement information in the second tracking time and the path record of the third tracking time, Fl ink ink automatically generates new processing tasks and seamlessly continues the interruption points to ensure that there are no omissions or duplications in the processing process, and the path tracking is continuous and seamless; stream processing execution and path monitoring, finally, Flink executes new tasks until the end of the current cycle, while continuously monitoring and recording the "third tracking time" information, providing a comprehensive real-time path view for data transmission of IoT devices in remote areas, assisting in analyzing network conditions and optimizing data flows.

[0028] Preferably, the method of determining the processing method of the real-time data stream generated by the IoT device by the Flink stream processing framework according to the processing effect includes: if the processing effect has no effect on the data disorder of the data stream: use Flink's DataStreamAPI to read the real-time data stream from the IoT device and connect it to Kafka through Flink; ensure that each event contains a timestamp, and Flink determines the order of events based on the timestamp; define windows according to requirements, and use event time to allocate windows; use a watermark mechanism to process delayed data; the processed data is connected to the database through S ink for further analysis or real-time display; if the processing effect has an impact on the data disorder of the data stream: all of the above steps: first, follow the processing steps of the data stream that will not cause data disorder; configure Flink's maximum tolerance disorder, that is, how long it is allowed to receive late data; use Side Output to process the timeout late data; design exception handling logic, including a retry mechanism and an alarm system; after processing all data, use Flink's Checkpointing and Savepoint functions to ensure state consistency and recovery capabilities; set up monitoring and logging to track the processing process of the data stream.

[0029] Preferably, the monitoring and adjusting the NTP server load includes: obtaining NTP monitoring data according to clock synchronization requirements, obtaining NTP load changes according to the NTP monitoring data, and determining the NTP server load adjustment method according to the NTP load changes in combination with the IoT device processing NTP request time and the time to adjust the local clock; wherein the IoT device processing NTP request time is set as the first request time, and the IoT device adjusts the local clock time is set as the first adjustment time; based on the first request time and the first adjustment time, the NTP server load adjustment method is determined using a fuzzy reasoning model, and the data reception and analysis processing order is divided according to the degree of influence of the IoT device.

[0030] Preferably, the method of determining the NTP server load adjustment method using a fuzzy inference model according to the first request time and the first adjustment time includes: collecting performance indicators of the NTP server and data of IoT devices requesting NTP services; based on the collected data, selecting features that have a significant impact on the NTP server load as input variables of the fuzzy inference system; defining fuzzy sets for each input variable, including "low", "medium" and "high" fuzzy sets, and using membership functions to quantify the membership of the input variable values ​​to these fuzzy sets; wherein: for the average response time (T avg ), three fuzzy sets are defined: “low”, “medium”, and “high”, and the corresponding membership functions are triangular functions:

[0031] "Low":

[0032] ,"middle":

[0033] ,"high":

[0034] , where t1, t2, and t3 are preset thresholds; define a series of fuzzy rules, including "if CPU utilization is high and memory utilization is high, then the NTP server load state is high", which are used to describe the relationship between input variables and output variables (NTP server load state); use fuzzy reasoning algorithm to determine the load state of the NTP server based on the current input variable values; convert the results of fuzzy reasoning into specific numerical values, that is, the actual load level of the NTP server; use historical data to train the neural network model to predict the NTP server load under given input features; help improve the accuracy of the fuzzy reasoning model; dynamically adjust the configuration of the NTP server based on the results of fuzzy reasoning and neural network models; continuously monitor system performance and regularly retrain the neural network model to adapt to changes in the IoT environment.

[0035] Preferably, the method of dividing the order of receiving and analyzing data according to the degree of influence of IoT devices includes: dividing the order of receiving data acquired by IoT devices according to the time synchronization of IoT devices combined with the degree of influence of data acquired by IoT devices on events; the specific method is: evaluating the time synchronization accuracy of each IoT device and the influence of the collected data on the current event; constructing a priority scoring system based on these two indicators, the device with greater data impact and more accurate time synchronization has a higher priority; and determining the order of data reception and processing based on this priority order.

[0036] Preferably, the evaluation of global time synchronization performance includes: analyzing the time synchronization performance of IoT devices using an LSTM model to obtain the phased global event synchronization performance; the specific method is: collecting timestamp data and synchronization status information of all IoT devices over a period of time to construct a time series input; using an LSTM model to learn long-term dependencies and periodic patterns in these time series; through training, the LSTM model can predict and evaluate the global event synchronization performance at different stages.

[0037] Preferably, the determination of the security protection level includes: based on the time synchronization performance evaluation result, using a neural network model to classify the event impact degree, and evaluating the security protection capability according to the classification result; the specific method is: collecting and inputting various parameters of time synchronization performance into a pre-trained neural network model; the model analyzes these parameters and learns the correlation between them and the impact degree of past events; the model will output a classification result, that is, classifying the event impact into different severity levels; based on the level of event impact, another set of evaluation mechanisms or algorithms are used to quantify the effectiveness of current security protection measures; if the event impact level is high, the system will require stricter security protection standards; otherwise, a relatively loose protection level will be adopted.

[0038] Beneficial effects: The present invention uses a decision tree model to improve the accuracy of IoT device data collection, and uses Flink stream processing technology to efficiently process these data, and then uses the Transformer model to predict the operating status of IoT devices. Based on the device status analysis, we subdivide the application scenarios into several categories, and each scenario corresponds to a specific time synchronization requirement, thereby customizing Flink's stream processing strategy. At the same time, the IoT devices are divided into regional paths, and regional path tracking and internal path real-time tracking are performed based on regional responses. In response to the time asynchrony phenomenon that occurs during intensive data transmission, the system actively monitors the NTP server load and adjusts it in time to ensure time synchronization. In addition, according to the importance of the IoT device's impact on the event, we set a priority sorting mechanism to process the data to prevent time deviations from causing serious consequences. By integrating multiple algorithms, the present invention significantly improves the speed and accuracy of security protection effectiveness evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention is a flow chart of a method for evaluating security protection capability based on Flink streaming technology. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1: A security protection capability evaluation method based on Flink streaming technology, such as Figure 1 As shown, the following steps are included:

[0042] S1: Collect IoT device data;

[0043] S2: predict equipment operating status;

[0044] S3: Analyze the scenario and evaluate the clock synchronization requirements;

[0045] S4: Monitor and adjust NTP server load;

[0046] S5: Evaluate global time synchronization performance;

[0047] S6: Determine the safety protection level.

[0048] Collecting IoT device data, including: ensuring the accuracy of collecting IoT device data by building a decision tree model; the specific method is: from the original data set, using the Bootstrap method to extract N sub-data sets, each sub-data set is used to train a decision tree; in each decision tree construction process, randomly select less than N / 2 features from all features as candidates; for each sub-data set, recursively build a decision tree; at each node, based on the candidate features after feature selection, select the best split point to split the data; the decision tree grows until it reaches a preset depth, number of nodes or purity threshold; repeat the above process to build multiple decision trees, each tree is trained independently, and finally a forest is formed; for new data points, let each tree in the forest make predictions separately, and then decide the final output according to the majority voting principle; among them, use information entropy and Gini impurity to measure the degree of confusion of the data set and help select the best split attribute; information entropy formula:

[0049]

[0050] , where p i is the proportion of category i in data set D; Gini impurity formula:

[0051] .

[0053] In the process of building each decision tree, less than N / 2 features are randomly selected from all features as candidates to further reduce the correlation between features and enhance the diversity of the model; by building a decision tree, the accuracy of data collected each time is ensured.

[0054] Predicting the equipment operation status includes: using Flink's DataStream API to clean the data based on the collected data and remove invalid and abnormal data; using Flink's stateful transforms to maintain and update status information; sending the preprocessed data to the Transformer model for training; deploying the trained model in the Flink environment for real-time prediction; using the trained model to make real-time predictions on new data to identify changes or anomalies in the equipment operation status; feeding the prediction results back to the equipment management system; monitoring and predicting equipment energy consumption, and predicting equipment operation status based on equipment energy consumption; in the feature engineering stage, the formula used is, mean:

[0055]

[0056] ,variance:

[0057]

[0058] , autocorrelation:

[0059]

[0060] , in the Transformer model, the key formula of the self-attention mechanism is:

[0061]

[0062] , where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0063] Analyze the scenarios and evaluate the clock synchronization requirements, including: analyzing the device application scenarios according to the device operation status, and classifying the application scenarios according to the time synchronization requirements. Specifically, the first category is that N IoT devices need to be time synchronized at the same time, and the second category is that N IoT devices are time synchronized at staggered times or less than N / 2 IoT devices need to be time synchronized at the same time; according to the application scenario classification results, combined with the IoT device data transmission mode, determine the Flink streaming processing framework's processing method for the real-time data stream generated by the IoT device; specifically, according to the IoT device energy-saving mode, determine the IoT device's intermittent data transmission mode, use the LSTM model to predict data disorder, and based on the prediction results, evaluate the Flink's Watermark mechanism to solve event time processing effects; based on the processing effect, determine the Flink streaming processing framework's processing method for the real-time data stream generated by the IoT device; based on the processing method, evaluate the clock synchronization requirements.

[0064] Among them, the data transmission mode of IoT devices is divided into sleep state and wake-up state; the time synchronization situation under simple conditions, that is, ideal power consumption and signal conditions, is recorded to the time synchronization situation under complex conditions, that is, the actual power consumption and signal conditions of IoT devices in mountainous areas, and the path from simple conditions to complex conditions is tracked in real time. By using the adjusted power consumption and signal conditions, the time asynchrony problem in the wake-up state is gradually approached, the trend of time asynchrony is observed, and the data of time asynchrony is found; the wake-up state devices in mountainous areas are marked; the specific marking method is: (when it is monitored that the device changes from sleep state to wake-up state, the wake-up state device is marked as a wake-up device; the location of the wake-up device in the mountainous area is obtained; the wake-up device area is marked as the wake-up area; if the wake-up area changes back to sleep state, the wake-up device is reset to a sleep device, and the area where the sleep device is located is the sleep area;) the wake-up area of ​​the wake-up state device is set as the first wake-up area; the interval area between the first wake-up area is set as the second wake-up area; the interval area between the second wake-up area is set as the third wake-up area; when the next or the next batch of devices are awakened, obtain The devices in the interval area around the next or next batch of awakened devices mark the response time of the awakening area based on the location information of the devices in the interval area; divide the area according to the response time of the awakening area devices and the marked response time of the awakening area, and plan the area path according to the area division result; the area path planning method is specifically as follows: (obtain the location information of the regional awakening device and the location information of the sleeping device, record the response time of the awakening device, sort according to the response time, and determine the first path according to the sorting result; divide the number of awakening devices, sort according to the division result, and determine the second path according to the sorting result; analyze according to the first path and the second path, and determine the subsequent paths of the first path and the second path, that is, the subsequent path of the first path and the subsequent path of the second path according to the analysis result; obtain the third path according to the subsequent path of the first path and the subsequent path of the second path); according to the regional planned path, actively wake up the area on the planned path, that is, adjust the devices in the area on the planned path from the sleeping state to the awakening state; adjust the data transmission mode based on the awakened state devices in combination with the marked awakening state devices and the actively awakening state devices.

[0065] Among them, the real-time path also includes: data trajectory initialization, using Flink's state preservation mechanism, the device sensor data in remote mountainous areas is initially filtered and preprocessed by the edge gateway and then forwarded to the Flink cluster; at this stage, the "first tracking time" is recorded as the latest processing timestamp of the data, and the "second tracking time" is introduced to store detailed metadata including the location, size, modification time and displacement information of the data flow; in addition, the "third tracking time" is added to mark each key jump time and location information of the data packet from the source to the processing node to form a network trajectory diagram of the data packet; continuous processing and trajectory update, the Flink cluster uses the "first tracking time" and "second tracking time" to maintain the continuity of data processing, and updates the "third tracking time" every time the data packet passes through an important network node or processing stage to dynamically depict the data flow path; fault recovery and trajectory restoration, in the face of processing interruption, Flink uses the checkpoint recovery mechanism, and by reading the second tracking time and third tracking time information, accurately locates the state and data packet trajectory before the interruption, ensuring that the processing after recovery is both continuous and accurately reflects the actual transmission path; adaptive task generation and trajectory connection, based on the displacement information in the second tracking time and the path record of the third tracking time, Fl ink ink automatically generates new processing tasks and seamlessly continues the interruption points to ensure that the processing process is neither missed nor repeated, and the path tracking is continuous and seamless; stream processing execution and path monitoring, finally, Fl ink executes new tasks until the end of the current cycle, while continuously monitoring and recording third tracking time information, providing a comprehensive real-time path view for data transmission of IoT devices in remote areas, assisting in analyzing network conditions and optimizing data flows.

[0066] Among them, the sleep state data is divided into recent data and long-term data, which is specifically used to compare the timestamp information sent by the sleep state data and the wake-up state data; the sleep state is specifically the state of the IoT device sending data on a daily basis, and the wake-up state is specifically the state of the IoT device sending data frequently under special events; the simple situation is specifically the sleep state of the IoT device, and the complex situation is specifically the wake-up state of the IoT device.

[0067] According to the processing effect, determine how the Flink stream processing framework processes the real-time data stream generated by the IoT device, including: If the processing effect has no effect on the data disorder of the data stream: Use Flink's DataStream API to read the real-time data stream from the IoT device and connect it to Kafka through Flink; Ensure that each event contains a timestamp, and Flink determines the order of events based on this timestamp; Define windows according to requirements and use event time for window allocation; Use the watermark mechanism to process delayed data; The processed data is connected to the database through Sink for further analysis or real-time display; If the processing effect affects the data disorder of the data stream: All of the above steps: First, follow the processing steps of the data stream that will not cause data disorder; By configuring Flink's maximum tolerance disorder, that is, how long it is allowed to receive late data; For timeout late data, use Side Output to handle it; Design exception handling logic, including retry mechanism and alarm system; After processing all data, use Flink's Checkpoint ing and Savepoint functions to ensure state consistency and recovery capability; Set up monitoring and logging to track the processing process of the data stream.

[0068] Among them, Apache Flink is an open source framework for processing unbounded and bounded data streams. It provides powerful stream processing and batch processing capabilities. When processing real-time data streams generated by IoT devices, Flink can effectively handle data disorder problems, which are common in many scenarios, such as network delays and system clock deviations. For timeout late data, Side Output is used to process it. These late data will be collected and processed separately to avoid interfering with the normal data stream processing flow. Windows are defined according to requirements, and event time is used for window allocation, so that even if the data arrives at inconsistent times, the data can be processed in the order of event occurrence. The watermark mechanism is used to process delayed data. The watermark helps the system identify whether the data is late and decide when to close the window, thereby ensuring the accuracy of the processing results. Design exception handling logic, including retry mechanism and alarm system, so that when a large amount of data is detected to be disordered or late, the system can respond and take measures in time. Set up monitoring and logging to track the processing process of the data stream, and once the data disorder problem is found, it can be quickly located and solved.

[0069] Monitoring and adjusting the NTP server load includes: obtaining NTP monitoring data according to clock synchronization requirements, obtaining NTP load changes according to the NTP monitoring data, and determining the NTP server load adjustment method according to the NTP load changes in combination with the time when the IoT device processes the NTP request and the time when the local clock is adjusted; wherein the time when the IoT device processes the NTP request is set as the first request time, and the time when the IoT device adjusts the local clock is set as the first adjustment time; according to the first request time and the first adjustment time, the NTP server load adjustment method is determined using a fuzzy reasoning model, and the data reception and analysis processing order is divided according to the degree of influence of the IoT device.

[0070] According to the first request time and the first adjustment time, a fuzzy inference model is used to determine the NTP server load adjustment method, including: collecting performance indicators of the NTP server and data of IoT devices requesting NTP services; based on the collected data, selecting features that have a significant impact on the NTP server load as input variables of the fuzzy inference system; defining fuzzy sets for each input variable, including "low", "medium" and "high" fuzzy sets, and using membership functions to quantify the membership of the input variable values ​​to these fuzzy sets; wherein: for the average response time (T avg ), three fuzzy sets are defined: “low”, “medium”, and “high”, and the corresponding membership functions are triangular functions:

[0071] "Low":

[0072] ,"middle":

[0073] ,"high":

[0074] , where t1, t2, and t3 are preset thresholds; define a series of fuzzy rules, including "if CPU utilization is high and memory utilization is high, then the NTP server load state is high", which are used to describe the relationship between input variables and output variables (NTP server load state); use fuzzy reasoning algorithm to determine the load state of the NTP server based on the current input variable values; convert the results of fuzzy reasoning into specific numerical values, that is, the actual load level of the NTP server; use historical data to train the neural network model to predict the NTP server load under given input features; help improve the accuracy of the fuzzy reasoning model; dynamically adjust the configuration of the NTP server based on the results of fuzzy reasoning and neural network models; continuously monitor system performance and regularly retrain the neural network model to adapt to changes in the IoT environment.

[0075] Among them, the performance indicators of the NTP server and the data of IoT devices requesting NTP services are collected; this includes but is not limited to: the CPU utilization, memory utilization, network bandwidth usage, request frequency and response time of the NTP server, the number and geographical distribution of IoT devices; based on the collected data, features that have a significant impact on the NTP server load are selected as input variables of the fuzzy inference system; these features include: average response time (T avg ), CPU utilization (U cpu ), memory utilization (U mem ), network bandwidth utilization (B net ), device request frequency (F req ); Based on the results of fuzzy reasoning and neural network models, the configuration of NTP servers is dynamically adjusted, including increasing or decreasing server resources, distributing load to multiple NTP servers, or optimizing parameters of NTP services; In IoT environments, ensuring time synchronization between devices is critical because many IoT applications rely on accurate timestamps; The NTP protocol is widely used to synchronize clocks in distributed networks; However, with the growth of the number of IoT devices, NTP servers face excessive loads, resulting in increased response delays and uncertainty in time synchronization; To solve this problem, a fuzzy reasoning model combined with machine learning is used to dynamically adjust the load of NTP servers.

[0076] The order of receiving, analyzing and processing data is divided according to the degree of influence of IoT devices, including: dividing the order of receiving data acquired by IoT devices according to the time synchronization of IoT devices and the degree of influence of data acquired by IoT devices on events; the specific method is: evaluating the time synchronization accuracy of each IoT device and the influence of the collected data on the current event; building a priority scoring system based on these two indicators, the device with greater data impact and more accurate time synchronization has a higher priority; the order of data reception and processing is determined according to this priority order.

[0077] The order of data reception and processing is determined according to this priority order, ensuring that critical and time-sensitive information is processed first.

[0078] Evaluate the global time synchronization performance, including: using the LSTM model to analyze the time synchronization performance of IoT devices to obtain the phased global event synchronization performance; the specific method is: collect the timestamp data and synchronization status information of all IoT devices in a period of time, and construct a time series input; use the LSTM model to learn the long-term dependencies and periodic patterns in these time series; through training, the LSTM model can predict and evaluate the global event synchronization performance at different stages.

[0079] Among them, the LSTM model is used to learn the long-term dependencies and periodic patterns in these time series. The model can capture the changing trends and potential anomalies of time synchronization performance. Through training, the LSTM model can predict and evaluate the global event synchronization performance at different stages, providing data support for optimizing the time synchronization strategy of the IoT network.

[0080] Determining the security protection level includes: according to the time synchronization performance evaluation result, using the neural network model to classify the event impact degree, and evaluating the security protection capability according to the classification result; the specific method is: collecting and inputting various parameters of time synchronization performance into a pre-trained neural network model; the model analyzes these parameters and learns the correlation between them and the impact degree of past events; the model will output a classification result, that is, classifying the event impact into different severity levels; according to the level of event impact, another set of evaluation mechanisms or algorithms are used to quantify the effectiveness of current security protection measures; if the event impact level is high, the system will require stricter security protection standards; otherwise, a relatively loose protection level will be adopted.

[0081] Among other things, this process ensures that security protection measures match the actual risks faced and achieves the rational allocation of protection resources.

[0082] The above is a detailed introduction to the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A security protection capability evaluation method based on Flink streaming technology, characterized in that: The following steps are involved: S1: Collect IoT device data; S2: predict equipment operating status; S3: Analyze the scenario and evaluate the clock synchronization requirements; S4: Monitor and adjust NTP server load; S5: Evaluate global time synchronization performance; S6: Determine the safety protection level.

2. A security protection capability evaluation method based on Flink streaming technology as claimed in claim 1, characterized in that: The IoT device data collection includes: ensuring the accuracy of IoT device data collection by building a decision tree model; the specific method is: extracting N sub-datasets from the original data set using the Bootstrap method, each sub-dataset is used to train a decision tree; in each decision tree construction process, randomly selecting less than N / 2 features from all features as candidates; for each sub-dataset, recursively building a decision tree; at each node, based on the candidate features after feature selection, selecting the best split point to split the data; the decision tree grows until a preset depth, number of nodes or purity threshold is reached; repeating the above process, building multiple decision trees, each tree is trained independently, and finally forming a forest; for new data points, let each tree in the forest make predictions separately, and then determine the final output according to the majority voting principle; wherein, information entropy and Gini impurity are used to measure the degree of disorder of the data set and help select the best split attribute; information entropy formula: ， Among them, p i is the proportion of category i in data set D; Gini impurity formula:

3. The security protection capability evaluation method based on Flink stream processing technology according to claim 1 is characterized in that: The predicting of the equipment operation status includes: using Flink's DataStream API to clean the data according to the collected data to remove invalid and abnormal data; using Flink's stateful transformations to maintain and update the status information; sending the preprocessed data to the Transformer model for training; deploying the trained model in the Flink environment for real-time prediction; using the trained model to perform real-time prediction on new data to identify changes or abnormalities in the equipment operation status; feeding back the prediction results to the equipment management system; monitoring and predicting the equipment energy consumption, and predicting the equipment operation status according to the equipment energy consumption; in the feature engineering stage, the formula used is, mean: ， variance: ， Autocorrelation: ， In the Transformer model, the key formula of the self-attention mechanism is: ， Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

4. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 1, characterized in that: The analysis scenario and evaluation of clock synchronization requirements include: analyzing the device application scenarios according to the device operation status, and classifying the application scenarios according to the time synchronization requirements, specifically, the first category, N IoT devices need to perform time synchronization at the same time, and the second category, N IoT devices perform time synchronization at staggered times or less than N / 2 IoT devices need to perform time synchronization at the same time; according to the application scenario classification results, combined with the IoT device data transmission mode, determine the Flink streaming processing framework's processing method for the real-time data stream generated by the IoT device; specifically, according to the IoT device energy-saving mode, determine the IoT device's intermittent data transmission mode, use the LSTM model to predict data disorder, and evaluate the Flink Watermark mechanism based on the prediction results to solve the event time processing effect; based on the processing effect, determine the Flink streaming processing framework's processing method for the real-time data stream generated by the IoT device; based on the processing method, evaluate the clock synchronization requirements; Among them, the intermittent data transmission mode is divided into a sleep state and a wake-up state; the time synchronization situation under simple conditions, that is, ideal power consumption and signal conditions, is recorded to the time synchronization situation under complex conditions, that is, the actual power consumption and signal conditions of IoT devices in mountainous areas, and the path from simple conditions to complex conditions is tracked in real time. By using the adjusted power consumption and signal conditions, the time asynchrony problem in the wake-up state is gradually approached, the trend of time asynchrony is observed, and the data of time asynchrony is found; the wake-up state devices in mountainous areas are marked; the specific marking method is: (when it is monitored that the device changes from a sleep state to a wake-up state, the wake-up state device is marked as a wake-up device; the location of the wake-up device in the mountainous area is obtained; the wake-up device area is marked as a wake-up area; if the wake-up area changes back to a sleep state, the wake-up device is reset to a sleep device, and the area where the sleep device is located is the sleep area;) the wake-up area of ​​the wake-up state device is set as the first wake-up area; the interval area of ​​the first wake-up area is set as the second wake-up area; the interval area of ​​the second wake-up area is set as the third wake-up area; when the next or next batch of devices are awakened, the next Devices in the interval area around one or the next batch of awakened devices, based on the location information of the devices in the interval area, mark the response time of the awakening area; divide the area according to the response time of the awakening area devices and the marked response time of the awakening area, and plan the area path according to the area division result; the area path planning method is specifically as follows: (obtain the location information of the awakening device and the sleep device in the area, record the response time of the awakening device, sort according to the response time, and determine the first path according to the sorting result; divide the number of awakening devices, sort according to the division result, and determine the second path according to the sorting result; analyze according to the first path and the second path, and determine the subsequent paths of the first path and the second path, that is, the subsequent path of the first path and the subsequent path of the second path according to the analysis result; obtain the third path according to the subsequent path of the first path and the subsequent path of the second path); according to the area planning path, actively wake up the area on the planned path, that is, adjust the devices in the area on the planned path from the sleep state to the awakened state; adjust the data transmission mode based on the awakened state devices combined with the marked awakened state devices and the actively awakened state devices; The real-time path also includes: data trajectory initialization, using Flink's state preservation mechanism, the device sensor data in remote mountainous areas is initially filtered and preprocessed by the edge gateway and then forwarded to the Flink cluster; at this stage, the "first tracking time" is recorded as the latest processing timestamp of the data, and the "second tracking time" is introduced to store detailed metadata including data flow location, size, modification time and displacement information; in addition, the "third tracking time" is added to mark each key jump time and location information of the data packet from the source to the processing node to form a network trajectory diagram of the data packet; continuous processing and trajectory update, the Flink cluster uses the first tracking time and the second tracking time to maintain the continuity of data processing, and updates the third tracking time every time the data packet passes through an important network node or processing stage to dynamically depict the data flow Path; Fault recovery and trajectory restoration. In the face of processing interruption, Flink uses the checkpoint recovery mechanism and reads the second tracking time and third tracking time information to accurately locate the state and data packet trajectory before the interruption, ensuring that the processing after recovery is both continuous and accurately reflects the actual transmission path; Adaptive task generation and trajectory connection. Based on the displacement information in the second tracking time and the path record in the third tracking time, Flink automatically generates new processing tasks and seamlessly connects to the interruption point to ensure that the processing process is neither missed nor repeated, and the path tracking is continuous and without faults; Stream processing execution and path monitoring. Finally, Flink executes the new task until the end of the current cycle, while continuously monitoring and recording the third tracking time information, providing a comprehensive real-time path view for data transmission of IoT devices in remote areas, assisting in analyzing network conditions and optimizing data flows.

5. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 4, characterized in that: According to the processing effect, the processing method of the Flink stream processing framework for the real-time data stream generated by the IoT device is determined, including: if the processing effect has no effect on the data disorder of the data stream: use Flink's DataStream API to read the real-time data stream from the IoT device and connect it to Kafka through Flink; ensure that each event contains a timestamp, and Flink determines the order of events based on this timestamp; define windows according to requirements, and use event time to allocate windows; use the watermark mechanism to process delayed data; the processed data is connected to the database through Sink for further analysis or real-time display; if the processing effect has an impact on the data disorder of the data stream: all of the above steps: first, follow the processing steps of the data stream that will not cause data disorder; configure Flink's maximum tolerance disorder, that is, how long it is allowed to receive late data; use Side Output to process the timeout late data; design exception handling logic, including retry mechanism and alarm system; after processing all data, use Flink's Checkpoint ing and Savepoint functions to ensure state consistency and recovery capability; set up monitoring and logging to track the processing process of the data stream.

6. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 1, characterized in that: The monitoring and adjusting of the NTP server load includes: obtaining NTP monitoring data according to clock synchronization requirements, obtaining NTP load changes according to the NTP monitoring data, and determining the NTP server load adjustment method according to the NTP load changes in combination with the time when the IoT device processes the NTP request and the time when the local clock is adjusted; wherein the time when the IoT device processes the NTP request is set as the first request time, and the time when the IoT device adjusts the local clock is set as the first adjustment time; according to the first request time and the first adjustment time, the NTP server load adjustment method is determined using a fuzzy reasoning model, and the data reception and analysis processing order is divided according to the impact degree of the IoT device.

7. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 6, characterized in that: The method of determining the NTP server load adjustment method by using a fuzzy inference model according to the first request time and the first adjustment time includes: collecting performance indicators of the NTP server and data of IoT devices requesting NTP services; based on the collected data, selecting features that have a significant impact on the NTP server load as input variables of the fuzzy inference system; defining fuzzy sets for each input variable, including "low", "medium", and "high" fuzzy sets, and using membership functions to quantify the membership of the input variable values ​​to these fuzzy sets; wherein: for the average response time (T avg ), three fuzzy sets are defined: "Low", "Medium", "High", the corresponding membership function is the triangular function: "Low": , "middle": , "high": , Among them, t1, t2, and t3 are preset thresholds; define a series of fuzzy rules, including "if CPU utilization is high and memory utilization is high, the NTP server load state is high", which are used to describe the relationship between input variables and output variables (NTP server load state); use fuzzy reasoning algorithm to determine the load state of the NTP server based on the current input variable values; convert the results of fuzzy reasoning into specific numerical values, that is, the actual load level of the NTP server; use historical data to train the neural network model to predict the NTP server load under given input features; help improve the accuracy of the fuzzy reasoning model; dynamically adjust the configuration of the NTP server based on the results of fuzzy reasoning and neural network models; continuously monitor system performance and regularly retrain the neural network model to adapt to changes in the IoT environment.

8. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 7, characterized in that: The order of receiving and analyzing data according to the impact of IoT devices includes: The order in which IoT devices receive data is divided according to the time synchronization of IoT devices and the degree of influence of the data obtained by IoT devices on events. The specific method is: evaluate the time synchronization accuracy of each IoT device and the influence of the collected data on the current event; build a priority scoring system based on these two indicators. The device with greater data impact and more accurate time synchronization has a higher priority; the order of data reception and processing is determined based on this priority order.

9. The method for evaluating security protection capability based on Flink stream processing technology according to claim 1 is characterized in that: The evaluation of global time synchronization performance includes: analyzing the time synchronization performance of IoT devices using an LSTM model to obtain phased global event synchronization performance; the specific method is: collecting timestamp data and synchronization status information of all IoT devices in a period of time to construct a time series input; using an LSTM model to learn long-term dependencies and periodic patterns in these time series; through training, the LSTM model can predict and evaluate the global event synchronization performance at different stages.

10. A security protection capability evaluation method based on Flink stream processing technology as claimed in claim 1, characterized in that: The method of determining the security protection level includes: according to the time synchronization performance evaluation result, using a neural network model to classify the event impact degree, and evaluating the security protection capability according to the classification result; the specific method is: collecting and inputting various parameters of time synchronization performance into a pre-trained neural network model; the model analyzes these parameters and learns the correlation between them and the impact degree of past events; the model will output a classification result, that is, classifying the event impact into different severity levels; according to the classification of the event impact level, another set of evaluation mechanisms or algorithms are used to quantify the effectiveness of current security protection measures; if the event impact level is high, the system will require stricter security protection standards; otherwise, a relatively loose protection level will be adopted.

Images