Big data network signal safety early warning system

Abstract

The invention relates to the technical field of network security analysis, in particular to a big data network signal security early warning system. The system comprises a network signal big data stream transmission module, a network signal ratio difference detection module, a network signal division result safety early warning module and a network signal abnormity safety response module. Data acquisition equipment can be deployed at an entrance and an exit of each network equipment, and network signal data real-time acquisition and big data stream type transmission conversion can be carried out, so that a network signal big data stream can be obtained; carrying out time sequence flow size and access frequency statistical analysis on the network signal big data flow, and carrying out network signal ratio difference detection division and abnormal safety early warning processing at the same time to generate a network signal abnormal safety early warning signal; and uploading the network signal abnormity safety early warning signal to a cloud platform to carry out abnormity safety response processing so as to generate a network signal abnormity safety response protection strategy. According to the invention, network signal abnormity can be efficiently monitored in real time and early warning can be provided.

Description

Technical Field

[0001] The present invention relates to the technical field of network security analysis, and in particular to a big data network signal security early warning system. Background Art

[0002] In recent years, network security technologies based on big data analysis and machine learning have been gradually proposed. Through real-time monitoring and dynamic analysis of network signal data, abnormal patterns in network traffic can be more flexibly identified and potential security risks can be predicted. By combining technologies such as big data analysis, machine learning and deep learning, various types of signal data in the network can be collected and analyzed in real time, and data flows, network topologies and communication modes can be comprehensively monitored. By building intelligent abnormal behavior models, potential network security threats can be identified, and through real-time early warning mechanisms, administrators can be alerted in the first place to take protective measures in a timely manner. However, traditional network signal security monitoring technologies mostly rely on traditional intrusion detection systems (IDS) and firewalls. These methods mainly rely on static rules or feature-based detection methods to defend against known attack patterns and abnormal behaviors, and often cannot effectively deal with new and complex network signal abnormal behaviors. Abnormal activities of network signals will be hidden in a large number of normal data streams, making it difficult to discover potential abnormal security threats in real time. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a big data network signal security early warning system to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a big data network signal security early warning system includes the following modules:

[0005] The network signal big data streaming transmission module is used to deploy data collection equipment at the entrance and exit of each network device, and use the data collection equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large data set of network signals; perform big data streaming transmission conversion on the large data set of network signals to obtain a large data stream of network signals;

[0006] The network signal ratio difference detection module is used to perform a statistical analysis of the timing flow size and access frequency of the network signal big data stream to obtain the corresponding network signal flow size and network signal flow access frequency under each timing segment; based on the corresponding network signal flow size and network signal flow access frequency under each timing segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain a network signal normal data substream and a network signal abnormal data substream;

[0007] The network signal division result safety warning module is used to continue iteratively detecting the corresponding network signal large data stream when it is determined to be a normal network signal data substream; when it is determined to be an abnormal network signal data substream, perform abnormal safety warning processing on the abnormal network signal data substream to generate an abnormal network signal safety warning signal;

[0008] The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0009] Furthermore, the network signal big data streaming transmission module includes the following functions:

[0010] By identifying and analyzing the data flow channel flow direction of each network device entrance and exit, the data flow direction of each network device entrance and exit communication channel is obtained;

[0011] Perform bandwidth and transmission delay requirement analysis on each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth requirement and transmission delay requirement corresponding to each data flow channel at the inlet and outlet of each network device;

[0012] Perform transmission mode recognition and analysis on the data flow direction of the communication channels at the entrances and exits of each network device to obtain the transmission mode corresponding to each data flow channel at the entrances and exits of each network device;

[0013] Based on the transmission bandwidth requirements, transmission delay requirements and transmission modes of each data flow channel at the entrance and exit of each network device, the data collection equipment selection and deployment optimization analysis is performed on each data flow channel corresponding to the entrance and exit of each network device, so as to generate the deployment optimization plan of the collection equipment at the entrance and exit of each network device; according to the deployment optimization plan of the collection equipment at the entrance and exit of each network device, data collection equipment is deployed at the entrance and exit of each network device, and the data collection equipment is used to collect network signal data at the entrance and exit of each network device in real time to obtain a large data set of network signals;

[0014] The network signal big data set is transformed into a big data streaming data to obtain a network signal big data stream.

[0015] Furthermore, the step of converting the network signal big data set into big data streaming includes:

[0016] The large data set of network signals is divided into time and space dimensions to obtain network signal data subsets in different time and space distribution dimensions;

[0017] By setting a streaming transmission conversion framework, and based on the streaming transmission conversion framework, performing signal stream concurrent optimization transmission on network signal data subsets in different time and space distribution dimensions, so as to obtain each network signal concurrent transmission substream;

[0018] Each concurrent transmission sub-stream of the network signal is preferentially streamlined and compressed for transmission to obtain a large network signal data stream.

[0019] Furthermore, the signal stream concurrent optimization transmission of network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes:

[0020] Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming slicing to obtain independent streaming segments of each network signal;

[0021] Perform statistical analysis on the signal strength and delay requirements of each network signal stream segment to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment;

[0022] Based on the network signal strength corresponding to each network signal segment, the network traffic packet loss rate of the corresponding network signal streaming independent segment is calculated to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, the transmission delay analysis of the corresponding network signal streaming independent segment is performed to obtain the network signal transmission delay corresponding to each network signal segment;

[0023] A network signal stream concurrent transmission channel is established through a streaming conversion framework, and the corresponding network signal stream concurrent transmission channel is optimized based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, so as to generate a signal stream concurrent transmission optimization channel;

[0024] Based on the signal stream concurrent transmission optimization channel, each network signal stream is independently divided into segments and signal stream concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-stream.

[0025] Furthermore, the prioritizing simplification and compressing transmission of each network signal concurrent transmission sub-stream includes:

[0026] Analyze the transmission bandwidth occupancy and transmission rate of each network signal concurrent transmission sub-flow to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow;

[0027] Based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-stream, a transmission priority quantitative calculation is performed on the corresponding network signal concurrent transmission sub-stream using a network signal transmission priority calculation formula to obtain a network transmission priority metric value corresponding to each network signal transmission sub-stream;

[0028] Performing transmission delay requirement analysis on each network information concurrent transmission sub-flow to obtain the network transmission delay requirement corresponding to each network signal transmission sub-flow;

[0029] Based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams for judgment. If the corresponding network transmission priority metric value is greater than or equal to the preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream;

[0030] The network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream are subjected to network signal compression transmission processing to obtain a network signal large data stream.

[0031] Furthermore, the network signal transmission priority calculation formula is specifically:

[0032]

[0033] Where P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influencing weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value.

[0034] Furthermore, the network signal ratio difference detection module includes the following functions:

[0035] Perform traffic fluctuation time-series segmentation on the network signal big data stream to obtain network signal data sub-streams under each time-series segment;

[0036] Performing a time series flow statistics analysis on the network signal data sub-flows under each time series segment to obtain the corresponding network signal flow size under each time series segment;

[0037] Performing a statistical analysis of the timing access frequency of the network signal data sub-flows under each timing segment to obtain the corresponding network signal traffic access frequency under each timing segment;

[0038] Based on the network signal flow size and the network signal flow access frequency corresponding to each time segment, the network signal flow fluctuation ratio calculation formula is used to perform a quantitative calculation of the network signal ratio of the network signal data sub-flow under the corresponding time segment to obtain the corresponding network signal sub-flow flow fluctuation ratio under each time segment;

[0039] According to the preset network signal traffic fluctuation threshold, the ratio difference detection and division of the network signal sub-stream traffic fluctuation ratio corresponding to each timing segment is performed; if the network signal sub-stream traffic fluctuation ratio is less than the preset network signal traffic fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal normal data sub-stream; if the network signal sub-stream traffic fluctuation ratio is greater than or equal to the preset network signal traffic fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal abnormal data sub-stream.

[0040] Furthermore, the processing of performing flow fluctuation time sequence segmentation on the network signal big data stream includes:

[0041] Analyze the fluctuation trend of network traffic on the network signal big data stream to obtain the fluctuation trend of network signal traffic in time series;

[0042] Based on the time series variation trend of network signal traffic fluctuation, the network signal big data stream is subjected to nonlinear distribution drawing of network traffic fluctuation to generate a time series distribution diagram of network signal traffic fluctuation;

[0043] Perform traffic fluctuation amplitude and fluctuation peak value statistics analysis on the network signal traffic fluctuation time series distribution diagram to obtain the increase and decrease amplitude of the network signal traffic time series fluctuation and the network signal traffic time series fluctuation peak value;

[0044] Based on the increase and decrease amplitude of the network signal traffic timing fluctuation and the peak value of the network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis of the network signal traffic fluctuation timing distribution diagram is performed to obtain the network signal traffic timing fluctuation point set and the network signal traffic timing fluctuation inflection point set;

[0045] Based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set, the network signal big data stream is segmented and processed between timing points to obtain the network signal data sub-stream under each timing segment.

[0046] Furthermore, the network signal flow fluctuation ratio calculation formula is specifically:

[0047]

[0048] In the formula, θ j is the network signal sub-flow flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow size at time t in the jth time segment, is the average flow rate of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time accumulation effect adjustment coefficient, ζ j (t) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio.

[0049] Furthermore, the abnormal safety warning processing of the network signal abnormal data sub-flow includes:

[0050] Perform time series and spatial distribution analysis on the network signal abnormal data sub-flow to obtain the time series distribution characteristics and spatial distribution characteristics of the network abnormal sub-flow;

[0051] Perform spatiotemporal correlation mining analysis on the temporal distribution characteristics of network abnormal sub-flows and the spatial distribution characteristics of network abnormal sub-flows to obtain the spatiotemporal abnormal correlation pattern between the temporal distribution and spatial distribution of network abnormal sub-flows;

[0052] Based on the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of the network anomaly sub-flow, the network signal anomaly data sub-flow is evaluated for anomaly threat, and the anomaly threat level of the network signal anomaly sub-flow is obtained;

[0053] Based on the abnormal threat level of the network signal abnormal sub-flow, abnormal safety warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal safety warning signal.

[0054] Beneficial effects of the present invention:

[0055] The big data network signal security early warning system proposed by the present invention is composed of a network signal big data streaming module, a network signal ratio difference detection module, a network signal division result security early warning module and a network signal abnormal security response module. Compared with the prior art, the beneficial effect of the present application is that by identifying and analyzing the flow direction of the data flow channel at the entrance and exit of each network device, the communication relationship between each network device and other devices can be accurately determined. This process helps network administrators understand the transmission path, direction and interaction mode of the data flow, and identify the transmission mode of each data flow channel. Data acquisition equipment is deployed at the entrance and exit of each network device, and the data acquisition equipment is used to collect network signal data at the entrance and exit of each network device in real time, which can provide a large amount of comprehensive raw data for network performance analysis. This process can monitor the status, flow and performance of each device in real time, timely discover signs of network failure or performance degradation, and provide instant feedback for network maintenance personnel to help quickly respond to and solve problems. Through comprehensive data collection, network administrators can obtain detailed information on indicators such as bandwidth usage, delay, packet loss rate, etc., which is helpful to evaluate the overall security status of the network and prevent network congestion, equipment failure or security vulnerabilities in advance. At the same time, by performing big data streaming conversion on the network signal big data set, real-time and dynamic data streams can be provided for network monitoring and optimization. This process enables network managers to obtain network performance data in real time and perform traffic analysis, load balancing, fault detection and other operations. Big data streaming processing can transform the traditional batch processing mode into a real-time response mode, improve the timeliness and accuracy of data processing, especially when processing massive data. Streaming conversion can effectively reduce latency and improve analysis efficiency. The processing of streaming data can help monitor the network status in real time and respond quickly to abnormal traffic, bottlenecks or attack behaviors, thereby providing strong data support for the implementation of subsequent network security warnings. Secondly, by performing time-series segmentation processing on the network signal big data stream, large-scale and mixed network data streams can be effectively decomposed into smaller sub-streams in several time periods, and statistical analysis of the time-series traffic size can be performed. The fluctuation of network traffic in each time period can be clearly grasped, helping to identify the growth trend or sharp change of network traffic. This statistical analysis of traffic size helps to detect high load and overload periods in the network, as well as identify traffic peaks that affect network stability, thereby providing accurate data support for the subsequent anomaly detection process. It also reflects the density of network requests and the usage of various services in the network by statistical analysis of the time-series access frequency. By performing frequency statistics on each time-series segment, network administrators can understand the behavior patterns behind the traffic, such as the time periods of frequent access and the usage trends of hot applications. Frequency analysis can not only reveal the network access patterns under normal circumstances, but also help identify abnormal traffic patterns.By performing network signal ratio difference detection and division on the corresponding network signal big data stream based on the corresponding network signal traffic size and network signal traffic access frequency in each time segment, the network signal data stream can be divided into "normal" and "abnormal" categories according to the stability and volatility of the network traffic, thereby realizing accurate management and abnormal detection of network traffic. The setting of thresholds can effectively distinguish normal fluctuations from abnormal fluctuations, avoid misjudgment and missed judgment, and thus ensure the accuracy of network performance monitoring. The normal sub-stream represents the traffic fluctuation within the stable and expected range in the network, while the abnormal sub-stream indicates potential network problems, such as burst traffic, peak traffic or malicious attacks. By dividing the abnormal data sub-streams, network failures or potential attack behaviors can be quickly identified, so that corresponding countermeasures can be taken to reduce the impact of network failures on user experience. It can effectively deal with new and complex network signal abnormal behaviors, and further improve the reliability and security of network signals. Then, for the normal network signal sub-flow, iterative detection will continue, and for those data sub-flows that are judged to be abnormal, security warning processing will be immediately carried out. The biggest benefit of this stage is that it can achieve rapid response and make timely response to abnormal traffic caused by malicious attacks or system failures. For example, if DDoS attacks, virus propagation or other types of abnormal traffic appear in the network, timely detection and warning of abnormal data can buy valuable response time for the security team and avoid serious impact on network services. The generated abnormal security warning signal can not only detect potential abnormal security threats in real time, but also improve the response efficiency of network managers and reduce the time delay of human intervention. Finally, by processing abnormal security warning signals on the cloud platform and generating network signal abnormal security response protection strategies, the cloud platform, as a centralized data processing and response center, can unify and centralize the data collected from various network devices and conduct in-depth analysis. The centralized processing of warning signals on the cloud platform can not only improve the overall monitoring capability of the network, but also automatically generate corresponding protection strategies according to the big data analysis results, so that emergency protection measures can be started in real time, such as traffic cleaning, traffic restriction, intrusion defense, etc., effectively preventing malicious traffic from further affecting the network system, thereby ensuring the continuous and stable operation of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0057] Figure 1 This is a module diagram of the big data network signal security early warning system of the present invention;

[0058] Figure 2 for Figure 1Schematic diagram of the functional flow of the network signal big data streaming module;

[0059] Figure 3 for Figure 1 Schematic diagram of the functional flow of the network signal ratio difference detection module. DETAILED DESCRIPTION

[0060] The technical system of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0061] To achieve this, please refer to Figures 1 to 3 The present invention provides a big data network signal security early warning system, the system includes the following modules:

[0062] The network signal big data streaming transmission module is used to deploy data collection equipment at the entrance and exit of each network device, and use the data collection equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large data set of network signals; perform big data streaming transmission conversion on the large data set of network signals to obtain a large data stream of network signals;

[0063] The network signal ratio difference detection module is used to perform a statistical analysis of the timing flow size and access frequency of the network signal big data stream to obtain the corresponding network signal flow size and network signal flow access frequency under each timing segment; based on the corresponding network signal flow size and network signal flow access frequency under each timing segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain a network signal normal data substream and a network signal abnormal data substream;

[0064] The network signal division result safety warning module is used to continue iteratively detecting the corresponding network signal large data stream when it is determined to be a normal network signal data substream; when it is determined to be an abnormal network signal data substream, perform abnormal safety warning processing on the abnormal network signal data substream to generate an abnormal network signal safety warning signal;

[0065] The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0066] In the embodiment of the present invention, please refer to Figure 1FIG. 1 is a schematic diagram of a module of a big data network signal security early warning system of the present invention. In this example, the big data network signal security early warning system includes the following modules:

[0067] S1: A network signal big data streaming module, which is used to deploy data collection equipment at the entrance and exit of each network device, and use the data collection equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large data set of network signals; and perform big data streaming conversion on the large data set of network signals to obtain a large data stream of network signals;

[0068] In an embodiment of the present invention, the traffic flow of each network device is monitored and analyzed at the inlet and outlet to clarify the communication flow direction of each network device. The data traffic of the inlet port and the outlet port of each network device is divided into different data flow channels during the analysis process. After the data flow channel flow direction is identified, the bandwidth and delay requirements of each data flow channel are analyzed. For each identified data flow channel, a network performance testing tool (such as iPerf, Wireshark) is used to measure the real-time bandwidth and transmission delay of the traffic. The transmission mode of the data flow direction of each communication channel is analyzed to identify the characteristics of the data flow. Different data streams have different transmission modes, such as TCP stream, UDP stream, broadcast, multicast, etc. By adopting streaming data analysis technology and using protocol analysis tools (such as NetFlow, sFlow) to identify the protocol type of the traffic, the transmission mode of each data flow channel of the network device will be recorded in detail. At the same time, based on the above analysis, the selection and deployment optimization analysis of the collection equipment at the entrances and exits of each network device is carried out. The specific operations include analyzing the performance requirements of the collection equipment based on the bandwidth requirements, delay requirements and transmission modes of each data flow channel, selecting network collection equipment with sufficient throughput according to the bandwidth requirements, and ensuring that it supports protocols compatible with the selected transmission mode (such as TCP, UDP, etc.), and considering the deployment location of the equipment to ensure that the collection equipment can cover all key entrances and exits, and using optimization algorithms (such as integer programming) to optimize the deployment of the collection equipment to ensure that the deployment of the equipment meets the maximum data collection coverage and minimizes the cost, so that the data collection equipment is deployed at the entrance and exit of each network device, where the collection equipment is connected to the physical port of the network device through a network interface (such as a network card, a fiber optic interface, etc.), and real-time data traffic and signals are collected. The collection process is performed by a high-precision collection card (such as Packet Shark, TAP equipment) to ensure that data is not lost and the collection accuracy is high, and it is transmitted to the centralized processing platform through a unified data collection system. These data will constitute a large-scale network signal data set, thereby obtaining a large network signal data set. Then, by using the big data processing framework (such as Apache Kafka, Apache Flink) to stream and transform the collected network signal big data sets, the collected raw data is pushed to the data processing platform through the transmission protocol (such as HTTP, MQTT, gRPC, etc.), and the stream processing engine is used to process the data in real time and convert it into a data stream that meets the subsequent analysis and monitoring needs. The data stream undergoes cleaning, formatting, classification and other operations in this process to obtain a structured network signal big data stream. This big data stream will be further used for real-time security detection, abnormal traffic identification, attack warning and other functions to realize the automation and refinement of network security management, and finally obtain a network signal big data stream.

[0069] S2: A network signal ratio difference detection module is used to perform a statistical analysis on the timing flow size and access frequency of the network signal big data stream to obtain the corresponding network signal flow size and network signal flow access frequency under each timing segment; based on the corresponding network signal flow size and network signal flow access frequency under each timing segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain a network signal normal data substream and a network signal abnormal data substream;

[0070] In an embodiment of the present invention, a large network signal data stream is collected by a data acquisition device and converted by streaming, including but not limited to source address, destination address, timestamp, transmission protocol type, data packet size, data transmission rate, network transmission flow and other information, and a time sliding window algorithm or a segmentation method based on event triggering is used to segment the network signal stream according to a preset time interval (i.e., time segments such as between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point), and the original network signal stream is divided into multiple time segments, each of which corresponds to a certain time segment. The network signal data sub-flow within the time range is obtained, and the traffic size of the network signal data sub-flow in each time segment obtained by the previous division is counted, and the data transmission volume in each time segment is analyzed. Specifically, according to the size of each data packet, the total data transmission volume in the time segment is calculated to obtain the traffic size of the time segment. For example, if a time segment contains 10 data packets, and the size of each data packet is 100KB, 200KB, 50KB, etc., then the traffic size of the time segment is the sum of all data packet sizes, so as to obtain the corresponding network signal traffic size under each time segment. At the same time, the access frequency is counted based on the network signal data sub-flow in each timing segment. The definition of access frequency refers to the number of times each source IP or target IP accesses the network signal in a certain timing segment. By analyzing the source IP and target IP information of each data packet in the timing segment, the number of visits of each IP address in each timing segment is counted. For example, if source IP address A visits 3 times and source IP address B visits 5 times in a certain timing segment, the access frequency of the timing segment is A: 3 / 8, B: 5 / 8, thereby obtaining the corresponding network signal traffic access frequency in each timing segment.Then, by combining the corresponding network signal flow size, network signal flow access frequency and related parameters under the timing segment, a suitable flow fluctuation ratio calculation formula is formed to perform a quantitative calculation of the network signal ratio on the network signal data sub-flow under the corresponding timing segment. By quantitatively calculating the ratio, the abnormal fluctuation of the network signal in the timing segment can be quantitatively evaluated. If the flow fluctuation ratio is large, it means that the network signal of the timing segment has an abnormal fluctuation; otherwise, it can be considered that the network signal of the timing segment is in a normal state, and a network signal flow is set according to historical data or the normal flow characteristics of the network environment. Fluctuation threshold, which represents the range of normal fluctuations. For example, if in a specific network environment, the network signal flow fluctuation usually does not exceed a certain percentage value (such as 20%), then this percentage value is the threshold. Then, by comparing the relationship between the network signal sub-flow flow fluctuation ratio of each timing segment and the preset threshold, if the fluctuation ratio of a certain timing segment is less than the threshold, the network signal in the timing segment is judged to be a normal fluctuation and is divided into a network signal normal data sub-flow; if the fluctuation ratio is greater than or equal to the threshold, the network signal in the timing segment is considered to be an abnormal fluctuation and is divided into a network signal abnormal data sub-flow.

[0071] S3: A network signal division result security warning module, which is used to, when it is determined that the network signal is a normal data substream, continue to iteratively detect the corresponding network signal large data stream; when it is determined that the network signal is an abnormal data substream, perform abnormal security warning processing on the network signal abnormal data substream to generate a network signal abnormal security warning signal;

[0072] In an embodiment of the present invention, if a substream is determined to be a normal network signal data substream, the data stream is iteratively monitored and further time series detection is performed. The normal traffic and access frequency of each time series segment should meet the pre-defined threshold range. If the abnormal detection result shows that the traffic or access mode has changed, the data stream needs to be further detected in real time. In this process, a certain dynamic threshold can be set, and the detection standard can be gradually adjusted according to historical data and traffic characteristics; and if a data substream is determined to be an abnormal network signal data substream, the corresponding abnormal security warning mechanism will be triggered. At this time, the abnormal data substream will be isolated separately and analyzed and processed in real time. By setting a multi-dimensional anomaly detection model, such as a classification algorithm based on machine learning or a rule-based discrimination method, various security risks such as traffic anomalies and protocol non-compliance can be accurately identified. When the data stream is identified as abnormal, a security warning signal will be generated in response to prompt relevant personnel or automatically trigger a security response mechanism, and finally a network signal abnormality security warning signal will be generated.

[0073] S4: Network signal abnormality security response module, used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0074] In an embodiment of the present invention, when the abnormal data substream is identified and an abnormal security warning signal is generated, the warning signal will be uploaded to the cloud platform, and the cloud platform is responsible for further processing and responding to these abnormal security warning signals. Specifically, after the network signal abnormal security warning signal is uploaded to the cloud platform, the cloud platform will automatically process it according to the preset security response mechanism. The security response mechanism includes but is not limited to traffic speed limit, source IP ban, protocol anomaly detection and protection strategy, etc. The cloud platform generates personalized protection strategies by utilizing artificial intelligence algorithms or rule engines, combined with historical attack patterns, network topology and real-time network status. These protection strategies may include modifying firewall strategies, adjusting router traffic distribution, or starting distributed denial of service (DDoS) attack protection modules, etc. The processed response strategy will be promptly fed back to the network device to achieve real-time protection of abnormal network signals. The generated protection strategy will be sent to each network device through encrypted transmission to ensure network security and prevent potential attacks or security threats from damaging the system, and finally generate a network signal abnormal security response protection strategy.

[0075] Further, as an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 FIG. 1 is a functional flow chart of a network signal big data streaming transmission module. In this embodiment, the functions of the network signal big data streaming transmission module include:

[0076] S11: by identifying and analyzing the data flow channel flow direction of each network device inlet and outlet, the data flow direction of each network device inlet and outlet communication channel is obtained;

[0077] In an embodiment of the present invention, the communication direction of each network device is clarified by monitoring and analyzing the traffic at the inlet and outlet of each network device. First, a traffic monitoring tool (such as a traffic analyzer, a network traffic probe) is used to capture real-time data of the physical and logical ports of the device. The data traffic of the inlet port and the outlet port of each network device is divided into different data flow channels during the analysis process. The data flow direction of each flow channel is identified through deep packet analysis (DPI) technology. The process also takes into account the network topology to ensure that the identified flow direction is consistent with the actual data flow path. Through this analysis, the directionality of the inlet and outlet traffic of each device can be obtained, and finally the data flow direction of the inlet and outlet communication channels of each network device can be obtained.

[0078] S12: Analyze the bandwidth and transmission delay requirements of each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth requirement and transmission delay requirement corresponding to each data flow channel at the inlet and outlet of each network device;

[0079] In an embodiment of the present invention, after the flow direction of the data flow channel is identified, the bandwidth and delay requirements of each data flow channel are analyzed, and for each identified data flow channel, a network performance testing tool (such as iPerf, Wireshark) is used to measure the real-time bandwidth and transmission delay of the traffic. This process will perform a stress test on each data flow channel, simulate the transmission conditions under different traffic conditions, and calculate the maximum bandwidth requirement and minimum delay requirement of each flow channel. The bandwidth requirement will be estimated based on the actual data traffic and network load prediction, and the transmission delay will be obtained by measuring the delay of the data packet from the source device to the target device. These analysis results will construct detailed bandwidth and delay requirement data, and finally obtain the transmission bandwidth requirement and transmission delay requirement corresponding to each data flow channel at the entrance and exit of each network device.

[0080] S13: performing transmission mode recognition and analysis on the data flow direction of the communication channels at the inlet and outlet of each network device to obtain the transmission mode corresponding to each data flow channel at the inlet and outlet of each network device;

[0081] In an embodiment of the present invention, the transmission mode of the data flow direction of the communication channel at the inlet and outlet of each network device is analyzed to identify the characteristics of the data flow. Different data flows have different transmission modes, such as TCP flow, UDP flow, broadcast, multicast, etc. By adopting streaming data analysis technology, protocol analysis tools (such as NetFlow, sFlow) are used to identify the protocol type of the traffic. During the analysis process, the different transmission modes of the traffic will be classified, and the transmission mode of each data flow channel of the network device will be recorded in detail. The identification result is crucial for subsequent equipment selection and deployment optimization, because traffic with different transmission modes has different requirements for network resources (such as bandwidth, routing strategy), and finally the transmission mode corresponding to each data flow channel at the inlet and outlet of each network device is obtained.

[0082] S14: Based on the transmission bandwidth requirements, transmission delay requirements and transmission modes corresponding to each data flow channel at the entrance and exit of each network device, the data collection equipment selection and deployment optimization analysis is performed on each data flow channel corresponding to the entrance and exit of each network device, so as to generate the deployment optimization plan of the collection equipment at the entrance and exit of each network device; according to the deployment optimization plan of the collection equipment at the entrance and exit of each network device, the data collection equipment is deployed at the entrance and exit of each network device, and the data collection equipment is used to collect network signal data at the entrance and exit of each network device in real time, so as to obtain a large data set of network signals;

[0083] In an embodiment of the present invention, the selection and deployment optimization analysis of the collection equipment at the entrances and exits of each network device is performed on the basis of the above analysis. The specific operations include analyzing the performance requirements of the collection equipment based on the bandwidth requirements, latency requirements and transmission mode of each data flow channel, selecting a network collection device with sufficient throughput capacity according to the bandwidth requirements, and ensuring that it supports a protocol compatible with the selected transmission mode (such as TCP, UDP, etc.). At the same time, the deployment location of the equipment is considered to ensure that the collection equipment can cover all key entrances and exits and support comprehensive collection of network signals. The deployment of the collection equipment is optimized using an optimization algorithm (such as integer programming) to ensure that the deployment of the equipment meets the requirements of maximizing data collection coverage and minimizing costs, thereby generating an optimized deployment plan for the collection equipment at the entrances and exits of each network device. Then, according to the generated collection equipment deployment optimization plan, data collection equipment is deployed at the entrance and exit of each network device. The collection equipment is connected to the physical port of the network device through a network interface (such as a network card, optical fiber interface, etc.) to collect the passing data flow and signal in real time. The collection process is executed by a high-precision collection card (such as Packet Shark, TAP device) to ensure that data is not lost and the collection accuracy is high. All collected signal data, including flow information, protocol data, delay data, etc., will be transmitted to the centralized processing platform through a unified data collection system. These data will constitute a large-scale network signal data set, and finally a large network signal data set will be obtained.

[0084] S15: Perform big data streaming conversion on the network signal big data set to obtain a network signal big data stream.

[0085] In an embodiment of the present invention, the collected network signal big data set is streamed and converted by utilizing a big data processing framework (such as Apache Kafka, Apache Flink), the collected raw data is pushed to the data processing platform through a transmission protocol (such as HTTP, MQTT, gRPC, etc.), and the stream processing engine is used to process the data in real time and convert it into a data stream that meets the subsequent analysis and monitoring requirements. The data stream undergoes cleaning, formatting, classification and other operations in this process to obtain a structured network signal big data stream, which will be further used for real-time security detection, abnormal traffic identification, attack warning and other functions to realize the automation and refinement of network security management, and finally obtain a network signal big data stream.

[0086] Furthermore, the step of converting the network signal big data set into big data streaming includes:

[0087] The large data set of network signals is divided into time and space dimensions to obtain network signal data subsets in different time and space distribution dimensions;

[0088] In an embodiment of the present invention, the acquired large data set of network signals is divided into time and space dimensions. The specific operation is to first analyze the timestamp and geographic location mark of the network signal data, and use a multidimensional data analysis method to process the data set. In the process of time and space division, the time dimension is subdivided into multiple time windows, such as hours, days or weeks, and the space dimension is divided by geographic information. The signal source can be located using longitude and latitude data. In the specific implementation, spatial clustering based on the K-means clustering algorithm is used to group signals in different areas. In addition, the sliding window technology is used to divide the time domain data. This process ensures that each data subset can reflect the changing characteristics of the network signal within a specific time and space range, so that the subsequent analysis can be deeply mined and processed for different time and space scenarios, and finally obtain network signal data subsets under different time and space distribution dimensions.

[0089] Preferably, a streaming conversion framework is set, and based on the streaming conversion framework, signal stream concurrent optimization transmission is performed on network signal data subsets in different time and space distribution dimensions to obtain each network signal concurrent transmission substream;

[0090] In an embodiment of the present invention, a streaming conversion framework is set to ensure that data can be transmitted and processed in real time through an efficient streaming processing method. The specific implementation method is: based on a distributed messaging system such as Apache Kafka, a subset of network signal data is converted into streaming data, and the data is transmitted through the consumer and producer mechanism in Kafka. In the streaming transmission process, corresponding priorities and bandwidth control strategies are set for network signal data subsets divided in different time and space, and concurrent optimized transmission is performed using traffic scheduling algorithms such as dynamic bandwidth allocation and traffic shaping. The framework automatically adjusts the transmission priority of each data substream according to the time sensitivity and spatial characteristics of the network signal to ensure efficient transmission of the network signal without packet loss. In this process, the traffic of the network signal is monitored in real time, and a sliding window-based algorithm is used to load balance the transmission of the signal stream to prevent packet loss caused by network congestion or a sudden increase in the signal volume, thereby achieving optimized concurrent transmission of the signal stream and ultimately obtaining each network signal concurrent transmission substream.

[0091] Preferably, each concurrently transmitted sub-stream of the network signal is preferentially streamlined and compressed for transmission to obtain a large network signal data stream.

[0092] In an embodiment of the present invention, the network signal sub-streams that have been transmitted concurrently are streamlined and compressed. The core purpose of the streamlined processing is to remove redundant data of non-priority transmission and retain key feature data, thereby reducing the amount of data transmission and improving efficiency. The specific implementation method is to analyze and screen out the corresponding non-priority transmission sub-streams, and remove redundant information that has no significant impact on subsequent analysis. At the same time, compression algorithms such as Huffman coding, LZ77, etc. are used to compress the priority transmission sub-streams and the streamlined non-priority transmission sub-streams to further reduce the size of the data stream. When compressing the data, it is necessary to ensure that the compressed data can still restore sufficient accuracy and signal characteristics to support subsequent security warning analysis. During the compression process, it is also necessary to dynamically adjust the parameters of the compression algorithm according to the network bandwidth situation to make the transmission of the data stream smoother and more efficient, and finally obtain a large network signal data stream.

[0093] Furthermore, the signal stream concurrent optimization transmission of network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes:

[0094] Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming slicing to obtain independent streaming segments of each network signal;

[0095] In an embodiment of the present invention, a streaming conversion framework is used to perform streaming slicing processing on network signal data subsets in different time and space distribution dimensions. The specific operation is to divide the network signal data set in the time dimension and the space dimension, and each network signal data subset will be independently sliced ​​according to different timestamps (such as every second, every minute or more detailed time periods) and spatial distributions (for example, different nodes and regions of the network). This processing method requires the real-time and data integrity of the signal flow, and the size of the slice needs to be reasonably set according to the network bandwidth, data transmission requirements and processing capabilities. Through the algorithm in the streaming framework, each data fragment (that is, the network signal streaming fragment) is time-stamped and its continuity during the streaming process is ensured to avoid data loss or disorder, ensure that the time and space dimension information of the network signal can be accurately identified, and finally obtain each network signal streaming independent fragment.

[0096] Preferably, a statistical analysis of the signal strength and delay requirements of each network signal stream independent segment is performed to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment;

[0097] In an embodiment of the present invention, a statistical analysis of the signal strength and delay requirements is performed on each independent segment of the network signal stream obtained by the previous division. Specifically, the signal strength data is first extracted from each signal segment, and the signal strength value corresponding to each segment is statistically analyzed according to the network transmission standard. These strength data include parameters such as the power and bit error rate of the received signal, which reflect the quality of the network signal. It is also necessary to perform statistics on the delay requirements according to the actual application scenarios of the network (such as data transmission, real-time communication, etc.). The delay requirement usually refers to the maximum tolerable delay time of each data segment, which depends on the quality of the network, the transmission protocol and the actual needs. For example, in a large data stream processing system, the transmission delay of video data is required not to exceed 200ms, while the delay requirement for ordinary data transmission is 500ms. At this time, the delay statistics and signal strength analysis based on the streaming data can provide basic data support for subsequent network optimization, and finally obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment.

[0098] Preferably, based on the network signal strength corresponding to each network signal segment, the network traffic packet loss rate of the corresponding network signal streaming independent segment is calculated to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, the transmission delay analysis of the corresponding network signal streaming independent segment is performed to obtain the network signal transmission delay corresponding to each network signal segment;

[0099] In the embodiment of the present invention, the network traffic packet loss rate is calculated and the transmission delay is analyzed for each network signal segment based on the signal strength and transmission delay requirements. First, the corresponding traffic packet loss rate is calculated according to the signal strength of each segment. The calculation of the packet loss rate is usually based on the ratio between the number of received data packets and the product of the number of sent data packets and the signal strength. If the signal strength of the segment is weak, the packet loss rate is high; on the contrary, the packet loss rate is low. In order to accurately calculate the packet loss rate, a certain error tolerance can be adopted, and dynamic adjustment can be made according to historical data and network environment, so as to obtain the network traffic packet loss rate corresponding to each network signal segment. Then, the transmission delay requirement of each segment is analyzed. The delay analysis includes calculating the delay time experienced by each signal segment in the network from the source node to the target node. The processing time of network devices such as routers and switches, the link transmission speed, and the transmission process of data packets in the network are taken into account during the analysis. A comprehensive evaluation is performed in combination with the network topology structure. For segments that do not meet the delay requirements, it is necessary to further optimize the network path or adjust the transmission rate, and finally obtain the network signal transmission delay corresponding to each network signal segment.

[0100] Preferably, a network signal stream concurrent transmission channel is established through a streaming conversion framework, and the corresponding network signal stream concurrent transmission channel is optimized based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, so as to generate a signal stream concurrent transmission optimization channel;

[0101] In an embodiment of the present invention, a concurrent transmission channel of a network signal stream is constructed by using a streaming conversion framework, and these channels are optimized in combination with the corresponding network traffic packet loss rate and network signal transmission delay. The specific operation includes generating multiple concurrent transmission channels based on the streaming conversion framework according to the packet loss rate and transmission delay requirements of the independent segments of the network signal stream obtained in the previous stage. These channels are finely configured and scheduled according to the signal strength, delay and packet loss rate to ensure the transmission quality and efficiency of the network signal. For example, in a large-scale data stream system, the concurrent channels of the network signal stream can be optimized by dynamic traffic scheduling algorithms. These algorithms take into account the real-time status of each segment in the network, the usage of bandwidth resources and the current load level, so as to select the most appropriate transmission path and concurrent channel. For signal streams with high packet loss rates or large delays, they can be guided to more stable paths in the network, and traffic distribution can be dynamically adjusted to reduce network congestion, thereby ensuring that the delay and quality requirements of each signal segment are met, and finally generating a signal stream concurrent transmission optimization channel.

[0102] Preferably, based on the signal stream concurrent transmission optimization channel, each network signal stream is independently divided into segments and signal stream concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-stream.

[0103] In an embodiment of the present invention, each network signal stream independent segment is concurrently optimized and transmitted based on an optimized signal stream concurrent transmission channel, so that each signal stream can be transmitted on multiple parallel paths through the optimized concurrent transmission channel, thereby improving the transmission rate and reducing the transmission delay. Specifically, each network signal stream independent segment is concurrently scheduled for transmission to ensure that these segments are processed smoothly and efficiently in the concurrent transmission channel, and dynamically adjusted according to the characteristics and traffic load of each segment to ensure full utilization of the network bandwidth, and adjust the traffic distribution according to the real-time status of each channel. For example, in a big data real-time monitoring scenario, the traffic of some network signals is large, and more concurrent channels are required to carry their transmission needs. In this way, the concurrent transmission strategy of the signal is dynamically adjusted, and high-traffic signal segments are scheduled to more transmission channels, thereby improving data transmission efficiency and meeting the real-time requirements of the network, and finally obtaining each network signal concurrent transmission sub-stream.

[0104] Furthermore, the prioritizing simplification and compressing transmission of each network signal concurrent transmission sub-stream includes:

[0105] Analyze the transmission bandwidth occupancy and transmission rate of each network signal concurrent transmission sub-flow to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow;

[0106] In the embodiment of the present invention, the bandwidth occupancy and transmission rate of each concurrently transmitted network signal sub-flow are analyzed in detail. In the specific implementation, each signal flow in the network is first monitored in real time by a network monitoring device or a traffic analysis tool (such as Wireshark, NetFlow or sFlow). These tools can accurately record the number of data packets, transmission frequency and network bandwidth usage of each sub-flow. Based on these data, the bandwidth occupancy of each sub-flow is calculated using the formula: At the same time, the transmission rate of each sub-stream is calculated. The transmission rate is usually determined by sampling the transmission time and data volume of each sub-stream. The formula is: Finally, the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow are obtained.

[0107] Preferably, based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-stream, a network signal transmission priority calculation formula is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-streams, so as to obtain the network transmission priority metric value corresponding to each network signal transmission sub-stream;

[0108] In an embodiment of the present invention, a suitable network signal transmission priority calculation formula is formed by combining the transmission bandwidth occupancy, transmission rate, network transmission delay, network transmission priority impact weight coefficient, network transmission duration, integral time variable parameter, network transmission condition impact factor, network transmission packet loss impact weight coefficient, network transmission packet loss rate and related parameters corresponding to each network signal transmission sub-stream to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-stream, so as to quantitatively calculate the priority situation corresponding to the corresponding transmission sub-stream, and finally obtain the network transmission priority measurement value corresponding to each network signal transmission sub-stream.

[0109] Preferably, a transmission delay requirement analysis is performed on each network information concurrent transmission sub-flow to obtain a network transmission delay requirement corresponding to each network signal transmission sub-flow;

[0110] In an embodiment of the present invention, by analyzing the delay requirements of each network signal transmission sub-stream, the delay requirement analysis usually determines the delay requirement of each sub-stream according to the application scenario of each sub-stream (such as video stream, voice stream, data transmission, etc.), and classifies each transmission sub-stream according to a preset delay requirement rule: for applications with high real-time requirements (such as VoIP, real-time video conferencing, etc.), the delay requirement is a low delay requirement; and for general data transmission applications (such as file download, backup, etc.), the delay requirement is a high delay requirement. In this way, the actual transmission delay of each sub-stream can be measured by a network performance testing tool (such as iperf, Ping, etc.), and compared with the preset delay standard for analysis. If the delay requirement of a sub-stream is low (lower than a certain threshold), the sub-stream will be marked as a low delay requirement sub-stream. If the delay requirement is high, it will be classified as a high delay requirement sub-stream, and finally the network transmission delay requirement corresponding to each network signal transmission sub-stream is obtained.

[0111] Preferably, based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams for judgment. If the corresponding network transmission priority metric value is greater than or equal to the preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream;

[0112] In an embodiment of the present invention, data flow division is determined for each network signal concurrent transmission sub-stream through the previously calculated transmission priority metric value and the delay requirement analysis result obtained through analysis. Specifically, first, the priority metric values ​​of all sub-streams are compared with a preset network transmission priority threshold. If the priority metric value of a sub-stream is greater than or equal to the threshold, it indicates that the sub-stream has a higher priority in the network. Next, the delay requirement of the sub-stream is checked. If the sub-stream has a low delay requirement and its priority metric value is high, optimized transmission can be performed. The optimized transmission can ensure that the sub-stream is transmitted with the lowest delay by adjusting the routing strategy, selecting a low-delay network path, increasing the bandwidth, etc., so as to obtain a network signal optimized transmission sub-stream. If the priority metric value of the sub-stream is low, or the delay requirement is a high delay requirement, it should be streamlined. The streamlined processing includes reducing the data flow or adopting a delay-tolerant transmission method to avoid interference from high-priority data streams. The streamlined sub-stream will become a network signal streamlined transmission sub-stream.

[0113] Preferably, the network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream are subjected to network signal compression transmission processing to obtain a network signal large data stream.

[0114] In an embodiment of the present invention, by compressing and transmitting the previously determined optimized transmission substream of the network signal and the simplified transmission substream of the network signal, an efficient compression algorithm (such as Huffman coding, LZ77 compression algorithm, etc.) can be used for the optimized transmission substream, and the bandwidth occupied during the transmission process can be reduced by removing redundant data and optimizing the coding method. The data compression processing of the optimized transmission substream will focus on improving transmission efficiency and reducing delay; while for the simplified transmission substream, a more powerful compression method can be adopted, such as directly discarding the non-critical data part, or compressing it through an algorithm with a higher compression ratio to further reduce bandwidth occupancy and transmission delay. This process can be implemented through a dedicated network signal compressor (such as ZIP, RAR and other tools) or a custom algorithm. After the optimized transmission and simplified transmission substreams are compressed, they are converged into a large data stream. This process integrates multiple small substreams into a unified large data stream through a centralized data transmission mechanism, and finally obtains a network signal large data stream.

[0115] Furthermore, the network signal transmission priority calculation formula is specifically:

[0116]

[0117] Where P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influencing weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value.

[0118] The present invention obtains a network signal transmission priority calculation formula by using a specific mathematical model and after verification, which is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-stream. The network signal transmission priority calculation formula can ensure that the ratio of the transmission rate and bandwidth occupancy of the network signal flow is effectively weighed by introducing the two parameters of transmission bandwidth occupancy and transmission rate. The flow with high rate and small bandwidth occupancy will have a higher priority, ensuring reasonable allocation of resources and avoiding bottlenecks caused by insufficient bandwidth. Delay is a key factor in measuring network transmission delay. The existence of the delay term in the formula reduces the priority of the transmission sub-stream with higher network delay, and the high-delay flow will be punished to avoid occupying too many transmission resources when the delay is high, thereby optimizing the overall network delay performance. By reducing the priority of the high-delay flow, the transmission efficiency of the low-delay flow can be optimized, and the network experience of applications with high real-time requirements (such as video conferencing, online games, etc.) can be improved. By combining the transmission duration with the time integration factor, the sensitivity of signal transmission condition changes is modeled. The network condition impact factor can reflect the quality fluctuations of the network in different time periods, such as delay, packet loss rate, etc. This helps to dynamically adjust the transmission priority according to the real-time network conditions. For long-term transmission under low-quality or unstable network conditions (such as big data upload), this formula helps to identify and prioritize those streams with better conditions or more urgent conditions, avoiding performance degradation in long-term data transmission under low-quality connections. Packet loss in network transmission usually means poor network conditions or excessive load. In the formula, streams with high packet loss rates are penalized and their transmission priority is reduced. By adding the packet loss impact term, it can be ensured that when the network is congested or unstable, the transmission of data streams with less packet loss and better quality is prioritized, thereby reducing retransmissions and delays. In addition, by introducing a correction coefficient to adjust the basic level of the transmission priority metric, it can be used as a global parameter to adjust the flexibility and accuracy of the entire priority model. For example, when the network conditions are particularly complex or there are special needs, the correction coefficient can be adjusted to ensure that the priority of a specific flow can adapt to the actual needs. This flexibility is very important for certain specific application scenarios, such as retaining the priority of key flows under high load. This formula can quantify the priority of each transmission sub-flow by comprehensively considering multiple factors (bandwidth occupancy, rate, delay, duration, network conditions, packet loss, etc.). According to the priority calculated by the formula, appropriate network resources can be allocated to each flow to ensure efficient operation of the network. In summary, this formula fully considers the network transmission priority metric P corresponding to the i-th network signal transmission sub-flow i , the transmission bandwidth occupied by the i-th network signal transmission sub-flow corresponding to B i , the transmission rate S corresponding to the i-th network signal transmission sub-flow i , the network transmission delay L corresponding to the i-th network signal transmission sub-flow i, network transmission priority influence weight coefficient α, network transmission time T corresponding to the i-th network signal transmission sub-flow i , integral time variable parameter τ, network transmission condition influencing factor β(τ) at time τ, network transmission packet loss influencing weight coefficient γ, network transmission packet loss rate R corresponding to the i-th network signal transmission subflow i , the correction coefficient η of the network transmission priority metric value, according to the network transmission priority metric value P corresponding to the i-th network signal transmission sub-flow i The correlation between the above parameters constitutes a functional relationship This formula can realize the transmission priority quantitative calculation process of the corresponding network signal concurrent transmission sub-streams. At the same time, the introduction of the correction coefficient η of the network transmission priority metric value can be adjusted according to the error situation occurring in the calculation process, thereby improving the accuracy and applicability of the network signal transmission priority calculation formula.

[0119] Further, as an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 The function flow diagram of the network signal ratio difference detection module in the embodiment includes:

[0120] S21: performing flow fluctuation time sequence segmentation processing on the network signal big data stream to obtain network signal data sub-streams under each time sequence segment;

[0121] In an embodiment of the present invention, a large network signal data stream is collected by a data acquisition device and converted through streaming, including but not limited to source address, destination address, timestamp, transmission protocol type, data packet size, data transmission rate, network transmission flow and other information, and the network signal stream is segmented and processed according to preset time intervals (i.e., timing segments such as between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point) through a time sliding window algorithm or an event-triggered segmentation method, and the original network signal stream is divided into multiple timing segments, each timing segment corresponds to a network signal data substream within a certain time range, and the time interval in the segmentation process can be flexibly adjusted according to the characteristics of the network signal, monitoring requirements or preset strategies, and finally the network signal data substreams under each timing segment are obtained.

[0122] S22: performing a time series flow statistics analysis on the network signal data sub-flows in each time series segment to obtain the corresponding network signal flow size in each time series segment;

[0123] In an embodiment of the present invention, traffic size statistics are performed on the network signal data sub-flows in each timing segment obtained by the previous division, and the data transmission volume in each timing segment is analyzed. Specifically, based on the size of each data packet, the total amount of data transmission in the timing segment is calculated to obtain the traffic size of the timing segment. For example, if a timing segment contains 10 data packets, and the size of each data packet is 100KB, 200KB, 50KB, etc., then the traffic size of the timing segment is the sum of the sizes of all data packets. The traffic statistics methods used in this step include the accumulation method, the average method, etc., and the traffic size calculation can be performed using a traffic statistics tool or a custom script to record the traffic size of each timing segment, and finally the corresponding network signal traffic size under each timing segment is obtained.

[0124] S23: performing a time sequence access frequency statistical analysis on the network signal data sub-flows under each time sequence segment to obtain the corresponding network signal flow access frequency under each time sequence segment;

[0125] In an embodiment of the present invention, the access frequency is counted based on the network signal data sub-flow in each timing segment. The definition of access frequency refers to the number of times each source IP or target IP accesses the network signal in a certain timing segment. By analyzing the source IP and target IP information of each data packet in the timing segment, the number of times each IP address accesses in each timing segment is counted. For example, if source IP address A accesses 3 times and source IP address B accesses 5 times in a certain timing segment, the access frequency of the timing segment is A: 3 / 8 and B: 5 / 8. In order to effectively capture the characteristics of frequent access, the access frequency statistical data can be normalized to reduce the impact of data fluctuations on the results, ensure that the statistical results can accurately reflect the trend of the access frequency, and finally obtain the corresponding network signal traffic access frequency under each timing segment.

[0126] S24: Based on the network signal flow size and the network signal flow access frequency corresponding to each time segment, a network signal flow fluctuation ratio calculation formula is used to perform a network signal ratio quantitative calculation on the network signal data sub-flow under the corresponding time segment, so as to obtain the network signal sub-flow flow fluctuation ratio corresponding to each time segment;

[0127] In an embodiment of the present invention, a suitable network signal traffic fluctuation ratio calculation formula is formed by combining the corresponding duration, time variable parameters, network signal traffic size, average traffic, network signal traffic access frequency, time cumulative effect adjustment coefficient, network environment fluctuation factor and related parameters under the timing segment to perform network signal ratio quantitative calculation on the network signal data sub-stream under the corresponding timing segment. By quantitatively calculating the ratio, the abnormal fluctuation of the network signal in the timing segment can be quantitatively evaluated. If the traffic fluctuation ratio is large, it means that the network signal of the timing segment has abnormal fluctuation; otherwise, it can be considered that the network signal of the timing segment is in a normal state, and finally the corresponding network signal sub-stream traffic fluctuation ratio under each timing segment is obtained.

[0128] S25: Perform ratio difference detection and division on the network signal sub-stream flow fluctuation ratios corresponding to each timing segment according to the preset network signal flow fluctuation threshold. If the network signal sub-stream flow fluctuation ratio is less than the preset network signal flow fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal normal data sub-stream; if the network signal sub-stream flow fluctuation ratio is greater than or equal to the preset network signal flow fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal abnormal data sub-stream.

[0129] In an embodiment of the present invention, a network signal flow fluctuation threshold is set according to historical data or normal flow characteristics of the network environment. The threshold represents the range of normal fluctuations. For example, if in a specific network environment, the network signal flow fluctuation usually does not exceed a certain percentage value (such as 20%), then this percentage value is the threshold. Then, by comparing the relationship between the network signal sub-flow flow fluctuation ratio of each time segment and the preset threshold, if the fluctuation ratio of a certain time segment is less than the threshold, the network signal in the time segment is judged to be a normal fluctuation and is divided into a network signal normal data sub-flow; if the fluctuation ratio is greater than or equal to the threshold, the network signal in the time segment is considered to be an abnormal fluctuation and is divided into a network signal abnormal data sub-flow. In this process, an automated tool or a rule-based system can be used to classify and mark the data in real time. The core purpose of this step is to quickly identify abnormal fluctuations in network signals through real-time detection and classification, thereby providing a basis for subsequent security warnings.

[0130] Furthermore, the processing of performing flow fluctuation time sequence segmentation on the network signal big data stream includes:

[0131] Analyze the fluctuation trend of network traffic on the network signal big data stream to obtain the fluctuation trend of network signal traffic in time series;

[0132] In an embodiment of the present invention, the corresponding network signal big data stream is obtained by previously collecting from various devices in the network (such as routers, switches, gateways, etc.) and converting them into streams. These data usually include source address, destination address, timestamp, transmission protocol type, data packet size, data transmission rate, network transmission traffic and other information. In order to analyze the fluctuation trend of the network signal traffic, the network signal big data stream is preprocessed and cleaned using a timing analysis technology to remove outliers, duplicate data or noise, and the network traffic timing data is analyzed for fluctuation trend using a timing analysis tool (for example, the Pandas library in Python or the timing data analysis tool in Matlab). The network traffic timing data is segmented using a sliding window technology (such as a window length of 5 seconds or 10 seconds). The network signal traffic fluctuation timing trend is finally obtained by calculating the traffic fluctuation amount in each time window.

[0133] Preferably, based on the network signal traffic fluctuation time series change trend, the network signal big data stream is subjected to network traffic fluctuation nonlinear distribution drawing to generate a network signal traffic fluctuation time series distribution diagram;

[0134] In an embodiment of the present invention, by combining the network signal traffic fluctuation time series change trend obtained by previous analysis, a nonlinear statistical analysis method is used to further model and analyze the network traffic fluctuation in the network signal big data stream. The specific method is to use a probability density function (PDF) for modeling, select a suitable nonlinear distribution model (for example, Lognormal distribution or Weibull distribution) to fit the fluctuation characteristics of the network traffic data, and use statistical analysis tools, such as the Scipy library in Python or the statistical toolbox of Matlab, to fit the model parameters by least squares method or maximum likelihood estimation and other methods to generate a time series distribution diagram of the traffic fluctuation. This distribution diagram shows the fluctuation degree, frequency of occurrence and nonlinear characteristics of the network signal traffic, and can intuitively reflect the law of traffic fluctuation and the trend of abnormal fluctuation, and finally draw a network signal traffic fluctuation time series distribution diagram.

[0135] Preferably, the network signal flow fluctuation time series distribution diagram is subjected to flow fluctuation amplitude and fluctuation peak value statistical analysis to obtain the increase and decrease amplitude of the network signal flow time series fluctuation and the network signal flow time series fluctuation peak value;

[0136] In an embodiment of the present invention, a statistical analysis of the traffic fluctuation amplitude and the fluctuation peak is performed based on the previously drawn network signal traffic timing distribution diagram. Specifically, the amplitude of the network traffic fluctuation in each time window is calculated to obtain the absolute value of the traffic change. Then, the peak portion of the traffic change curve is identified to calculate the fluctuation peak, that is, the difference between the maximum and minimum values ​​of the traffic fluctuation. Automated tools (such as the NumPy library in Python for peak detection) can be used to extract the peak points in the data, and the increase or decrease amplitude of the traffic fluctuation is calculated based on the fluctuation amplitude. The increase or decrease amplitude refers to the amount of traffic change between consecutive time points, and the fluctuation peak is the extreme point in the traffic fluctuation. After statistical analysis, the fluctuation amplitude and peak value in each time period can be obtained, and finally the increase or decrease amplitude of the network signal traffic timing fluctuation and the network signal traffic timing fluctuation peak value are obtained.

[0137] Preferably, based on the increase and decrease amplitude of the network signal traffic timing fluctuation and the peak value of the network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis is performed on the network signal traffic timing distribution diagram to obtain a network signal traffic timing fluctuation point set and a network signal traffic timing fluctuation inflection point set;

[0138] In an embodiment of the present invention, based on the fluctuation increase and decrease amplitude and fluctuation peak value of the network signal traffic obtained by previous analysis, the traffic time series distribution diagram is further mined and analyzed for fluctuation points and inflection points, and an inflection point detection algorithm for time series data (for example, an inflection point detection method based on the second derivative or a CUSUM control chart method) is applied to identify the inflection points of the fluctuation trend. The inflection point is where the traffic change trend changes significantly, representing abnormal events such as network failures, attacks or burst traffic, and traffic fluctuation points are identified by using a specific fluctuation point detection algorithm (such as a threshold-based method, setting a threshold for traffic fluctuations, and when the fluctuation amplitude exceeds the threshold, it is regarded as a fluctuation point). Furthermore, the time series data can be classified by cluster analysis or pattern recognition methods (such as K-means clustering or DBSCAN algorithm) to identify obvious fluctuation clusters as a time series fluctuation point set and a fluctuation inflection point set, and finally a network signal traffic time series fluctuation point set and a network signal traffic time series fluctuation inflection point set are obtained.

[0139] Preferably, the network signal large data stream is segmented and processed between timing points based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set to obtain the network signal data sub-stream under each timing segment.

[0140] In an embodiment of the present invention, a time series data segmentation process is performed on a network signal big data stream based on a previously obtained time series fluctuation point set and corresponding points in a time series fluctuation inflection point set. First, for each interval between a time series fluctuation point and an inflection point, a time series segmentation algorithm is used to divide the network signal flow data into a plurality of sub-segments. The specific operation is: according to the time series fluctuation point set and the timestamps at each point in the inflection point set, the big data stream is divided into a plurality of smaller time series data segments, that is, time series segments such as between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point. Each time series segment corresponds to a stable flow subset. The boundary of the sub-segment is determined by using a time window algorithm (such as a dynamic time window or a fixed time window). Each sub-segment represents a relatively independent network signal flow state. By analyzing the flow fluctuation characteristics in each sub-segment, it is helpful to identify abnormal flow fluctuations or security threats in the network, and finally obtain the network signal data sub-stream under each time series segment.

[0141] Furthermore, the network signal flow fluctuation ratio calculation formula is specifically:

[0142]

[0143] In the formula, θ j is the network signal sub-flow flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow size at time t in the jth time segment, is the average flow rate of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time accumulation effect adjustment coefficient, ζ j (t) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio.

[0144] The present invention obtains a network signal flow fluctuation ratio calculation formula by using a specific mathematical model and after verification, which is used to perform quantitative calculation of the network signal ratio for the network signal data substream under the corresponding time segment. The network signal flow fluctuation ratio calculation formula quantifies the volatility of the network signal flow, thereby accurately reflecting the degree of fluctuation of the flow in a certain time segment. By calculating the difference of the flow at different time points, the instantaneous fluctuation of the network signal can be captured, and by comparing this fluctuation measurement with the overall accumulation of the flow (i.e., the integral of the flow), the fluctuation intensity of the time segment can be effectively measured. This volatility analysis can effectively distinguish between normal and abnormal flow, especially for the scenario where the network signal undergoes sudden changes in a short period of time or tends to be stable for a long time. In this formula, by considering the parameter (i.e., flow access frequency) and combining the access frequency with flow fluctuation, the judgment basis of flow fluctuation is further improved. A network signal with a high access frequency may be normal even if the flow itself fluctuates greatly; while a signal with low frequency access but high fluctuation may represent an abnormal phenomenon. By introducing the consideration of access frequency, the understanding of network behavior is enhanced, and the one-sidedness of judging traffic fluctuations based solely on traffic size is avoided, so that abnormal traffic can be identified more accurately. By introducing the dynamic adjustment of the network environment fluctuation factor, the fluctuation of the network environment state under each time segment is represented. This factor can effectively take into account the impact of the external environment (such as changes in network device performance, network load fluctuations, etc.) on signal traffic. When calculating traffic fluctuations, considering this external environment impact can avoid misjudging the fluctuations caused by environmental changes as abnormal signals. By introducing the time cumulative effect adjustment coefficient, the fluctuations in the current period can be adjusted according to historical data. As time goes by, traffic fluctuations may be affected by factors such as long-term trends and periodic changes. At this time, reasonable time adjustment is required. An effective time adjustment mechanism can smooth the errors caused by short-term fluctuations, making traffic analysis within a long span more stable and reliable. In addition, the introduction of the correction coefficient can provide a global adjustment mechanism for the overall correction of the traffic fluctuation ratio. This coefficient can comprehensively adjust the influence of all factors, including access frequency, environmental fluctuations, etc., to ensure that the final fluctuation ratio conforms to the characteristics of actual network operation. The correction coefficient can flexibly adjust the calculation results, so that the calculation formula can maintain good adaptability in different network environments and reduce inaccurate judgments in special circumstances. In summary, the formula fully considers the network signal sub-flow flow fluctuation ratio θ corresponding to the jth timing segment j , the duration T of the jth time segment j , time variable parameter t, network signal flow size L at time t in the jth time segment j (t), the average flow rate of the jth time segment The corresponding network signal traffic access frequency f under the jth time segmentj , time cumulative effect adjustment coefficient ε, network environment fluctuation factor ζ at time t in the jth time segment j (t), the correction coefficient ξ of the network signal sub-flow flow fluctuation ratio, according to the network signal sub-flow flow fluctuation ratio θ corresponding to the j-th time segment j The correlation between the above parameters constitutes a functional relationship This formula can realize the quantitative calculation process of the network signal ratio of the network signal data sub-stream under the corresponding timing segmentation. At the same time, by introducing the correction coefficient ξ of the network signal sub-stream flow fluctuation ratio, it can be adjusted according to the error situation occurring in the calculation process, thereby improving the accuracy and applicability of the network signal flow fluctuation ratio calculation formula.

[0145] Furthermore, the abnormal safety warning processing of the network signal abnormal data sub-flow includes:

[0146] Perform time series and spatial distribution analysis on the network signal abnormal data sub-flow to obtain the time series distribution characteristics and spatial distribution characteristics of the network abnormal sub-flow;

[0147] In an embodiment of the present invention, by collecting the previously identified network signal abnormal data substreams and performing time series analysis on them, the goal of time series analysis is to identify the trend of network signal abnormal data substreams changing over time, including data fluctuations, abnormal bursts, etc. The specific operation includes recording the signal strength, frequency, delay, etc. at each time point in the data stream in detail, eliminating the influencing factors other than normal fluctuations through data cleaning and preprocessing, and ensuring the accuracy of the analysis. Time series data analysis algorithms, such as autoregressive integrated moving average (ARIMA) model, seasonal adjustment model, etc., can be used to identify the periodicity, trend and abnormal points in the data. In the spatial distribution analysis, the spatial distribution area of ​​the data substream is first determined, and the network topology structure and data flow direction information are used, and the spatial distribution is drawn and analyzed in combination with the location information of the network equipment, the propagation path of the data stream and the location of the target node. Through spatial data analysis methods such as K-means clustering algorithm, it is analyzed whether the network abnormal substream has a trend of concentrated distribution in space, so as to identify the network attack source or fault area where the abnormal substream exists, and finally obtain the network abnormal substream time series distribution characteristics and network abnormal substream spatial distribution characteristics.

[0148] Preferably, a spatiotemporal correlation mining analysis is performed on the temporal distribution characteristics of the network abnormal sub-flows and the spatial distribution characteristics of the network abnormal sub-flows to obtain a spatiotemporal abnormal correlation pattern between the temporal distribution and the spatial distribution of the network abnormal sub-flows;

[0149] In an embodiment of the present invention, a spatiotemporal correlation analysis is performed by utilizing the temporal distribution characteristics and spatial distribution characteristics obtained by the previous analysis. In this step, the potential correlation pattern between the temporal and spatial data is found by jointly mining the temporal and spatial data. Specifically, a spatiotemporal correlation analysis algorithm, such as spatiotemporal frequent pattern mining (Spatiotemporal Frequent Pattern Mining) or a spatiotemporal graph convolutional network (ST-GCN) method, is used to combine the spatiotemporal attributes of the network signal to reveal the common characteristics of the abnormal sub-flows of the network signal in time and space. The process of spatiotemporal correlation mining involves constructing a spatiotemporal data model, grouping the abnormal data according to time periods (such as every hour, every day, etc.), and segmenting the data according to the relative distance of the spatial position, and then analyzing the relationship between the spatiotemporal data, so as to obtain the spatiotemporal anomaly correlation pattern of the network abnormal sub-flow. For example, the network equipment in a certain area has abnormal fluctuations in a specific time period, or in certain time periods, the network abnormal traffic at multiple spatial locations is consistent, from which the abnormal activities or potential security threats closely related in time and space are identified, and finally the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of the network abnormal sub-flow is obtained.

[0150] Preferably, based on the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of the network abnormal sub-flow, an abnormal threat assessment analysis is performed on the network signal abnormal data sub-flow to obtain the abnormal threat level of the network signal abnormal sub-flow;

[0151] In an embodiment of the present invention, a threat assessment is performed on the corresponding network abnormal sub-flow based on the spatiotemporal association pattern. In this step, a threat assessment model for a network signal abnormal sub-flow is first established through the aforementioned spatiotemporal abnormal association pattern. The model can use a machine learning algorithm, such as a support vector machine (SVM) or a random forest. By inputting temporal features and spatial features (for example, the time frequency fluctuation of the signal, the flow change in the area, etc.), the threat level of each network abnormal sub-flow is calculated. In the process of threat assessment, the features of the abnormal sub-flow need to be standardized so that the model can accurately assess the relative threat level of each abnormality. Specifically, the assessment process includes the following aspects: first, based on the spatiotemporal distribution characteristics of the sub-flow, possible attack modes (such as DDoS attacks, large-scale data leaks, etc.) are identified; second, combined with known threat cases in historical data, the assessment model is trained to predict the threat of newly emerging abnormal sub-flows. The assessment results will divide the abnormal sub-flows into multiple threat levels, such as low, medium, and high, and finally obtain the abnormal threat level of the network signal abnormal sub-flow.

[0152] Preferably, based on the abnormal threat level of the abnormal network signal sub-flow, abnormal safety warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal safety warning signal.

[0153] In an embodiment of the present invention, the threat level of each network signal abnormal sub-flow is confirmed based on the aforementioned threat assessment result. Once the threat level of a sub-flow is assessed as "high", "medium" or "low", the corresponding network signal abnormal data sub-flow will be automatically triggered with a corresponding security warning processing mechanism. Specifically, the warning processing includes the following operations: according to the threat assessment level of the network abnormal sub-flow, a corresponding security warning signal is generated. A sub-flow with a high threat level will trigger an emergency warning to notify network security personnel to take immediate action, while a sub-flow with a medium threat level will issue a reminder warning, requiring security personnel to pay attention, and a sub-flow with a low threat level will respond with a flashing reminder as soon as possible, and finally respond to generate a corresponding network signal abnormality security warning signal.

[0154] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A big data network signal security early warning system, characterized in that: Includes the following modules: The network signal big data streaming transmission module is used to deploy data collection equipment at the entrance and exit of each network device, and use the data collection equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large data set of network signals; perform big data streaming transmission conversion on the large data set of network signals to obtain a large data stream of network signals; The network signal ratio difference detection module is used to perform a statistical analysis of the timing flow size and access frequency of the network signal big data stream to obtain the corresponding network signal flow size and network signal flow access frequency under each timing segment; based on the corresponding network signal flow size and network signal flow access frequency under each timing segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain a network signal normal data substream and a network signal abnormal data substream; The network signal division result security warning module is used to continue iteratively detecting the corresponding network signal big data stream when determining that the network signal is a normal data substream; When it is determined to be a network signal abnormal data sub-flow, abnormal safety warning processing is performed on the network signal abnormal data sub-flow to generate a network signal abnormal safety warning signal; The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

2. The big data network signal security early warning system according to claim 1 is characterized in that: The network signal big data streaming transmission module includes the following functions: By identifying and analyzing the data flow channel flow direction of each network device entrance and exit, the data flow direction of each network device entrance and exit communication channel is obtained; Perform bandwidth and transmission delay requirement analysis on each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth requirement and transmission delay requirement corresponding to each data flow channel at the inlet and outlet of each network device; Perform transmission mode recognition and analysis on the data flow direction of the communication channels at the entrances and exits of each network device to obtain the transmission mode corresponding to each data flow channel at the entrances and exits of each network device; Based on the transmission bandwidth requirements, transmission delay requirements and transmission modes of each data flow channel at the entrance and exit of each network device, the collection equipment selection and deployment optimization analysis is performed on each data flow channel corresponding to the entrance and exit of each network device to generate the collection equipment deployment optimization plan for each network device entrance and exit; According to the optimization plan for the deployment of data acquisition equipment at the entrances and exits of each network device, data acquisition equipment is deployed at the entrances and exits of each network device, and the data acquisition equipment is used to collect network signal data at the entrances and exits of each network device in real time to obtain a large data set of network signals; The network signal big data set is transformed into a big data streaming data to obtain a network signal big data stream.

3. The big data network signal security early warning system according to claim 2 is characterized in that: The step of performing big data streaming conversion on the big data set of network signals includes: The large data set of network signals is divided into time and space dimensions to obtain network signal data subsets in different time and space distribution dimensions; By setting a streaming transmission conversion framework, and based on the streaming transmission conversion framework, performing signal stream concurrent optimization transmission on network signal data subsets in different time and space distribution dimensions, so as to obtain each network signal concurrent transmission substream; Each concurrent transmission sub-stream of the network signal is preferentially streamlined and compressed for transmission to obtain a large network signal data stream.

4. The big data network signal security early warning system according to claim 3 is characterized in that: The method of performing concurrent optimized signal stream transmission on network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes: Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming slicing to obtain independent streaming segments of each network signal; Perform statistical analysis on the signal strength and delay requirements of each network signal stream segment to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment; Based on the network signal strength corresponding to each network signal segment, the network traffic packet loss rate of the corresponding network signal streaming independent segment is calculated to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, the transmission delay analysis of the corresponding network signal streaming independent segment is performed to obtain the network signal transmission delay corresponding to each network signal segment; A network signal stream concurrent transmission channel is established through a streaming conversion framework, and the corresponding network signal stream concurrent transmission channel is optimized based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, so as to generate a signal stream concurrent transmission optimization channel; Based on the signal stream concurrent transmission optimization channel, each network signal stream is independently divided into segments and signal stream concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-stream.

5. The big data network signal security early warning system according to claim 3 is characterized in that: The prioritizing simplification and compressing transmission processing of each concurrent transmission sub-stream of the network signal includes: Analyze the transmission bandwidth occupancy and transmission rate of each network signal concurrent transmission sub-flow to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow; Based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-stream, a transmission priority quantitative calculation is performed on the corresponding network signal concurrent transmission sub-stream using a network signal transmission priority calculation formula to obtain a network transmission priority metric value corresponding to each network signal transmission sub-stream; Performing transmission delay requirement analysis on each network information concurrent transmission sub-flow to obtain the network transmission delay requirement corresponding to each network signal transmission sub-flow; Based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams for judgment. If the corresponding network transmission priority metric value is greater than or equal to the preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream; The network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream are subjected to network signal compression transmission processing to obtain a network signal large data stream.

6. The big data network signal security early warning system according to claim 5 is characterized in that: The network signal transmission priority calculation formula is specifically: Where P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influencing weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value.

7. The big data network signal security early warning system according to claim 1 is characterized in that: The network signal ratio difference detection module includes the following functions: Perform traffic fluctuation time-series segmentation on the network signal big data stream to obtain network signal data sub-streams under each time-series segment; Performing a time series flow statistics analysis on the network signal data sub-flows under each time series segment to obtain the corresponding network signal flow size under each time series segment; Performing a statistical analysis of the time sequence access frequency of the network signal data sub-flows under each time sequence segment to obtain the corresponding network signal flow access frequency under each time sequence segment; Based on the network signal flow size and the network signal flow access frequency corresponding to each time segment, the network signal flow fluctuation ratio calculation formula is used to perform a quantitative calculation of the network signal ratio of the network signal data sub-flow under the corresponding time segment to obtain the corresponding network signal sub-flow flow fluctuation ratio under each time segment; According to the preset network signal traffic fluctuation threshold, the ratio difference detection and division of the network signal sub-stream traffic fluctuation ratio corresponding to each timing segment is performed; if the network signal sub-stream traffic fluctuation ratio is less than the preset network signal traffic fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal normal data sub-stream; if the network signal sub-stream traffic fluctuation ratio is greater than or equal to the preset network signal traffic fluctuation threshold, the network signal data sub-stream under the corresponding timing segment is divided into a network signal abnormal data sub-stream.

8. The big data network signal security early warning system according to claim 7 is characterized in that: The processing of performing flow fluctuation time sequence segmentation on the network signal big data stream includes: Analyze the fluctuation trend of network traffic on the network signal big data stream to obtain the fluctuation trend of network signal traffic in time series; Based on the time series variation trend of network signal traffic fluctuation, the network signal big data stream is subjected to nonlinear distribution drawing of network traffic fluctuation to generate a time series distribution diagram of network signal traffic fluctuation; Perform traffic fluctuation amplitude and fluctuation peak value statistics analysis on the network signal traffic fluctuation time series distribution diagram to obtain the increase and decrease amplitude of the network signal traffic time series fluctuation and the network signal traffic time series fluctuation peak value; Based on the increase and decrease amplitude of the network signal traffic timing fluctuation and the peak value of the network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis of the network signal traffic fluctuation timing distribution diagram is performed to obtain the network signal traffic timing fluctuation point set and the network signal traffic timing fluctuation inflection point set; Based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set, the network signal big data stream is segmented and processed between timing points to obtain the network signal data sub-stream under each timing segment.

9. The big data network signal security early warning system according to claim 7 is characterized in that: The network signal flow fluctuation ratio calculation formula is specifically: In the formula, θ j is the network signal sub-flow flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow size at time t in the jth time segment, is the average flow rate of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time accumulation effect adjustment coefficient, ζ j (t) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio.

10. The big data network signal security early warning system according to claim 1 is characterized in that: The abnormal safety early warning processing of the network signal abnormal data sub-flow includes: Perform time series and spatial distribution analysis on the network signal abnormal data sub-flow to obtain the time series distribution characteristics and spatial distribution characteristics of the network abnormal sub-flow; Perform spatiotemporal correlation mining analysis on the temporal distribution characteristics of network abnormal sub-flows and the spatial distribution characteristics of network abnormal sub-flows to obtain the spatiotemporal abnormal correlation pattern between the temporal distribution and spatial distribution of network abnormal sub-flows; Based on the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of the network anomaly sub-flow, the network signal anomaly data sub-flow is evaluated for anomaly threat, and the anomaly threat level of the network signal anomaly sub-flow is obtained; Based on the abnormal threat level of the network signal abnormal sub-flow, abnormal safety warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal safety warning signal.

Images