A big data network signal security early warning system

Abstract

The present invention relates to the field of network security analysis technology, and in particular to a big data network signal security early warning system. The system includes a network signal big data streaming module, a network signal ratio difference detection module, a network signal division result security early warning module, and a network signal abnormality security response module. The system can obtain a network signal big data stream by deploying data acquisition equipment at the entrance and exit of each network device and performing real-time network signal data acquisition and big data streaming conversion; perform statistical analysis of the time series flow size and access frequency of the network signal big data stream, and simultaneously perform network signal ratio difference detection and division and abnormality security early warning processing to generate a network signal abnormality security early warning signal; upload the network signal abnormality security early warning signal to a cloud platform for abnormality security response processing to generate a network signal abnormality security response protection strategy. The present invention can efficiently monitor network signal anomalies in real time and provide early warnings.

Description

Technical Field

[0001] The present invention relates to the technical field of network security analysis, and in particular to a big data network signal security early warning system. Background Art

[0002] In recent years, network security technologies based on big data analysis and machine learning have been gradually introduced. By real-time monitoring and dynamic analysis of network signal data, they can more flexibly identify abnormal patterns in network traffic and predict potential security risks. By combining big data analysis, machine learning, and deep learning technologies, various types of network signal data can be collected and analyzed in real time, comprehensively monitoring data flows, network topologies, and communication patterns. By building intelligent models of abnormal behavior, these technologies can identify potential network security threats and, through real-time early warning mechanisms, alert administrators immediately, enabling them to take timely protective measures. However, traditional network signal security monitoring technologies mostly rely on traditional intrusion detection systems (IDS) and firewalls. These methods primarily rely on static rules or feature-based detection to defend against known attack patterns and abnormal behavior. However, they are often unable to effectively address new and complex network signal anomalies. Anomalous network signal activity can be hidden within a large volume of normal data streams, making it difficult to detect potential abnormal security threats in real time. Summary of the Invention

[0003] Based on this, it is necessary for the present invention to provide a big data network signal security early warning system to solve at least one of the above technical problems.

[0004] To achieve the above objectives, a big data network signal security early warning system includes the following modules:

[0005] The network signal big data streaming module is used to deploy data acquisition equipment at the entrance and exit of each network device, and use the data acquisition equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large network signal data set; and perform big data streaming conversion on the large network signal data set to obtain a network signal big data stream;

[0006] The network signal ratio difference detection module is used to perform a statistical analysis of the time series traffic size and access frequency of the network signal big data stream to obtain the corresponding network signal traffic size and network signal traffic access frequency under each time series segment; based on the corresponding network signal traffic size and network signal traffic access frequency under each time series segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain normal network signal data sub-streams and abnormal network signal data sub-streams;

[0007] The network signal division result security warning module is used to continue iteratively detecting the corresponding network signal large data stream when it is determined to be a normal network signal data substream; when it is determined to be an abnormal network signal data substream, it performs abnormal security warning processing on the abnormal network signal data substream to generate a network signal abnormal security warning signal;

[0008] The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0009] Furthermore, the network signal big data streaming transmission module includes the following functions:

[0010] By identifying and analyzing the data flow channel flow direction of each network device entrance and exit, the data flow direction of each network device entrance and exit communication channel is obtained;

[0011] Analyze the bandwidth and transmission delay requirements of each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth and transmission delay requirements corresponding to each data flow channel at the inlet and outlet of each network device;

[0012] Perform transmission mode recognition and analysis on the data flow direction of the communication channels at the entrances and exits of each network device to obtain the transmission mode corresponding to each data flow channel at the entrances and exits of each network device;

[0013] Based on the transmission bandwidth requirements, transmission delay requirements, and transmission mode of each data flow channel at the entrance and exit of each network device, the data collection equipment selection and deployment optimization analysis is performed on each data flow channel corresponding to the entrance and exit of each network device to generate an optimization plan for the deployment of data collection equipment at each network device entrance and exit; according to the optimization plan for the deployment of data collection equipment at each network device entrance and exit, data collection equipment is deployed at each network device entrance and exit, and network signal data is collected in real time at each network device entrance and exit using the data collection equipment to obtain a large network signal data set;

[0014] The network signal big data set is transformed into big data streaming to obtain a network signal big data stream.

[0015] Furthermore, the step of converting the network signal data set into a large data stream includes:

[0016] The large dataset of network signals is divided into time and space dimensions to obtain subsets of network signal data in different time and space distribution dimensions;

[0017] By setting a streaming conversion framework, and performing signal stream concurrent optimization transmission on network signal data subsets in different time and space distribution dimensions based on the streaming conversion framework, each network signal concurrent transmission sub-stream is obtained;

[0018] Prioritize streamlining and compression for transmission of each concurrently transmitted sub-stream of network signals to obtain a large network signal data stream.

[0019] Furthermore, the concurrent optimized transmission of signal streams for network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes:

[0020] Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming sharding to obtain independent streaming segments of each network signal;

[0021] Perform statistical analysis on the signal strength and delay requirements of each network signal stream segment to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment;

[0022] Based on the network signal strength corresponding to each network signal segment, the network traffic packet loss rate of the corresponding independent segment of the network signal stream is calculated to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, the transmission delay of the corresponding independent segment of the network signal stream is analyzed to obtain the network signal transmission delay corresponding to each network signal segment;

[0023] Establishing a concurrent transmission channel for network signal streams through a streaming conversion framework, and optimizing the corresponding concurrent transmission channel for network signal streams based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, so as to generate an optimized concurrent transmission channel for signal streams;

[0024] Based on the signal flow concurrent transmission optimization channel, each network signal flow is independently divided into segments and signal flow concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-flow.

[0025] Furthermore, the prioritizing streamlining and compressing transmission of each concurrently transmitted sub-stream of the network signal includes:

[0026] Analyze the transmission bandwidth occupancy and transmission rate of each concurrent transmission sub-flow of the network signal to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow;

[0027] Based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow, a network signal transmission priority calculation formula is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-flow to obtain the network transmission priority metric value corresponding to each network signal transmission sub-flow;

[0028] Analyze the transmission delay requirements of each concurrent network information transmission sub-flow to obtain the network transmission delay requirements corresponding to each network signal transmission sub-flow;

[0029] Based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams for judgment. If the corresponding network transmission priority metric value is greater than or equal to the preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream;

[0030] The network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream are subjected to network signal compression transmission processing to obtain a network signal big data stream.

[0031] Furthermore, the network signal transmission priority calculation formula is specifically as follows:

[0032]

[0033] Where, P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influence weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value.

[0034] Furthermore, the network signal ratio difference detection module includes the following functions:

[0035] Perform traffic fluctuation time series segmentation on the network signal big data stream to obtain the network signal data sub-streams under each time series segment;

[0036] Performing statistical analysis on the network signal data sub-flows in each time segment to obtain the corresponding network signal flow size in each time segment;

[0037] Performing a statistical analysis of the time series access frequency of the network signal data sub-flows under each time series segment to obtain the corresponding network signal traffic access frequency under each time series segment;

[0038] Based on the network signal traffic size and network signal traffic access frequency corresponding to each time segment, the network signal traffic fluctuation ratio calculation formula is used to perform a quantitative calculation of the network signal ratio of the network signal data sub-flow under the corresponding time segment to obtain the corresponding network signal sub-flow traffic fluctuation ratio under each time segment;

[0039] According to the preset network signal traffic fluctuation threshold, the network signal sub-stream traffic fluctuation ratio corresponding to each timing segment is detected and divided into ratio differences. If the network signal sub-stream traffic fluctuation ratio is less than the preset network signal traffic fluctuation threshold, the network signal data sub-stream under its corresponding timing segment is divided into a normal network signal data sub-stream; if the network signal sub-stream traffic fluctuation ratio is greater than or equal to the preset network signal traffic fluctuation threshold, the network signal data sub-stream under its corresponding timing segment is divided into an abnormal network signal data sub-stream.

[0040] Furthermore, the processing of performing flow fluctuation time sequence segmentation on the network signal big data stream includes:

[0041] Analyze the network traffic fluctuation trend of the network signal big data stream to obtain the network signal traffic fluctuation time series trend;

[0042] Based on the time series change trend of network signal traffic fluctuation, the network signal big data stream is subjected to nonlinear distribution drawing of network traffic fluctuation to generate a time series distribution diagram of network signal traffic fluctuation;

[0043] Perform statistical analysis on the traffic fluctuation amplitude and fluctuation peak value of the network signal traffic fluctuation time series distribution diagram to obtain the increase and decrease amplitude of the network signal traffic time series fluctuation and the network signal traffic time series fluctuation peak value;

[0044] Based on the increase and decrease amplitude of network signal traffic timing fluctuation and the peak value of network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis of the network signal traffic timing distribution graph are performed to obtain the network signal traffic timing fluctuation point set and the network signal traffic timing fluctuation inflection point set;

[0045] Based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set, the network signal big data stream is segmented and processed between timing points to obtain the network signal data sub-stream under each timing segment.

[0046] Furthermore, the network signal flow fluctuation ratio calculation formula is specifically as follows:

[0047]

[0048] Where θ j is the network signal sub-flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow rate at time t in the jth time segment, is the average flow of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time cumulative effect adjustment coefficient, ζ j (t) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio.

[0049] Furthermore, the abnormal security warning processing of the network signal abnormal data sub-flow includes:

[0050] Perform time series and spatial distribution analysis on the abnormal network signal data sub-flows to obtain the time series distribution characteristics and spatial distribution characteristics of the abnormal network sub-flows;

[0051] Perform spatiotemporal correlation mining analysis on the temporal distribution characteristics and spatial distribution characteristics of network anomaly sub-flows to obtain the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of network anomaly sub-flows;

[0052] Based on the spatiotemporal anomaly correlation pattern between the temporal and spatial distribution of network anomaly sub-flows, anomaly threat assessment and analysis of network signal anomaly data sub-flows are performed to obtain the anomaly threat level of network signal anomaly sub-flows;

[0053] Based on the abnormal threat level of the network signal abnormal sub-flow, abnormal security warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal security warning signal.

[0054] Beneficial effects of the present invention:

[0055] The big data network signal security warning system proposed by the present invention is generally composed of a network signal big data streaming module, a network signal ratio difference detection module, a network signal division result security warning module, and a network signal abnormality security response module. Compared with the existing technology, the beneficial effect of the present application is that by identifying and analyzing the data flow channel flow direction of each network device entrance and exit, it can accurately determine the communication relationship between each network device and other devices. This process helps network administrators understand the transmission path, direction, and interaction mode of the data flow, and identify the transmission mode of each data flow channel. Data collection equipment is deployed at the entrance and exit of each network device, and the data collection equipment is used to collect network signal data at the entrance and exit of each network device in real time, which can provide a large amount of comprehensive raw data for network performance analysis. This process can monitor the status, traffic, and performance of each device in real time, promptly detect signs of network failure or performance degradation, and provide immediate feedback to network maintenance personnel to help quickly respond to and solve problems. Through comprehensive data collection, network administrators can obtain detailed information on indicators such as bandwidth usage, latency, and packet loss rate, which helps to assess the overall security status of the network and prevent network congestion, equipment failure, or security vulnerabilities in advance. At the same time, by performing big data streaming conversion on large data sets of network signals, real-time and dynamic data streams can be provided for network monitoring and optimization. This process enables network managers to obtain network performance data in real time and perform operations such as traffic analysis, load balancing, and fault detection. Big data streaming processing can transform the traditional batch processing mode into a real-time response mode, improving the timeliness and accuracy of data processing. Especially when processing massive amounts of data, streaming conversion can effectively reduce latency and improve analysis efficiency. The processing of streaming data can help monitor the network status in real time and respond quickly to abnormal traffic, bottlenecks, or attack behaviors, thereby providing strong data support for the implementation of subsequent network security warnings. Secondly, by performing time-series segmentation processing on large network signal data streams, large-scale and mixed network data streams can be effectively decomposed into smaller sub-streams within several time periods. Statistical analysis of the time-series traffic size can clearly grasp the fluctuation of network traffic in each time period and help identify growth trends or sharp changes in network traffic. This statistical analysis of traffic size helps detect high load and overload periods in the network and identify traffic peaks that affect network stability, thereby providing accurate data support for the subsequent anomaly detection process. Statistical analysis of time-series access frequencies also reveals the density of network requests and the usage of various network services. By analyzing the frequency of each time-series segment, network administrators can understand the behavioral patterns behind traffic, such as frequently accessed periods and usage trends of popular applications. Frequency analysis not only reveals normal network access patterns but also helps identify abnormal traffic patterns.By performing network signal ratio difference detection and division on the corresponding network signal big data stream based on the corresponding network signal traffic size and network signal traffic access frequency in each time segment, the network signal data stream can be divided into "normal" and "abnormal" categories according to the network traffic stability and volatility, thereby achieving precise management of network traffic and anomaly detection. The threshold setting can effectively distinguish normal fluctuations from abnormal fluctuations, avoid misjudgment and missed judgment, and thus ensure the accuracy of network performance monitoring. The normal sub-stream represents the traffic fluctuation within the stable and expected range in the network, while the abnormal sub-stream indicates potential network problems, such as burst traffic, peak traffic or malicious attacks. By dividing the abnormal data sub-stream, network failures or potential attack behaviors can be quickly identified, and corresponding countermeasures can be taken to reduce the impact of network failures on user experience. It can effectively deal with new and complex network signal abnormal behaviors, further improving the reliability and security of network signals. Then, for normal network signal sub-flows, iterative detection continues, while for those data sub-flows determined to be abnormal, security warning processing is immediately performed. The greatest benefit of this stage is that it can achieve rapid response and make timely responses to abnormal traffic caused by malicious attacks or system failures. For example, if a DDoS attack, virus transmission, or other types of abnormal traffic occurs in the network, timely detection and warning of abnormal data can buy valuable response time for the security team and avoid serious impact on network services. The generated abnormal security warning signal can not only detect potential abnormal security threats in real time, but also improve the response efficiency of network managers and reduce the time delay of human intervention. Finally, by processing abnormal security warning signals on the cloud platform and generating network signal abnormal security response protection strategies, the cloud platform, as a centralized data processing and response center, can unify and analyze the data collected from various network devices. Centralized processing of warning signals on the cloud platform not only improves the overall monitoring capability of the network, but also automatically generates corresponding protection strategies based on the big data analysis results. In this way, emergency protection measures such as traffic cleaning, traffic limiting, and intrusion prevention can be initiated in real time, effectively preventing malicious traffic from further affecting the network system, thereby ensuring the continuous and stable operation of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0057] Figure 1 This is a module diagram of the big data network signal security early warning system of the present invention;

[0058] Figure 2 for Figure 1Schematic diagram of the functional flow of the network signal big data streaming module;

[0059] Figure 3 for Figure 1 Schematic diagram of the functional flow of the network signal ratio difference detection module. DETAILED DESCRIPTION

[0060] The following is a clear and complete description of the technical system of the present invention in conjunction with the accompanying drawings. It is obvious that the embodiments described are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present invention.

[0061] To achieve this, please refer to Figures 1 to 3 The present invention provides a big data network signal security early warning system, which includes the following modules:

[0062] The network signal big data streaming module is used to deploy data acquisition equipment at the entrance and exit of each network device, and use the data acquisition equipment to collect network signal data at the entrance and exit of each network device in real time to obtain a large network signal data set; and perform big data streaming conversion on the large network signal data set to obtain a network signal big data stream;

[0063] The network signal ratio difference detection module is used to perform a statistical analysis of the time series traffic size and access frequency of the network signal big data stream to obtain the corresponding network signal traffic size and network signal traffic access frequency under each time series segment; based on the corresponding network signal traffic size and network signal traffic access frequency under each time series segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain normal network signal data sub-streams and abnormal network signal data sub-streams;

[0064] The network signal division result security warning module is used to continue iteratively detecting the corresponding network signal large data stream when it is determined to be a normal network signal data substream; when it is determined to be an abnormal network signal data substream, it performs abnormal security warning processing on the abnormal network signal data substream to generate a network signal abnormal security warning signal;

[0065] The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0066] In the embodiment of the present invention, please refer to Figure 1FIG. 1 is a schematic diagram of the modules of the big data network signal security early warning system of the present invention. In this example, the big data network signal security early warning system includes the following modules:

[0067] S1: Network signal big data streaming module, which is used to deploy data acquisition equipment at the entrance and exit of each network device, and use the data acquisition equipment to collect network signal data at each network device entrance and exit in real time to obtain a large network signal data set; and perform big data streaming conversion on the large network signal data set to obtain a network signal big data stream;

[0068] In an embodiment of the present invention, traffic is monitored and analyzed at the inlet and outlet of each network device to clarify the communication flow direction of each network device. The data traffic of the inlet port and outlet port of each network device is divided into different data flow channels during the analysis process. After the data flow channel flow direction is identified, the bandwidth and delay requirements of each data flow channel are analyzed. For each identified data flow channel, a network performance testing tool (such as iPerf, Wireshark) is used to measure the real-time bandwidth and transmission delay of the traffic. The transmission mode of the data flow direction of each communication channel is analyzed to identify the characteristics of the data flow. Different data streams have different transmission modes, such as TCP stream, UDP stream, broadcast, multicast, etc. By adopting streaming data analysis technology and using protocol analysis tools (such as NetFlow, sFlow) to identify the protocol type of the traffic, the transmission mode of each data flow channel of the network device will be recorded in detail. At the same time, based on the above analysis, the selection and deployment optimization analysis of the collection equipment at the entrances and exits of each network device is carried out. The specific operations include analyzing the performance requirements of the collection equipment based on the bandwidth requirements, delay requirements and transmission mode of each data flow channel, selecting network collection equipment with sufficient throughput according to the bandwidth requirements, and ensuring that it supports protocols compatible with the selected transmission mode (such as TCP, UDP, etc.), while considering the deployment location of the equipment to ensure that the collection equipment can cover all key entrances and exits, and using optimization algorithms (such as integer programming) to optimize the deployment of the collection equipment to ensure that the deployment of the equipment meets the requirements of maximizing data collection coverage and minimizing costs, so that data collection equipment is deployed at the entrances and exits of each network device. The collection equipment is connected to the physical port of the network device through a network interface (such as a network card, optical fiber interface, etc.) to collect the passing data traffic and signals in real time. The collection process is performed by a high-precision collection card (such as Packet Shark, TAP device) to ensure that data is not lost and the collection accuracy is high. The data is transmitted to the centralized processing platform through a unified data collection system. These data will constitute a large-scale network signal data set, thereby obtaining a large network signal data set. Then, by using big data processing frameworks (such as Apache Kafka and Apache Flink) to stream and transform the collected network signal large data sets, the collected raw data is pushed to the data processing platform through transmission protocols (such as HTTP, MQTT, gRPC, etc.), and the stream processing engine is used to process the data in real time, converting it into a data stream that meets the needs of subsequent analysis and monitoring. In this process, the data stream undergoes operations such as cleaning, formatting, and classification to obtain a structured network signal big data stream. This big data stream will be further used for real-time security detection, abnormal traffic identification, attack warning and other functions to realize the automation and refinement of network security management, and finally obtain a network signal big data stream.

[0069] S2: A network signal ratio difference detection module is used to perform a statistical analysis of the time series traffic size and access frequency of the network signal big data stream to obtain the corresponding network signal traffic size and network signal traffic access frequency under each time series segment; based on the corresponding network signal traffic size and network signal traffic access frequency under each time series segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain normal network signal data sub-streams and abnormal network signal data sub-streams;

[0070] In an embodiment of the present invention, a large network signal data stream is collected by a data acquisition device and converted into a stream, including but not limited to source address, destination address, timestamp, transmission protocol type, packet size, data transmission rate, network transmission flow and other information, and a time sliding window algorithm or a segmentation method based on event triggering is used to segment the network signal stream according to a preset time interval (i.e., between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point). The original network signal stream is divided into multiple time segments, each of which corresponds to a certain time segment. The network signal data sub-flow within the time range is obtained, and the traffic size of the network signal data sub-flow in each time segment obtained by the previous division is counted, and the data transmission volume in each time segment is analyzed. Specifically, according to the size of each data packet, the total data transmission volume in the time segment is calculated to obtain the traffic size of the time segment. For example, if a time segment contains 10 data packets, and the size of each data packet is 100KB, 200KB, 50KB, etc., then the traffic size of the time segment is the sum of all data packet sizes, thereby obtaining the corresponding network signal traffic size under each time segment. At the same time, the access frequency is counted based on the network signal data sub-flow in each time segment. The definition of access frequency refers to the number of times each source IP or target IP accesses the network signal in a certain time segment. By analyzing the source IP and target IP information of each data packet in the time segment, the number of visits to each IP address in each time segment is counted. For example, if source IP address A visits 3 times and source IP address B visits 5 times in a certain time segment, the access frequency of the time segment is A: 3 / 8, B: 5 / 8, thereby obtaining the corresponding network signal traffic access frequency under each time segment.Then, by combining the corresponding network signal traffic size, network signal traffic access frequency and related parameters under the time segment, a suitable traffic fluctuation ratio calculation formula is formed to perform network signal ratio quantitative calculation on the network signal data sub-flow under the corresponding time segment. By quantitatively calculating the ratio, the abnormal fluctuation of the network signal in the time segment can be quantitatively evaluated. If the traffic fluctuation ratio is large, it means that the network signal of the time segment has abnormal fluctuation; otherwise, it can be considered that the network signal of the time segment is in a normal state, and a network signal traffic is set according to the historical data or the normal traffic characteristics of the network environment. Fluctuation threshold, which represents the range of normal fluctuations. For example, if in a specific network environment, the network signal flow fluctuation usually does not exceed a certain percentage value (such as 20%), then this percentage value is the threshold. Then, by comparing the relationship between the network signal sub-flow flow fluctuation ratio of each time segment and the preset threshold, if the fluctuation ratio of a certain time segment is less than the threshold, it is judged that the network signal in the time segment is a normal fluctuation and is divided into a network signal normal data sub-flow; if the fluctuation ratio is greater than or equal to the threshold, the network signal in the time segment is considered to be an abnormal fluctuation and is divided into a network signal abnormal data sub-flow.

[0071] S3: A network signal segmentation result security warning module is used to, when a normal network signal data substream is determined, continue to iteratively detect the corresponding network signal large data stream; when a network signal data substream is determined to be abnormal, perform abnormal security warning processing on the abnormal network signal data substream to generate a network signal abnormal security warning signal;

[0072] In an embodiment of the present invention, if a sub-stream is determined to be a normal network signal data sub-stream, the data stream will continue to be iteratively monitored and further time series detection will be performed. The normal traffic and access frequency of each time series segment should meet the pre-defined threshold range. If the abnormal detection result shows that the traffic or access pattern has changed, the data stream needs to be further detected in real time. In this process, a certain dynamic threshold can be set, and the detection standard can be gradually adjusted according to historical data and traffic characteristics; and if a data sub-stream is determined to be an abnormal network signal data sub-stream, the corresponding abnormal security warning mechanism will be triggered. At this time, the abnormal data sub-stream will be isolated separately and analyzed and processed in real time. By setting a multi-dimensional anomaly detection model, such as a classification algorithm based on machine learning or a rule-based discrimination method, various security risks such as traffic anomalies and protocol non-compliance can be accurately identified. When the data stream is identified as abnormal, a security warning signal will be generated in response to prompt relevant personnel or automatically trigger a security response mechanism, and finally a network signal abnormality security warning signal will be generated.

[0073] S4: Network signal abnormality security response module, used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

[0074] In an embodiment of the present invention, when the abnormal data substream is identified and an abnormal security warning signal is generated, the warning signal will be uploaded to the cloud platform, which is responsible for further processing and responding to these abnormal security warning signals. Specifically, after the network signal abnormal security warning signal is uploaded to the cloud platform, the cloud platform will automatically process it according to the preset security response mechanism. The security response mechanism includes but is not limited to traffic speed limiting, source IP blocking, protocol anomaly detection and protection strategy, etc. The cloud platform generates personalized protection strategies by utilizing artificial intelligence algorithms or rule engines, combined with historical attack patterns, network topology and real-time network status. These protection strategies may include modifying firewall strategies, adjusting router traffic distribution, or starting distributed denial of service (DDoS) attack protection modules, etc. The processed response strategies will be promptly fed back to the network devices to achieve real-time protection of abnormal network signals. The generated protection strategies will be sent to each network device through encrypted transmission to ensure network security and prevent potential attacks or security threats from damaging the system, and finally generate network signal abnormal security response protection strategies.

[0075] Further, as an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 FIG. 1 is a functional flow diagram of a network signal big data streaming transmission module. In this embodiment, the functions of the network signal big data streaming transmission module include:

[0076] S11: By identifying and analyzing the data flow channel flow direction of each network device inlet and outlet, the data flow direction of each network device inlet and outlet communication channel is obtained;

[0077] In an embodiment of the present invention, traffic is monitored and analyzed at the inlet and outlet of each network device to clarify the communication flow direction of each network device. First, traffic monitoring tools (such as traffic analyzers and network traffic probes) are used to capture real-time data of the physical and logical ports of the device. The data traffic of the inlet and outlet ports of each network device is divided into different data flow channels during the analysis process. The data flow direction of each flow channel is identified through deep packet analysis (DPI) technology. The process also takes into account the network topology to ensure that the identified flow direction is consistent with the actual data flow path. Through this analysis, the directionality of the inlet and outlet traffic of each device can be obtained, and finally the data flow direction of the inlet and outlet communication channels of each network device can be obtained.

[0078] S12: Analyze the bandwidth and transmission delay requirements of each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth requirement and transmission delay requirement corresponding to each data flow channel at the inlet and outlet of each network device;

[0079] In an embodiment of the present invention, after the flow direction of the data flow channel is identified, the bandwidth and latency requirements of each data flow channel are analyzed. For each identified data flow channel, network performance testing tools (such as iPerf and Wireshark) are used to measure the real-time bandwidth and transmission latency of the traffic. This process will stress test each data flow channel, simulate the transmission conditions under different traffic conditions, and calculate the maximum bandwidth requirement and minimum latency requirement of each flow channel. The bandwidth requirement will be estimated based on the actual data traffic and network load forecast, and the transmission latency will be obtained by measuring the delay of the data packet from the source device to the destination device. These analysis results will construct detailed bandwidth and latency requirement data, and ultimately obtain the transmission bandwidth requirement and transmission latency requirement corresponding to each data flow channel at the entrance and exit of each network device.

[0080] S13: Performing transmission mode recognition and analysis on the data flow direction of the communication channels at the inlet and outlet of each network device to obtain the transmission mode corresponding to each data flow channel at the inlet and outlet of each network device;

[0081] In an embodiment of the present invention, the transmission mode of the data flow direction of the communication channel at the inlet and outlet of each network device is analyzed to identify the characteristics of the data flow. Different data flows have different transmission modes, such as TCP flow, UDP flow, broadcast, multicast, etc. By adopting streaming data analysis technology, protocol analysis tools (such as NetFlow, sFlow) are used to identify the protocol type of the traffic. During the analysis process, the different transmission modes of the traffic will be classified, and the transmission mode of each data flow channel of the network device will be recorded in detail. The identification result is crucial for subsequent equipment selection and deployment optimization, because traffic with different transmission modes has different requirements for network resources (such as bandwidth, routing strategy), and ultimately the transmission mode corresponding to each data flow channel at the inlet and outlet of each network device is obtained.

[0082] S14: Based on the transmission bandwidth requirements, transmission delay requirements, and transmission modes of each data flow channel at the entrance and exit of each network device, a collection device selection and deployment optimization analysis is performed for each data flow channel corresponding to the entrance and exit of each network device, so as to generate an optimization plan for the deployment of collection devices at the entrance and exit of each network device; data collection devices are deployed at the entrance and exit of each network device according to the optimization plan for the deployment of collection devices at the entrance and exit of each network device, and network signal data is collected in real time at the entrance and exit of each network device using the data collection devices to obtain a large network signal data set;

[0083] In an embodiment of the present invention, the selection and deployment optimization analysis of the collection equipment at the entrances and exits of each network device is performed based on the above analysis. The specific operations include analyzing the performance requirements of the collection equipment based on the bandwidth requirements, latency requirements and transmission mode of each data flow channel, selecting a network collection device with sufficient throughput capacity according to the bandwidth requirements, and ensuring that it supports a protocol compatible with the selected transmission mode (such as TCP, UDP, etc.). At the same time, the deployment location of the equipment is considered to ensure that the collection equipment can cover all key entrances and exits and support comprehensive collection of network signals. The deployment of the collection equipment is optimized using an optimization algorithm (such as integer programming) to ensure that the deployment of the equipment meets the requirements of maximizing data collection coverage and minimizing costs, thereby generating an optimized deployment plan for the collection equipment at the entrances and exits of each network device. Then, according to the generated collection equipment deployment optimization plan, data collection equipment is deployed at the entrance and exit of each network device. The collection equipment is connected to the physical port of the network device through a network interface (such as a network card, optical fiber interface, etc.) to collect the passing data traffic and signals in real time. The collection process is executed by a high-precision collection card (such as Packet Shark, TAP device) to ensure that data is not lost and the collection accuracy is high. All collected signal data, including flow information, protocol data, delay data, etc., will be transmitted to the centralized processing platform through a unified data collection system. These data will constitute a large-scale network signal data set, and ultimately a large network signal data set will be obtained.

[0084] S15: Perform big data streaming conversion on the network signal big data set to obtain a network signal big data stream.

[0085] In an embodiment of the present invention, the collected network signal big data set is streamed and converted by utilizing a big data processing framework (such as Apache Kafka, Apache Flink), and the collected raw data is pushed to the data processing platform through a transmission protocol (such as HTTP, MQTT, gRPC, etc.), and the stream processing engine is used to process the data in real time and convert it into a data stream that meets the subsequent analysis and monitoring requirements. The data stream undergoes cleaning, formatting, classification and other operations in this process to obtain a structured network signal big data stream. This big data stream will be further used for real-time security detection, abnormal traffic identification, attack warning and other functions to realize the automation and refinement of network security management, and finally obtain a network signal big data stream.

[0086] Furthermore, the step of converting the network signal data set into a large data stream includes:

[0087] The large dataset of network signals is divided into time and space dimensions to obtain subsets of network signal data in different time and space distribution dimensions;

[0088] In an embodiment of the present invention, the acquired large data set of network signals is divided into time and space dimensions. The specific operation is to first analyze the timestamp and geographic location mark of the network signal data, and use a multidimensional data analysis method to process the data set. In the time and space division process, the time dimension is subdivided into multiple time windows, such as by hour, day or week, and the spatial dimension is divided by geographic information. The latitude and longitude data can be used to locate the signal source. In the specific implementation, spatial clustering based on the K-means clustering algorithm is used to group the signals in different areas. In addition, the sliding window technology is used to divide the time domain data. This process ensures that each data subset can reflect the changing characteristics of the network signal within a specific time and space range, so that the subsequent analysis can be carried out in-depth mining and processing for different time and space scenarios, and finally obtain the network signal data subsets under different time and space distribution dimensions.

[0089] Preferably, by setting a streaming conversion framework, and performing signal stream concurrent optimization transmission on subsets of network signal data in different time and space distribution dimensions based on the streaming conversion framework, each network signal concurrent transmission sub-stream is obtained;

[0090] In an embodiment of the present invention, a streaming conversion framework is established to ensure that data can be transmitted and processed in real time through efficient streaming processing. The specific implementation method is as follows: based on a distributed messaging system such as Apache Kafka, a subset of network signal data is converted into streaming data, and the data is transmitted through the consumer and producer mechanism in Kafka. During the streaming transmission process, corresponding priorities and bandwidth control strategies are set for network signal data subsets divided by different time and space, and traffic scheduling algorithms such as dynamic bandwidth allocation and traffic shaping are used for concurrent optimized transmission. The framework automatically adjusts the transmission priority of each data substream based on the time sensitivity and spatial characteristics of the network signal to ensure efficient and loss-free transmission of the network signal. During this process, the network signal traffic is monitored in real time, and a sliding window-based algorithm is used to load balance the transmission of the signal stream to prevent packet loss caused by network congestion or sudden increase in signal volume, thereby achieving optimized concurrent transmission of the signal stream and ultimately obtaining each network signal concurrent transmission substream.

[0091] Preferably, each concurrently transmitted sub-stream of the network signal is preferentially streamlined and compressed for transmission to obtain a large network signal data stream.

[0092] In an embodiment of the present invention, the network signal sub-streams that have been transmitted concurrently are streamlined and compressed. The core purpose of the streamlining process is to remove redundant data that is not transmitted in priority and retain key feature data, thereby reducing the amount of data transmission and improving efficiency. The specific implementation method is to analyze and screen out the corresponding non-priority transmission sub-streams, and remove redundant information that has no significant impact on subsequent analysis. At the same time, compression algorithms such as Huffman coding, LZ77, etc. are used to compress the priority transmission sub-streams and the streamlined non-priority transmission sub-streams to further reduce the size of the data stream. When compressing the data, it is necessary to ensure that the compressed data can still restore sufficient accuracy and signal characteristics to support subsequent security warning analysis. During the compression process, the parameters of the compression algorithm need to be dynamically adjusted according to the network bandwidth situation to make the transmission of the data stream smoother and more efficient, and finally obtain a large network signal data stream.

[0093] Furthermore, the concurrent optimized transmission of signal streams for network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes:

[0094] Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming sharding to obtain independent streaming segments of each network signal;

[0095] In an embodiment of the present invention, a streaming conversion framework is utilized to perform streaming sharding processing on subsets of network signal data in different time and space distribution dimensions. The specific operation is to divide the network signal data set into time and space dimensions, and each network signal data subset will be independently sharded according to different timestamps (such as every second, every minute or more detailed time periods) and spatial distributions (for example, different nodes and regions of the network). This processing method requires the real-time and data integrity of the signal flow, and the size of the shard needs to be reasonably set according to the network bandwidth, data transmission requirements and processing capabilities. Through the algorithm in the streaming framework, each data fragment (i.e., the network signal streaming fragment) is time-stamped and its continuity during the streaming process is ensured to avoid data loss or disorder, ensure that the time and space dimension information of the network signal can be accurately identified, and finally obtain each network signal streaming independent fragment.

[0096] Preferably, a statistical analysis of the signal strength and delay requirements of each network signal stream independent segment is performed to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment;

[0097] In an embodiment of the present invention, a statistical analysis of the signal strength and delay requirements is performed on each independent segment of the network signal stream obtained by the previous division. Specifically, signal strength data is first extracted from each signal segment, and the signal strength value corresponding to each segment is statistically analyzed according to the network transmission standard. These strength data include parameters such as the power and bit error rate of the received signal, which reflect the quality of the network signal. It is also necessary to perform statistics on the delay requirements according to the actual application scenarios of the network (such as data transmission, real-time communication, etc.). The delay requirement usually refers to the maximum tolerable delay time of each data segment, which depends on the quality of the network, the transmission protocol and the actual needs. For example, in a large data stream processing system, the transmission delay of video data is required to not exceed 200ms, while the delay requirement of ordinary data transmission is 500ms. At this time, the delay statistics and signal strength analysis based on the streaming data can provide basic data support for subsequent network optimization, and finally obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment.

[0098] Preferably, based on the network signal strength corresponding to each network signal segment, a network traffic packet loss rate is calculated for the corresponding independent segment of the network signal stream to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, a transmission delay analysis is performed on the corresponding independent segment of the network signal stream to obtain the network signal transmission delay corresponding to each network signal segment;

[0099] In an embodiment of the present invention, network traffic packet loss rate calculation and transmission delay analysis are performed for each network signal segment based on signal strength and transmission delay requirements. First, the corresponding traffic packet loss rate is calculated based on the signal strength of each segment. The packet loss rate calculation is generally based on the ratio between the number of received packets and the product of the number of sent packets and the signal strength. If the signal strength of the segment is weak, the packet loss rate is high; conversely, if the packet loss rate is low, a certain error tolerance can be adopted to accurately calculate the packet loss rate, and dynamic adjustment can be made based on historical data and network environment to obtain the network traffic packet loss rate corresponding to each network signal segment. Then, the transmission delay requirement of each segment is analyzed. The delay analysis includes calculating the delay time experienced by each signal segment from the source node to the destination node in the network. The analysis takes into account the processing time of network devices such as routers and switches, the link transmission speed, and the transmission process of the data packet in the network, and conducts a comprehensive evaluation in combination with the network topology. For segments that do not meet the delay requirements, it is necessary to further optimize the network path or adjust the transmission rate. Finally, the network signal transmission delay corresponding to each network signal segment is obtained.

[0100] Preferably, a network signal stream concurrent transmission channel is established through a streaming conversion framework, and based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, the corresponding network signal stream concurrent transmission channel is optimized to generate a signal stream concurrent transmission optimized channel;

[0101] In an embodiment of the present invention, a streaming conversion framework is used to construct concurrent transmission channels for network signal streams, and these channels are optimized in combination with the corresponding network traffic packet loss rate and network signal transmission delay. The specific operation includes generating multiple concurrent transmission channels based on the streaming conversion framework according to the packet loss rate and transmission delay requirements of the independent segments of the network signal stream obtained in the previous stage. These channels are finely configured and scheduled according to the signal strength, delay and packet loss rate to ensure the transmission quality and efficiency of the network signal. For example, in a large-scale data stream system, the concurrent channels of the network signal stream can be optimized by dynamic traffic scheduling algorithms. These algorithms take into account the real-time status of each segment in the network, the usage of bandwidth resources and the current load level, so as to select the most appropriate transmission path and concurrent channel. For signal streams with high packet loss rate or large delay, they can be guided to a more stable path in the network, and the traffic distribution can be dynamically adjusted to reduce network congestion, thereby ensuring that the delay and quality requirements of each signal segment are met, and finally generating an optimized channel for concurrent transmission of the signal stream.

[0102] Preferably, based on the signal stream concurrent transmission optimization channel, each network signal stream is independently divided into segments and signal stream concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-stream.

[0103] In an embodiment of the present invention, each network signal stream independent segment is concurrently optimized and transmitted based on the optimized signal stream concurrent transmission channel, so that each signal stream can be transmitted on multiple parallel paths through the optimized concurrent transmission channel, thereby improving the transmission rate and reducing the transmission delay. Specifically, each network signal stream independent segment will be concurrently scheduled for transmission to ensure that these segments are processed smoothly and efficiently in the concurrent transmission channel, and dynamically adjusted according to the characteristics and traffic load of each segment to ensure full utilization of the network bandwidth, and adjust the traffic distribution according to the real-time status of each channel. For example, in a big data real-time monitoring scenario, the traffic of some network signals is large, and more concurrent channels are required to carry their transmission needs. In this way, the concurrent transmission strategy of the signal will be dynamically adjusted, and high-traffic signal segments will be scheduled to more transmission channels, thereby improving data transmission efficiency and meeting the real-time requirements of the network, and finally obtaining each network signal concurrent transmission sub-stream.

[0104] Furthermore, the prioritizing streamlining and compressing transmission of each concurrently transmitted sub-stream of the network signal includes:

[0105] Analyze the transmission bandwidth occupancy and transmission rate of each concurrent transmission sub-flow of the network signal to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow;

[0106] In the embodiment of the present invention, a detailed analysis of the bandwidth occupancy and transmission rate of each concurrently transmitted network signal sub-flow is performed. In a specific implementation, each signal flow in the network is first monitored in real time using a network monitoring device or a traffic analysis tool (such as Wireshark, NetFlow, or sFlow). These tools can accurately record the number of data packets, transmission frequency, and network bandwidth usage of each sub-flow. Based on this data, the bandwidth occupancy of each sub-flow is calculated using the formula: At the same time, the transmission rate of each sub-stream is calculated. The transmission rate is usually determined by sampling the transmission time and data volume of each sub-stream. The formula is: Finally, the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow are obtained.

[0107] Preferably, based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-stream, a network signal transmission priority calculation formula is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-streams to obtain a network transmission priority metric value corresponding to each network signal transmission sub-stream;

[0108] In an embodiment of the present invention, a suitable network signal transmission priority calculation formula is formed by combining the transmission bandwidth occupancy, transmission rate, network transmission delay, network transmission priority impact weight coefficient, network transmission time, integral time variable parameter, network transmission condition impact factor, network transmission packet loss impact weight coefficient, network transmission packet loss rate and related parameters corresponding to each network signal transmission sub-stream to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-stream, so as to quantitatively calculate the priority situation corresponding to the corresponding transmission sub-stream, and finally obtain the network transmission priority measurement value corresponding to each network signal transmission sub-stream.

[0109] Preferably, a transmission delay requirement analysis is performed on each network information concurrent transmission sub-flow to obtain a network transmission delay requirement corresponding to each network signal transmission sub-flow;

[0110] In an embodiment of the present invention, the latency requirements of each network signal transmission sub-flow are analyzed. The latency requirement analysis typically determines the latency requirement of each sub-flow based on its application scenario (e.g., video streaming, voice streaming, data transmission, etc.). Each transmission sub-flow is classified according to preset latency requirement rules: for applications with high real-time requirements (e.g., VoIP, real-time video conferencing, etc.), the latency requirement is low; while for general data transmission applications (e.g., file download, backup, etc.), the latency requirement is high. In this way, the actual transmission delay of each sub-flow can be measured using network performance testing tools (e.g., iperf, Ping, etc.) and compared with the preset latency standard. If the latency requirement of a sub-flow is low (below a certain threshold), the sub-flow is labeled as a low-latency-requirement sub-flow. If the latency requirement is high, it is classified as a high-latency-requirement sub-flow. Ultimately, the network transmission latency requirements corresponding to each network signal transmission sub-flow are determined.

[0111] Preferably, based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams and judged. If the corresponding network transmission priority metric value is greater than or equal to a preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream.

[0112] In an embodiment of the present invention, data stream division is determined for each concurrently transmitted substream of a network signal based on a previously calculated transmission priority metric and a latency requirement analysis result. Specifically, the priority metric values of all substreams are first compared with a preset network transmission priority threshold. If the priority metric value of a substream is greater than or equal to the threshold, it indicates that the substream has a higher priority in the network. Next, the latency requirement of the substream is checked. If the substream has a low latency requirement and a high priority metric value, optimized transmission can be performed. Optimized transmission can be achieved by adjusting routing strategies, selecting low-latency network paths, increasing bandwidth, and other means to ensure that the substream is transmitted with the lowest latency, thereby obtaining a network signal optimized transmission substream. If the priority metric value of the substream is low or the latency requirement is high, it should be streamlined. Streamlining includes reducing data traffic or adopting a delay-tolerant transmission method to avoid interference from high-priority data streams. The streamlined substream becomes a network signal streamlined transmission substream.

[0113] Preferably, the network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream are subjected to network signal compression transmission processing to obtain a network signal large data stream.

[0114] In an embodiment of the present invention, the optimized transmission sub-stream of the network signal and the simplified transmission sub-stream of the network signal obtained by the previous judgment are compressed and transmitted. For the optimized transmission sub-stream, an efficient compression algorithm (such as Huffman coding, LZ77 compression algorithm, etc.) can be used to reduce the bandwidth occupied during the transmission process by removing redundant data and optimizing the encoding method. The data compression processing of the optimized transmission sub-stream will focus on improving transmission efficiency and reducing delay. For the simplified transmission sub-stream, a more powerful compression method can be adopted, such as directly discarding non-critical data parts, or compressing with an algorithm with a higher compression ratio to further reduce bandwidth occupancy and transmission delay. This process can be implemented through a dedicated network signal compressor (such as ZIP, RAR, etc.) or a custom algorithm. After compression processing, the optimized transmission and simplified transmission sub-streams are aggregated into a large data stream. This process integrates multiple small sub-streams into a unified large data stream through a centralized data transmission mechanism, and ultimately obtains a large network signal data stream.

[0115] Furthermore, the network signal transmission priority calculation formula is specifically as follows:

[0116]

[0117] Where, P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influence weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value.

[0118] The present invention obtains a network signal transmission priority calculation formula by using a specific mathematical model and verifying it, which is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-streams. The network signal transmission priority calculation formula can ensure that the ratio of the transmission rate and bandwidth occupancy of the network signal flow is effectively balanced by introducing the two parameters of transmission bandwidth occupancy and transmission rate. The flow with high rate and small bandwidth occupancy will have a higher priority, ensuring the reasonable allocation of resources and avoiding the bottleneck phenomenon caused by insufficient bandwidth. Delay is a key factor in measuring network transmission delay. The existence of the delay term in the formula reduces the priority of the transmission sub-stream with higher network delay, and the high-delay flow will be penalized to avoid occupying too many transmission resources when the delay is high, thereby optimizing the overall network delay performance. By reducing the priority of the high-delay flow, the transmission efficiency of the low-delay flow can be optimized, and the network experience of applications with high real-time requirements (such as video conferencing, online games, etc.) can be improved. By combining transmission duration with a time integration factor, the sensitivity of signal transmission conditions to changes is modeled. The network condition impact factor reflects fluctuations in network quality over different time periods, such as latency and packet loss rate. This helps dynamically adjust transmission priorities based on real-time network conditions. For long-term transmissions over low-quality or unstable networks (such as large data uploads), this formula helps identify and prioritize flows with better conditions or more urgent workloads, avoiding performance degradation over long periods of low-quality data transmission. Packet loss during network transmission typically indicates poor network conditions or excessive load. In this formula, flows with high packet loss rates are penalized and their transmission priority is lowered. By incorporating the packet loss impact factor, data flows with less packet loss and better quality are prioritized during periods of network congestion or instability, thereby reducing retransmissions and delays. Furthermore, a correction factor is introduced to adjust the baseline level of the transmission priority metric. This serves as a global parameter to adjust the flexibility and accuracy of the entire priority model. For example, when the network conditions are particularly complex or there are special needs, the correction coefficient can be adjusted to ensure that the priority of a specific flow can adapt to the actual needs. This flexibility is very important for certain specific application scenarios, such as retaining the priority of key flows under high load. This formula can quantify the priority of each transmission sub-flow by comprehensively considering multiple factors (bandwidth occupancy, rate, delay, duration, network conditions, packet loss, etc.). According to the priority calculated by the formula, appropriate network resources can be allocated to each flow to ensure efficient operation of the network. In summary, this formula fully considers the network transmission priority metric P corresponding to the i-th network signal transmission sub-flow i , the transmission bandwidth occupied by the i-th network signal transmission sub-flow corresponding to B i , the transmission rate S corresponding to the i-th network signal transmission sub-flow i , the network transmission delay L corresponding to the i-th network signal transmission sub-flow i, network transmission priority influence weight coefficient α, the network transmission time T corresponding to the i-th network signal transmission sub-flow i , integral time variable parameter τ, network transmission condition influencing factor β(τ) at time τ, network transmission packet loss influence weight coefficient γ, network transmission packet loss rate R corresponding to the i-th network signal transmission sub-flow i , the correction coefficient η of the network transmission priority metric value, according to the network transmission priority metric value P corresponding to the i-th network signal transmission sub-flow i The mutual correlation between the above parameters constitutes a functional relationship This formula can realize the quantitative calculation process of the transmission priority of the corresponding network signal concurrent transmission sub-streams. At the same time, the introduction of the correction coefficient η of the network transmission priority metric value can be adjusted according to the error situation occurring in the calculation process, thereby improving the accuracy and applicability of the network signal transmission priority calculation formula.

[0119] Further, as an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Schematic diagram of the functional flow of the network signal ratio difference detection module in FIG. 1 . In this embodiment, the functions of the network signal ratio difference detection module include:

[0120] S21: Perform traffic fluctuation time sequence segmentation on the network signal data stream to obtain network signal data sub-streams under each time sequence segment;

[0121] In an embodiment of the present invention, a large network signal data stream is collected by a data acquisition device and converted into a streaming format, including but not limited to source address, destination address, timestamp, transmission protocol type, packet size, data transmission rate, network transmission traffic and other information, and a time sliding window algorithm or an event-triggered segmentation method is used to segment the network signal stream according to preset time intervals (i.e., time segments such as between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point). The original network signal stream is divided into multiple time segments, each of which corresponds to a network signal data sub-stream within a certain time range. The time interval in the segmentation process can be flexibly adjusted according to the characteristics of the network signal, monitoring requirements or preset strategies, and finally the network signal data sub-stream under each time segment is obtained.

[0122] S22: performing a time series flow statistics analysis on the network signal data sub-flows in each time series segment to obtain the corresponding network signal flow size in each time series segment;

[0123] In an embodiment of the present invention, traffic size statistics are performed on the network signal data sub-flows in each time segment obtained by the previous division, and the data transmission volume in each time segment is analyzed. Specifically, based on the size of each data packet, the total amount of data transmission in the time segment is calculated to obtain the traffic size of the time segment. For example, if a time segment contains 10 data packets, and the size of each data packet is 100KB, 200KB, 50KB, etc., then the traffic size of the time segment is the sum of the sizes of all data packets. The traffic statistics methods used in this step include the accumulation method, the average method, etc., and the traffic size calculation can be performed using a traffic statistics tool or a custom script to record the traffic size of each time segment, and finally the corresponding network signal traffic size under each time segment is obtained.

[0124] S23: performing a time sequence access frequency statistical analysis on the network signal data sub-flows under each time sequence segment to obtain the corresponding network signal traffic access frequency under each time sequence segment;

[0125] In an embodiment of the present invention, the access frequency is counted based on the network signal data sub-flow in each time segment. The definition of access frequency refers to the number of times each source IP or target IP accesses the network signal in a certain time segment. By analyzing the source IP and target IP information of each data packet in the time segment, the number of times each IP address accesses in each time segment is counted. For example, if source IP address A accesses 3 times and source IP address B accesses 5 times in a certain time segment, the access frequency of the time segment is A: 3 / 8 and B: 5 / 8. In order to effectively capture the characteristics of frequent access, the access frequency statistical data can be normalized to reduce the impact of data fluctuations on the results, ensure that the statistical results can accurately reflect the trend of the access frequency, and finally obtain the corresponding network signal traffic access frequency under each time segment.

[0126] S24: Based on the network signal traffic size and the network signal traffic access frequency corresponding to each time segment, a network signal traffic fluctuation ratio calculation formula is used to perform a network signal ratio quantitative calculation on the network signal data sub-flow under the corresponding time segment to obtain the network signal sub-flow traffic fluctuation ratio corresponding to each time segment;

[0127] In an embodiment of the present invention, a suitable network signal traffic fluctuation ratio calculation formula is formed by combining the corresponding duration, time variable parameters, network signal traffic size, average traffic, network signal traffic access frequency, time cumulative effect adjustment coefficient, network environment fluctuation factor and related parameters under the timing segment to perform network signal ratio quantitative calculation on the network signal data sub-flow under the corresponding timing segment. By quantitatively calculating the ratio, the abnormal fluctuation of the network signal in the timing segment can be quantitatively evaluated. If the traffic fluctuation ratio is large, it means that the network signal of the timing segment has abnormal fluctuation; otherwise, it can be considered that the network signal of the timing segment is in a normal state, and finally the corresponding network signal sub-flow traffic fluctuation ratio under each timing segment is obtained.

[0128] S25: Perform ratio difference detection and division on the network signal sub-flow flow fluctuation ratios corresponding to each timing segment according to the preset network signal flow fluctuation threshold. If the network signal sub-flow flow fluctuation ratio is less than the preset network signal flow fluctuation threshold, the network signal data sub-flow under the corresponding timing segment is divided into a network signal normal data sub-flow; if the network signal sub-flow flow fluctuation ratio is greater than or equal to the preset network signal flow fluctuation threshold, the network signal data sub-flow under the corresponding timing segment is divided into a network signal abnormal data sub-flow.

[0129] In an embodiment of the present invention, a network signal flow fluctuation threshold is set based on historical data or the normal flow characteristics of the network environment. The threshold represents the range of normal fluctuations. For example, if in a specific network environment, the network signal flow fluctuation usually does not exceed a certain percentage value (such as 20%), then this percentage value is the threshold. Then, by comparing the relationship between the network signal sub-flow flow fluctuation ratio of each time segment and the preset threshold, if the fluctuation ratio of a certain time segment is less than the threshold, the network signal in the time segment is judged to be a normal fluctuation and is divided into a network signal normal data sub-flow; if the fluctuation ratio is greater than or equal to the threshold, the network signal in the time segment is considered to be an abnormal fluctuation and is divided into a network signal abnormal data sub-flow. In this process, an automated tool or a rule-based system can be used to classify and mark the data in real time. The core purpose of this step is to quickly identify abnormal fluctuations in network signals through real-time detection and classification, thereby providing a basis for subsequent security warnings.

[0130] Furthermore, the processing of performing flow fluctuation time sequence segmentation on the network signal big data stream includes:

[0131] Analyze the network traffic fluctuation trend of the network signal big data stream to obtain the network signal traffic fluctuation time series trend;

[0132] In an embodiment of the present invention, a corresponding network signal big data stream is obtained by previously collecting from various devices in the network (such as routers, switches, gateways, etc.) and converting them into streams. These data usually include source address, destination address, timestamp, transmission protocol type, packet size, data transmission rate, network transmission traffic and other information. In order to analyze the fluctuation trend of the network signal traffic, the network signal big data stream is preprocessed and cleaned using a timing analysis technology to remove outliers, duplicate data or noise, and the network traffic time series data is analyzed for fluctuation trends using a timing analysis tool (for example, the Pandas library in Python or the timing data analysis tool in Matlab). The network traffic time series data is segmented using a sliding window technology (such as a window length of 5 seconds or 10 seconds). By calculating the traffic fluctuation amount in each time window, the network signal traffic fluctuation time series trend is finally obtained.

[0133] Preferably, a nonlinear distribution of network traffic fluctuation is plotted on the network signal big data stream based on the time series variation trend of the network signal traffic fluctuation to generate a time series distribution graph of the network signal traffic fluctuation;

[0134] In an embodiment of the present invention, by combining the time-series change trend of network signal traffic fluctuations obtained through previous analysis, a nonlinear statistical analysis method is used to further model and analyze the network traffic fluctuations in the network signal big data stream. The specific method is to use a probability density function (PDF) for modeling, select a suitable nonlinear distribution model (for example, Lognormal distribution or Weibull distribution) to fit the fluctuation characteristics of the network traffic data, and use statistical analysis tools, such as the Scipy library in Python or the statistical toolbox of Matlab, to fit the model parameters through least squares or maximum likelihood estimation methods to generate a time-series distribution diagram of the traffic fluctuations. This distribution diagram shows the degree of fluctuation of the network signal traffic, the frequency of occurrence, and the nonlinear characteristics of the traffic fluctuations, and can intuitively reflect the laws of traffic fluctuations and the trend of abnormal fluctuations, and finally draw a time-series distribution diagram of the network signal traffic fluctuations.

[0135] Preferably, a traffic fluctuation amplitude and a fluctuation peak value are statistically analyzed on the network signal traffic fluctuation time series distribution diagram to obtain the increase and decrease amplitude of the network signal traffic time series fluctuation and the network signal traffic time series fluctuation peak value;

[0136] In an embodiment of the present invention, a statistical analysis of the traffic fluctuation amplitude and fluctuation peak is performed based on the previously drawn network signal traffic timing distribution diagram. Specifically, the amplitude of the network traffic fluctuation in each time window is calculated to obtain the absolute value of the traffic change. Then, the peak part of the traffic change curve is identified to calculate the fluctuation peak, that is, the difference between the maximum and minimum values of the traffic fluctuation. Automated tools (such as the NumPy library in Python for peak detection) can be used to extract the peak points in the data, and the increase or decrease amplitude of the traffic fluctuation is calculated based on the fluctuation amplitude. The increase or decrease amplitude refers to the amount of traffic change between consecutive time points, and the fluctuation peak is the extreme point in the traffic fluctuation. After statistical analysis, the fluctuation amplitude and peak value in each time period can be obtained, and finally the increase or decrease amplitude of the network signal traffic timing fluctuation and the network signal traffic timing fluctuation peak value are obtained.

[0137] Preferably, based on the increase and decrease amplitude of the network signal traffic timing fluctuation and the peak value of the network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis is performed on the network signal traffic timing distribution diagram to obtain a network signal traffic timing fluctuation point set and a network signal traffic timing fluctuation inflection point set;

[0138] In an embodiment of the present invention, based on the network signal traffic fluctuation increase and decrease amplitude and fluctuation peak value obtained by previous analysis, the traffic time series distribution diagram is further mined and analyzed for fluctuation points and inflection points, and an inflection point detection algorithm for time series data (for example, an inflection point detection method based on the second derivative or a CUSUM control chart method) is applied to identify the inflection points of the fluctuation trend. The inflection point is where the traffic change trend changes significantly, representing abnormal events such as network failures, attacks or burst traffic, and traffic fluctuation points are identified by using a specific fluctuation point detection algorithm (such as a threshold-based method, setting a traffic fluctuation threshold, and when the fluctuation amplitude exceeds the threshold, it is regarded as a fluctuation point). Furthermore, the time series data can be classified by cluster analysis or pattern recognition methods (such as K-means clustering or DBSCAN algorithm) to identify obvious fluctuation clusters as a time series fluctuation point set and a fluctuation inflection point set, and finally a network signal traffic time series fluctuation point set and a network signal traffic time series fluctuation inflection point set are obtained.

[0139] Preferably, the network signal big data stream is segmented and cut between timing points based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set to obtain the network signal data sub-stream under each timing segment.

[0140] In an embodiment of the present invention, a time series data segmentation process is performed on a network signal big data stream based on a previously obtained time series fluctuation point set and corresponding points in a time series fluctuation inflection point set. First, for each interval between a time series fluctuation point and an inflection point, a time series segmentation algorithm is used to divide the network signal traffic data into multiple sub-segments. The specific operation is: according to the time series fluctuation point set and the timestamps at each point in the inflection point set, the big data stream is divided into multiple smaller time series data segments, that is, time series segments such as between a fluctuation point and another fluctuation point, between a fluctuation point and an inflection point, and between an inflection point and another inflection point. Each time series segment corresponds to a stable traffic subset. The boundary of the sub-segment is determined by using a time window algorithm (such as a dynamic time window or a fixed time window). Each sub-segment represents a relatively independent network signal traffic state. By analyzing the traffic fluctuation characteristics in each sub-segment, it is helpful to identify abnormal traffic fluctuations or security threats in the network, and finally obtain the network signal data sub-stream under each time series segment.

[0141] Furthermore, the network signal flow fluctuation ratio calculation formula is specifically as follows:

[0142]

[0143] Where θ j is the network signal sub-flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow rate at time t in the jth time segment, is the average flow of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time cumulative effect adjustment coefficient, ζ j (t) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio.

[0144] The present invention obtains a network signal flow fluctuation ratio calculation formula by using a specific mathematical model and verification, which is used to quantitatively calculate the network signal ratio of the network signal data sub-stream under the corresponding time segment. The network signal flow fluctuation ratio calculation formula quantifies the volatility of the network signal flow, thereby accurately reflecting the degree of flow fluctuation within a certain time segment. By calculating the difference in flow at different time points, the instantaneous fluctuation of the network signal can be captured, and by comparing this fluctuation measurement with the overall accumulation of flow (i.e., the integral of flow), the fluctuation intensity of the time segment can be effectively measured. This volatility analysis can effectively distinguish between normal and abnormal flow, especially for scenarios where the network signal undergoes sudden changes in a short period of time or tends to be stable for a long time. In this formula, by considering the parameter (i.e., flow access frequency) and combining the access frequency with flow fluctuation, the judgment basis of flow fluctuation is further improved. A network signal with a high access frequency may be normal even if the flow itself fluctuates greatly; while a signal with low access frequency but high fluctuation may represent an abnormal phenomenon. The inclusion of access frequency enhances understanding of network behavior, avoiding the one-sided nature of traffic fluctuations based solely on traffic volume, leading to more accurate identification of abnormal traffic. By dynamically adjusting the network environment fluctuation factor, representing network environmental fluctuations within each time segment, this factor effectively accounts for the impact of external factors (such as changes in network device performance and network load fluctuations) on signal traffic. By factoring these external environmental influences into traffic fluctuation calculations, fluctuations caused by environmental changes can be avoided from being misidentified as abnormal signals. The introduction of a time-accumulation effect adjustment factor adjusts for fluctuations in the current period based on historical data. Over time, traffic fluctuations may be influenced by long-term trends and cyclical changes, necessitating appropriate time adjustments. An effective time adjustment mechanism can smooth out errors caused by short-term fluctuations, making traffic analysis over long time spans more stable and reliable. Furthermore, the introduction of a correction factor provides a global adjustment mechanism for the overall correction of the traffic fluctuation ratio. This factor comprehensively adjusts the impact of all factors, including access frequency and environmental fluctuations, ensuring that the final fluctuation ratio conforms to the characteristics of actual network operation. The correction coefficient can flexibly adjust the calculation results, so that the calculation formula can maintain good adaptability in different network environments and reduce inaccurate judgments in special cases. In summary, the formula fully considers the network signal sub-flow flow fluctuation ratio θ corresponding to the jth time segment j , the duration T of the jth time segment j , time variable parameter t, network signal flow size L at time t under the jth time segment j (t), the average flow of the jth time segment The corresponding network signal traffic access frequency f under the jth time segmentj , time cumulative effect adjustment coefficient ε, network environment fluctuation factor ζ at time t under the jth time segment j (t), the correction coefficient ξ of the network signal sub-flow flow fluctuation ratio, according to the network signal sub-flow flow fluctuation ratio θ corresponding to the j-th time segment j The mutual correlation between the above parameters constitutes a functional relationship This formula can realize the quantitative calculation process of the network signal ratio of the network signal data sub-stream under the corresponding timing segmentation. At the same time, by introducing the correction coefficient ξ of the network signal sub-stream flow fluctuation ratio, it can be adjusted according to the error situation occurring in the calculation process, thereby improving the accuracy and applicability of the network signal flow fluctuation ratio calculation formula.

[0145] Furthermore, the abnormal security warning processing of the network signal abnormal data sub-flow includes:

[0146] Perform time series and spatial distribution analysis on the abnormal network signal data sub-flows to obtain the time series distribution characteristics and spatial distribution characteristics of the abnormal network sub-flows;

[0147] In an embodiment of the present invention, by collecting the previously identified network signal abnormal data sub-streams and performing time series analysis on them, the goal of time series analysis is to identify the trend of network signal abnormal data sub-streams changing over time, including data fluctuations, abnormal bursts, etc. The specific operation includes recording the signal strength, frequency, delay, etc. at each time point in the data stream in detail. Through data cleaning and preprocessing, factors other than normal fluctuations are eliminated to ensure the accuracy of the analysis. Time series data analysis algorithms, such as the autoregressive integrated moving average (ARIMA) model and the seasonal adjustment model, can be used to identify periodicity, trends, and anomalies in the data. In spatial distribution analysis, the spatial distribution area of the data sub-stream is first determined. Using the network topology and data flow information, combined with the location information of the network equipment, the propagation path of the data flow, and the location of the target node, the spatial distribution is mapped and analyzed. Through spatial data analysis methods such as the K-means clustering algorithm, whether the network abnormal sub-stream has a concentrated distribution trend in space is analyzed, so as to identify the network attack source or fault area where the abnormal sub-stream exists, and finally the time series distribution characteristics and spatial distribution characteristics of the network abnormal sub-stream are obtained.

[0148] Preferably, a spatiotemporal correlation mining analysis is performed on the temporal distribution characteristics of the network abnormal sub-flows and the spatial distribution characteristics of the network abnormal sub-flows to obtain a spatiotemporal abnormal correlation pattern between the temporal distribution and the spatial distribution of the network abnormal sub-flows;

[0149] In an embodiment of the present invention, spatiotemporal correlation analysis is performed by utilizing the temporal distribution characteristics and spatial distribution characteristics obtained from the previous analysis. This step jointly mines the temporal and spatial data to find potential correlation patterns between the two. Specifically, a spatiotemporal correlation analysis algorithm, such as spatiotemporal frequent pattern mining or spatiotemporal graph convolutional network (ST-GCN), is used to combine the spatiotemporal properties of network signals to reveal the common temporal and spatial characteristics of abnormal sub-flows of network signals. The spatiotemporal correlation mining process involves constructing a spatiotemporal data model, grouping abnormal data by time period (such as hourly, daily, etc.), and segmenting the data according to the relative distance of spatial locations. Then, the relationship between the spatiotemporal data is analyzed to obtain the spatiotemporal anomaly correlation patterns of network anomaly sub-flows. For example, if network equipment in a certain area experiences abnormal fluctuations in a specific time period, or if there is consistency in abnormal network traffic at multiple spatial locations within certain time periods, the network anomaly sub-flows can be identified to identify abnormal activities or potential security threats that are closely related in time and space, and ultimately obtain the spatiotemporal anomaly correlation patterns between the temporal distribution and spatial distribution of the network anomaly sub-flows.

[0150] Preferably, an abnormal threat assessment analysis is performed on the network signal abnormal data subflow based on the spatiotemporal abnormal correlation pattern between the temporal distribution and spatial distribution of the network abnormal subflow to obtain the abnormal threat level of the network signal abnormal subflow;

[0151] In an embodiment of the present invention, a threat assessment is performed on the corresponding network abnormal sub-flow based on the spatiotemporal correlation pattern. In this step, a threat assessment model for the network signal abnormal sub-flow is first established based on the aforementioned spatiotemporal abnormal correlation pattern. The model can use a machine learning algorithm, such as a support vector machine (SVM) or a random forest, to calculate the threat level of each network abnormal sub-flow by inputting temporal features and spatial features (for example, the temporal frequency fluctuation of the signal, the traffic change within the area, etc.). During the threat assessment process, the features of the abnormal sub-flow need to be standardized so that the model can accurately assess the relative threat level of each abnormality. Specifically, the assessment process includes the following aspects: first, based on the spatiotemporal distribution characteristics of the sub-flow, possible attack modes (for example, DDoS attacks, large-scale data leaks, etc.) are identified; second, combined with known threat cases in historical data, the assessment model is trained to predict the threat of newly emerging abnormal sub-flows. The assessment results will classify the abnormal sub-flows into multiple threat levels, such as low, medium, and high, and finally the abnormal threat level of the network signal abnormal sub-flow is obtained.

[0152] Preferably, based on the abnormal threat level of the network signal abnormal sub-flow, abnormal security warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal security warning signal.

[0153] In an embodiment of the present invention, the threat level of each network signal abnormality sub-flow is confirmed based on the aforementioned threat assessment results. Once the threat level of a sub-flow is assessed as "high", "medium" or "low", the corresponding security warning processing mechanism will be automatically triggered for the corresponding network signal abnormality data sub-flow. Specifically, the warning processing includes the following operations: according to the threat assessment level of the network abnormality sub-flow, a corresponding security warning signal is generated. A sub-flow with a high threat level will trigger an emergency warning, notifying network security personnel to take immediate action, while a sub-flow with a medium threat level will issue a reminder warning, requiring security personnel to pay attention, and a sub-flow with a low threat level will respond with a flashing reminder as soon as possible, and finally a corresponding network signal abnormality security warning signal is generated in response.

[0154] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A big data network signal security early warning system, characterized in that: Includes the following modules: The network signal big data streaming module is used to deploy data acquisition equipment at the entrance and exit of each network device, and use the data acquisition equipment to collect network signal data at each network device entrance and exit in real time to obtain a large network signal data set; and perform big data streaming conversion on the large network signal data set to obtain a network signal big data stream. Among them, it includes the following functions: By identifying and analyzing the data flow channel flow direction of each network device entrance and exit, the data flow direction of each network device entrance and exit communication channel is obtained; Analyze the bandwidth and transmission delay requirements of each data flow channel corresponding to the inlet and outlet of each network device to obtain the transmission bandwidth and transmission delay requirements corresponding to each data flow channel at the inlet and outlet of each network device; Perform transmission mode recognition and analysis on the data flow direction of the communication channels at the entrances and exits of each network device to obtain the transmission mode corresponding to each data flow channel at the entrances and exits of each network device; Based on the transmission bandwidth requirements, transmission delay requirements, and transmission mode of each data flow channel at the entrance and exit of each network device, the data collection equipment selection and deployment optimization analysis is performed on each data flow channel corresponding to the entrance and exit of each network device to generate an optimization plan for the deployment of data collection equipment at each network device entrance and exit; according to the optimization plan for the deployment of data collection equipment at each network device entrance and exit, data collection equipment is deployed at each network device entrance and exit, and network signal data is collected in real time at each network device entrance and exit using the data collection equipment to obtain a large network signal data set; Perform big data streaming conversion on the network signal big data set to obtain a network signal big data stream; this includes: The large dataset of network signals is divided into time and space dimensions to obtain subsets of network signal data in different time and space distribution dimensions; By setting a streaming conversion framework, and performing signal stream concurrent optimization transmission on network signal data subsets in different time and space distribution dimensions based on the streaming conversion framework, each network signal concurrent transmission sub-stream is obtained; Prioritize streamlining and compression for each concurrently transmitted sub-stream of network signals to obtain a large network signal data stream, including: Analyze the transmission bandwidth occupancy and transmission rate of each concurrent transmission sub-flow of the network signal to obtain the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-flow; Based on the transmission bandwidth occupancy and transmission rate corresponding to each network signal transmission sub-stream, a network signal transmission priority calculation formula is used to perform transmission priority quantitative calculation on the corresponding network signal concurrent transmission sub-stream to obtain a network transmission priority metric value corresponding to each network signal transmission sub-stream; wherein, the network signal transmission priority calculation formula is specifically: Where, P i is the network transmission priority metric corresponding to the i-th network signal transmission sub-flow, B i is the transmission bandwidth occupied by the i-th network signal transmission sub-flow, S i is the transmission rate corresponding to the i-th network signal transmission sub-flow, L i is the network transmission delay corresponding to the i-th network signal transmission sub-flow, α is the network transmission priority impact weight coefficient, T i is the network transmission time corresponding to the ith network signal transmission sub-flow, τ is the integral time variable parameter, β(τ) is the network transmission condition influencing factor at time τ, γ is the network transmission packet loss influence weight coefficient, R i is the network transmission packet loss rate corresponding to the i-th network signal transmission sub-flow, and η is the correction coefficient of the network transmission priority metric value; Analyze the transmission delay requirements of each concurrent network information transmission sub-flow to obtain the network transmission delay requirements corresponding to each network signal transmission sub-flow; Based on the network transmission priority metric value and the network transmission delay requirement corresponding to each network signal transmission sub-stream, the corresponding network signal concurrent transmission sub-stream is divided into data streams for judgment. If the corresponding network transmission priority metric value is greater than or equal to the preset network transmission priority threshold and the network transmission delay requirement is a low delay requirement, the corresponding network signal concurrent transmission sub-stream is optimized for transmission to obtain a network signal optimized transmission sub-stream; otherwise, the corresponding network signal concurrent transmission sub-stream is streamlined to obtain a network signal streamlined transmission sub-stream; Performing network signal compression and transmission processing on the network signal optimized transmission sub-stream and the network signal simplified transmission sub-stream to obtain a network signal big data stream; The network signal ratio difference detection module is used to perform statistical analysis on the time series traffic size and access frequency of the network signal big data stream to obtain the corresponding network signal traffic size and network signal traffic access frequency under each time series segment; based on the corresponding network signal traffic size and network signal traffic access frequency under each time series segment, the corresponding network signal big data stream is subjected to network signal ratio difference detection and division to obtain normal network signal data sub-streams and abnormal network signal data sub-streams; wherein, the following functions are included: Perform traffic fluctuation time series segmentation on the network signal big data stream to obtain the network signal data sub-streams under each time series segment; Performing statistical analysis on the network signal data sub-flows in each time segment to obtain the corresponding network signal flow size in each time segment; Performing a statistical analysis of the time series access frequency of the network signal data sub-flows under each time series segment to obtain the corresponding network signal traffic access frequency under each time series segment; Based on the network signal traffic size and network signal traffic access frequency corresponding to each time segment, a network signal traffic fluctuation ratio calculation formula is used to perform a network signal ratio quantitative calculation on the network signal data sub-stream under the corresponding time segment to obtain the network signal sub-stream traffic fluctuation ratio corresponding to each time segment; wherein, the network signal traffic fluctuation ratio calculation formula is specifically: Where θ j is the network signal sub-flow fluctuation ratio corresponding to the jth time segment, T j is the duration of the jth time segment, t is the time variable parameter, L j (t) is the network signal flow rate at time t in the jth time segment, is the average flow of the jth time segment, f j is the network signal traffic access frequency corresponding to the jth time segment, ε is the time cumulative effect adjustment coefficient, ζ j (i) is the network environment fluctuation factor at time t in the jth time segment, and ξ is the correction coefficient of the network signal sub-flow flow fluctuation ratio; According to the preset network signal traffic fluctuation threshold, the network signal sub-flow traffic fluctuation ratio corresponding to each time segment is detected and divided into ratio differences. If the network signal sub-flow traffic fluctuation ratio is less than the preset network signal traffic fluctuation threshold, the network signal data sub-flow under the corresponding time segment is divided into a normal network signal data sub-flow; if the network signal sub-flow traffic fluctuation ratio is greater than or equal to the preset network signal traffic fluctuation threshold, the network signal data sub-flow under the corresponding time segment is divided into an abnormal network signal data sub-flow; The network signal division result security warning module is used to continue iteratively detecting the corresponding network signal large data stream when it is determined to be a normal network signal data substream; when it is determined to be an abnormal network signal data substream, it performs abnormal security warning processing on the abnormal network signal data substream to generate a network signal abnormal security warning signal; The network signal abnormality security response module is used to upload the network signal abnormality security warning signal to the cloud platform for abnormal security response processing to generate a network signal abnormality security response protection strategy.

2. The big data network signal security early warning system according to claim 1 is characterized in that: The method of performing concurrent optimized signal stream transmission on network signal data subsets in different time and space distribution dimensions based on the streaming transmission conversion framework includes: Based on the streaming conversion framework, the network signal data subsets in different time and space distribution dimensions are processed by streaming sharding to obtain independent streaming segments of each network signal; Perform statistical analysis on the signal strength and delay requirements of each network signal stream segment to obtain the network signal strength and network transmission delay requirements corresponding to each network signal segment; Based on the network signal strength corresponding to each network signal segment, the network traffic packet loss rate of the corresponding independent segment of the network signal stream is calculated to obtain the network traffic packet loss rate corresponding to each network signal segment; based on the network transmission delay requirement corresponding to each network signal segment, the transmission delay of the corresponding independent segment of the network signal stream is analyzed to obtain the network signal transmission delay corresponding to each network signal segment; Establishing a concurrent transmission channel for network signal streams through a streaming conversion framework, and optimizing the corresponding concurrent transmission channel for network signal streams based on the network traffic packet loss rate and network signal transmission delay corresponding to each network signal segment, so as to generate an optimized concurrent transmission channel for signal streams; Based on the signal flow concurrent transmission optimization channel, each network signal flow is independently divided into segments and signal flow concurrent optimization transmission is performed to obtain each network signal concurrent transmission sub-flow.

3. The big data network signal security early warning system according to claim 1 is characterized in that: The processing of performing traffic fluctuation time sequence segmentation on the network signal big data stream includes: Analyze the network traffic fluctuation trend of the network signal big data stream to obtain the network signal traffic fluctuation time series trend; Based on the time series change trend of network signal traffic fluctuation, the network signal big data stream is subjected to nonlinear distribution drawing of network traffic fluctuation to generate a time series distribution diagram of network signal traffic fluctuation; Perform statistical analysis on the traffic fluctuation amplitude and fluctuation peak value of the network signal traffic fluctuation time series distribution diagram to obtain the increase and decrease amplitude of the network signal traffic timing fluctuation and the network signal traffic timing fluctuation peak value; Based on the increase and decrease amplitude of network signal traffic timing fluctuation and the peak value of network signal traffic timing fluctuation, the timing fluctuation point and inflection point mining analysis of the network signal traffic timing distribution graph are performed to obtain the network signal traffic timing fluctuation point set and the network signal traffic timing fluctuation inflection point set; Based on each timing fluctuation point in the network signal flow timing fluctuation point set and each timing fluctuation inflection point in the network signal flow timing fluctuation inflection point set, the network signal big data stream is segmented and processed between timing points to obtain the network signal data sub-stream under each timing segment.

4. The big data network signal security early warning system according to claim 1 is characterized in that: The abnormal security warning processing of the abnormal network signal data sub-flow includes: Perform time series and spatial distribution analysis on the abnormal network signal data sub-flows to obtain the time series distribution characteristics and spatial distribution characteristics of the abnormal network sub-flows; Perform spatiotemporal correlation mining analysis on the temporal distribution characteristics and spatial distribution characteristics of network anomaly sub-flows to obtain the spatiotemporal anomaly correlation pattern between the temporal distribution and spatial distribution of network anomaly sub-flows; Based on the spatiotemporal anomaly correlation pattern between the temporal and spatial distribution of network anomaly sub-flows, anomaly threat assessment and analysis of network signal anomaly data sub-flows are performed to obtain the anomaly threat level of network signal anomaly sub-flows; Based on the abnormal threat level of the network signal abnormal sub-flow, abnormal security warning processing is performed on the corresponding network signal abnormal data sub-flow to generate a network signal abnormal security warning signal.

Images