Big data-based private network data security detection method

Abstract

The invention discloses a private network data security detection method based on big data, and belongs to the field of security detection, and the method comprises the steps: obtaining a real-time data packet from private network traffic, encoding the data packet through a forward error correction code technology, generating redundancy check information, and obtaining an encoded data packet; if a bit error is detected in the transmission process of the coded data packet, requesting a sending end to retransmit a damaged data packet through an automatic retransmission request mechanism to obtain a retransmitted data packet; adjusting the redundancy of a forward error correction code and a trigger threshold of an automatic retransmission request according to the adaptive error correction parameter, generating an optimized error correction scheme, and obtaining a repaired data packet; and aiming at the alarm processing instruction, automatically adjusting a network flow control strategy, limiting the transmission bandwidth of an abnormal destination, updating the sensitive data fingerprint database, and obtaining optimized protection configuration.

Description

Technical Field

[0001] The present invention belongs to the field of security detection, and in particular relates to a special network data security detection method based on big data. Background Art

[0002] In the information age, private networks, as critical infrastructure, necessitate research on data security detection methods of crucial strategic importance. With the widespread adoption of big data technologies, the amount of information carried by private networks has skyrocketed. Data security not only impacts communication quality but also directly impacts national security, business operations, and personal privacy. Big data-based detection methods, through the processing and analysis of massive amounts of data, offer a new technical approach to network security.

[0003] However, existing solutions still exhibit significant shortcomings in practical applications. Traditional data security detection technologies often rely on single error correction or protection mechanisms, lacking adaptability to complex network environments, and struggle to fully guarantee data integrity and confidentiality in high-interference or high-risk scenarios. These limitations render dedicated networks vulnerable to diverse data corruption and leakage threats. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a dedicated network data security detection method based on big data, comprising:

[0005] Obtaining a real-time data packet from dedicated network traffic, encoding the real-time data packet using a forward error correction code technology, generating redundant check information, and obtaining an encoded data packet;

[0006] If a bit error is detected during the transmission of the encoded data packet, the transmitting end is requested to retransmit the damaged data packet through an automatic retransmission request mechanism to obtain a retransmitted data packet;

[0007] For the encoded data packet and the retransmitted data packet, a hybrid error correction strategy is adopted to obtain adaptive error correction parameters by analyzing historical error correction patterns and calculating current network interference intensity;

[0008] Adjusting the redundancy of the forward error correction code and the triggering threshold of the automatic repeat request according to the adaptive error correction parameters to generate an optimized error correction scheme and obtain a repaired data packet;

[0009] Extracting network egress traffic from the repaired data packet, parsing the header and content of the data packet using deep packet inspection technology combined with content-aware analysis and context analysis to obtain traffic feature data;

[0010] Based on the traffic characteristic data, an abnormal traffic pattern recognition algorithm is used, combined with pre-established historical baseline data, to calculate the traffic deviation value, determine abnormal traffic behavior, and obtain an abnormal traffic identification;

[0011] If the abnormal traffic indicator indicates a potential data leak, the data content is extracted from the traffic feature data, and by matching it with the sensitive data fingerprint library, it is determined whether it contains sensitive information to obtain a data leak detection result;

[0012] Based on the data leakage detection results, an alarm log containing abnormal traffic identification and sensitive information matching details is generated and transmitted to the security management platform through a preset encrypted channel to obtain alarm processing instructions;

[0013] In response to the alarm processing instructions, the network traffic control strategy is automatically adjusted, the transmission bandwidth of the abnormal destination is limited, the sensitive data fingerprint library is updated, and an optimized protection configuration is obtained.

[0014] Preferably, the process of obtaining the encoded data packet includes:

[0015] Acquire real-time data packets through a dedicated network interface, generate initial error correction information using forward error correction code technology, and obtain coded data packets;

[0016] Verify the redundant check information in the coded data packet using the preset error correction code rules to determine the verified data packet;

[0017] If packet loss is detected in the verified data packet, the data is reconstructed using the redundant verification information to obtain a reconstructed data packet;

[0018] Based on the integrity of the reconstructed data packet, traffic analysis tools are used to detect data consistency and determine the consistency results;

[0019] Adjust the encoding parameters based on the consistency results, generate optimized error correction codes, and obtain optimized data packets;

[0020] For optimized data packets, obtain the traffic characteristics during transmission and determine the changing trend of traffic characteristics;

[0021] According to the changing trend of traffic characteristics, the encoding strategy is adjusted through machine learning algorithms to obtain the final encoded data packet.

[0022] Preferably, the process of obtaining the retransmitted data packet includes:

[0023] Analyze bit errors in data packet transmission through an error detection mechanism to determine whether there are error bits and obtain error detection results;

[0024] If the error detection result indicates that there is a bit error, a retransmission request is generated and sent to the sender through the request mechanism to obtain the response status of the sender;

[0025] According to the response status of the sending end, the retransmitted data packet is received, the integrity of the data packet is determined, and the retransmitted data packet is obtained;

[0026] Use the data integrity verification algorithm to verify the retransmitted data packet to determine whether there are still bit errors and obtain the verification result;

[0027] If the check result shows no bit errors, the retransmitted data packet is integrated into the transmission sequence to confirm the continuity of the transmission process and obtain the updated data stream;

[0028] The updated data stream is verified twice using a cyclic redundancy check algorithm to determine the integrity of the data stream and obtain the final verification result;

[0029] According to the final verification result, a complete data packet sequence is output to obtain the retransmitted data packet.

[0030] Preferably, the process of obtaining the adaptive error correction parameters includes:

[0031] The error correction pattern distribution is obtained through the stored historical error correction data, and the pattern analysis results are obtained. The pattern analysis results are used to extract data packet analysis features from the encoded data and retransmitted data to obtain feature acquisition output;

[0032] Based on the feature acquisition output, the statistical value of network interference is calculated to obtain the interference intensity value. If the interference intensity value exceeds the preset threshold, the calculation strategy is adjusted through intensity analysis to obtain the optimized intensity analysis result;

[0033] Based on the optimized strength analysis results, the linear regression algorithm is used to calculate the adaptive parameters, and the parameter calculation output is obtained. The parameter calculation output is obtained and combined with the current data packet analysis characteristics to determine the final adaptive error correction parameters.

[0034] Preferably, the process of obtaining the repaired data packet includes:

[0035] Obtain the adaptive error correction parameters of the transmission environment and determine the initial forward error correction code redundancy and automatic repeat request triggering threshold by analyzing the channel noise level and packet loss rate;

[0036] If the channel noise level is higher than a preset threshold, the forward error correction code redundancy is increased and the error correction code is generated through the Reed-Solomon code to obtain a data packet with enhanced error correction capability;

[0037] If the packet loss rate is higher than the preset threshold, the automatic retransmission request trigger threshold is lowered, and the error data packets are marked through the selective repeat protocol to obtain a retransmission request sequence;

[0038] According to the data packets with enhanced error correction capability, a forward error correction decoding algorithm is used to repair the damaged data packets to obtain preliminary repaired data packets;

[0039] For the preliminary repair data packet, residual errors are detected by cyclic redundancy check to obtain error detection results;

[0040] If the error detection result shows residual errors, the automatic retransmission request mechanism is used to retransmit the corresponding data packet according to the retransmission request sequence to obtain the final repaired data packet.

[0041] Preferably, the process of obtaining the flow characteristic data includes:

[0042] Through deep packet inspection technology, network egress traffic is extracted from the repaired data packets, and the packet header and content are analyzed to obtain the original traffic data;

[0043] Using content-aware analysis technology, the original traffic data is parsed to extract the protocol type, source address, and destination address in the data packet to obtain content feature data;

[0044] Through context analysis technology, the content feature data is correlated and analyzed, and the context feature data is obtained by combining the timestamp and session identifier of the data packet;

[0045] If the protocol type in the context feature data is the preset protocol of interest, the support vector machine algorithm is used to classify the context feature data to determine whether the traffic is abnormal and obtain an abnormal traffic identifier;

[0046] According to the abnormal traffic identifier, the context feature data is filtered, the data packet content corresponding to the abnormal traffic is extracted, and the abnormal traffic feature data is obtained;

[0047] The random forest algorithm is used to sort the abnormal traffic feature data by feature importance, determine the key features, and obtain the key traffic feature data;

[0048] A preset threshold judgment method is used. If the frequency of source address appearance in key traffic feature data exceeds the threshold, it is marked as high-risk traffic and high-risk traffic feature data is obtained.

[0049] Preferably, the process of obtaining the abnormal traffic identifier includes:

[0050] Obtain traffic characteristic data, clean and standardize the data through the preprocessing module to obtain a normalized traffic data set;

[0051] The random forest algorithm is used to analyze the characteristics of the normalized traffic data set in combination with pre-established historical baseline data to generate traffic size deviation, time deviation value and destination deviation;

[0052] Through the deviation value generation module, the difference between each deviation value and the historical baseline data is calculated to obtain the deviation feature vector;

[0053] If the modulus of the deviation feature vector is greater than the preset threshold, the abnormal traffic identification algorithm is used to classify the traffic data and determine the abnormal traffic behavior;

[0054] Based on the classification results of abnormal traffic behavior, an abnormal traffic identification is generated and stored in the abnormal behavior database.

[0055] Preferably, the process of obtaining the data leakage detection result includes:

[0056] If the traffic monitoring system detects a traffic anomaly indicator, traffic characteristic data is obtained from the network traffic, and data content is extracted using a data packet parsing technology to obtain first data content;

[0057] Using a feature extraction algorithm from the first data content, obtaining structured features of the data content, and generating a first feature set;

[0058] Matching the first feature set with a pre-established sensitive data fingerprint library; if the matching degree exceeds a preset threshold, it is determined that sensitive information is contained, and a first matching result is obtained;

[0059] Based on the first matching result, a decision tree algorithm is used to classify the possibility of data leakage and generate a data leakage detection result;

[0060] Extracting context information of the leakage event from the data leakage detection result, obtaining associated traffic source information through log analysis technology, and generating a first traffic source set;

[0061] For the first traffic source set, an abnormal traffic analysis technology is used to determine whether the traffic source is continuously abnormal, and an abnormal traffic confirmation result is obtained;

[0062] The final data leakage detection report is generated by jointly analyzing the abnormal traffic confirmation results and the first matching results.

[0063] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the computing program.

[0064] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the method when executed by a processor.

[0065] Compared with the prior art, the present invention has the following advantages and technical effects:

[0066] The present invention discloses a method for preventing network data leakage, which monitors dedicated network traffic in real time and adopts a hybrid error correction strategy combining forward error correction code and automatic retransmission request to perform adaptive error correction on data packets during transmission, thereby improving the reliability of data transmission. At the same time, deep packet inspection and abnormal traffic identification technology are used to perform content analysis and behavior judgment on the repaired data packets to achieve accurate detection of potential data leakage. When abnormal traffic is found, the present invention automatically matches the sensitive data fingerprint library to determine whether sensitive information has been leaked, and generates a detailed alarm log. Finally, according to the alarm processing instructions, the network traffic control strategy is dynamically adjusted and the protection configuration is updated to form a closed-loop data leakage protection mechanism. Through multi-level data protection and intelligent analysis, the present invention effectively improves network security protection capabilities and reduces the risk of sensitive information leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0068] Figure 1 This is a flowchart of a method for detecting data security in a dedicated network based on big data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0070] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0071] Example 1

[0072] like Figure 1 As shown, this embodiment provides a dedicated network data security detection method based on big data, including:

[0073] Step S101: obtain a real-time data packet from a dedicated network flow, encode the data packet using a forward error correction code technology, generate redundant check information, and obtain an encoded data packet.

[0074] Real-time data packets are acquired through a dedicated network interface, and forward error correction code technology is used to generate initial error correction information to obtain an encoded data packet. The redundant checksum information in the encoded data packet is verified using preset error correction code rules to determine the verified data packet. If packet loss is detected in the verified data packet, the data is reconstructed using the redundant checksum information to obtain a reconstructed data packet. Based on the integrity of the reconstructed data packet, a traffic analysis tool is used to check data consistency and determine the consistency result. Based on the consistency result, the encoding parameters are adjusted to generate an optimized error correction code, resulting in an optimized data packet. For the optimized data packet, the traffic characteristics during its transmission are obtained to determine the changing trends of the traffic characteristics. Based on the changing trends of the traffic characteristics, the encoding strategy is adjusted using a machine learning algorithm to obtain the final encoded data packet.

[0075] For example, when capturing real-time data packets from a private network, a packet capture tool such as tcpdump, based on the libpcap library, can be used. The filter rule is set to "host192.168.1.100 and port 443" to capture data from a specific IP address and HTTPS port, with a sampling rate of 1000 packets per second. After the captured raw data packets are fragmented using the MTU, each fragment is 1500 bytes long, consisting of a 14-byte Ethernet header, a 20-byte IP header, a 32-byte TCP header, and a 1434-byte payload. RS(255,223) Reed-Solomon code is used for forward error correction, dividing the 223-byte raw data into 16 14-byte blocks (with trailing zeros). A 32-byte checksum is calculated using the generator polynomial g(x) = (x-α^0)(x-α^1)(x-α^31) over the Galois Field GF(2^8), which can correct up to 16 bytes of errors. During the encoding process, each 1434-byte payload is divided into eight 223-byte groups, generating a total of 8 × 32 = 256 bytes of redundant checksum information, which is concatenated with the original data into a 1690-byte encoded packet. Error correction capability is verified by calculating the Hamming distance: data can be fully recovered when the bit error rate is less than 1.2% (16 / 1357). To improve efficiency, SIMD instructions are used to parallelize finite field multiplication, achieving an encoding throughput of 12 Gbps under the Intel AVX2 instruction set. The encoded data packet is checked for integrity using a CRC32 checksum with the polynomial 0x04C11DB7, generating a 4-byte checksum that is appended to the packet.

[0076] Step S102: If a bit error is detected in the transmission process of the encoded data packet, the automatic retransmission request mechanism is used to request the transmitting end to retransmit the damaged data packet to obtain a retransmitted data packet.

[0077] An error detection mechanism analyzes bit errors in data packet transmission to determine whether any bit errors exist and obtain an error detection result. If the error detection result indicates a bit error, a retransmission request is generated and sent to the sender through the request mechanism to obtain the sender's response status. Based on the sender's response status, the retransmitted data packet is received, the integrity of the data packet is determined, and a retransmitted data packet is obtained. A data integrity verification algorithm is used to verify the retransmitted data packet to determine whether any bit errors still exist and obtain a verification result. If the verification result indicates no bit errors, the retransmitted data packet is integrated into the transmission sequence to confirm the continuity of the transmission process and obtain an updated data stream. The updated data stream is then re-verified using a cyclic redundancy check algorithm to determine the integrity of the data stream and obtain a final verification result. Based on the final verification result, a complete data packet sequence is output to confirm the reliability of the transmission process and obtain the final data output.

[0078] For example, during data transmission, when the receiving end detects a bit error in a data packet using the cyclic redundancy check (CRC-32) algorithm—for example, if the checksum 0xEDB88320 does not match the expected value 0x1A2B3C4D—the automatic repeat request (ARQ) mechanism is triggered. The receiving end immediately constructs a feedback frame containing a negative acknowledgement (NACK) signal and a sequence number (e.g., Seq = 5) and sends it to the source end via the fast retransmit mechanism in the TCP protocol, under a 200ms RTT network. After parsing the NACK, the source end extracts the original data packet (e.g., a 1500-byte IP packet) from the send buffer, performs forward error correction using Hamming (7,4) encoding, appends a 16-bit checksum 0xABCD, and retransmits the data packet using a sliding window protocol, adjusting the window size to four packets. If a NACK is still received after three consecutive retransmissions (with a threshold set to MAX_RETRY = 3), the selective repeat request (SACK) mechanism is activated, retransmitting only the lost fragment (e.g., offsets 256-512 bytes). The receiving end performs a secondary check on the retransmitted data packet. If the CRC check passes and the sequence number is continuous, the data is delivered to the application layer and an ACK signal is returned (e.g., ACKSeq = 5). The receive window is also updated to sequence number 6. Throughout this process, the retransmission interval is adjusted using an exponential backoff algorithm, with an initial timeout of 500ms and a backoff factor of 2 to ensure network congestion control. For scenarios with high real-time requirements, a forward error correction code (e.g., Reed-Solomon (255, 223)) can be combined to embed 32 bytes of redundant data in the data packet, achieving a 95% error correction success rate even with a 5% random bit error rate.

[0079] Step S103: for the encoded data packet and the retransmitted data packet, a hybrid error correction strategy is adopted to obtain adaptive error correction parameters by analyzing historical error correction modes and calculating the current network interference intensity.

[0080] Through the stored historical error correction data, the error correction pattern distribution is obtained to obtain the pattern analysis results. The pattern analysis results are used to extract data packet analysis features from the coded data and retransmitted data to obtain the feature acquisition output. Based on the feature acquisition output, the statistical value of the network interference is calculated to obtain the interference intensity value. If the interference intensity value exceeds the preset threshold, the calculation strategy is adjusted through intensity analysis to obtain the optimized intensity analysis results. Based on the optimized intensity analysis results, the adaptive parameters are calculated using the linear regression algorithm to obtain the parameter calculation output. The parameter calculation output is obtained and combined with the current data packet analysis features to determine the final adaptive error correction parameters. Based on the final adaptive error correction parameters, the error correction strategy of the coded data and retransmitted data is adjusted to obtain the updated data processing solution.

[0081] For example, when encoding data packets, a Reed-Solomon (255, 223) encoding scheme is used. Each data packet contains a 223-byte payload and a 32-byte checksum, which can correct up to 16 bytes of errors. When a data packet fails to transmit and needs to be retransmitted, the system records the historical error correction mode. For example, in the last 10 transmissions, an average of 12 bytes of errors need to be corrected each time, with a standard deviation of 3 bytes. Based on these historical data, an exponentially weighted moving average algorithm (α=0.2) is used to calculate the current network interference intensity index. When the index exceeds the threshold of 0.75, the adaptive adjustment mechanism is triggered. The adjustment process uses the least squares method to fit the relationship curve y=1.2x+5 (where x is the interference intensity and y is the required redundant bytes) between the historical error correction requirements and the network status. Based on the current interference intensity of 0.8, it is calculated that 14.6 bytes of additional protection are required, and the system automatically increases the error correction capability to 18 bytes. At the same time, the system dynamically adjusts the mix of forward error correction (FEC) and automatic repeat request (ARQ). When the packet loss rate is below 5%, a 7:3 FEC-first strategy is adopted. Between 5% and 15%, a 5:5 balanced strategy is adopted. Above 15%, a 3:7 ARQ-first mode is switched. After each adjustment, the historical database is updated to ensure that subsequent decisions are based on the latest network status information. To optimize performance, the system recalculates network parameters every 30 seconds, using a sliding window mechanism to maintain a record of the last 100 data packets for analysis.

[0082] Step S104: According to the adaptive error correction parameters, the redundancy of the forward error correction code and the triggering threshold of the automatic repeat request are adjusted to generate an optimized error correction scheme and obtain a repaired data packet.

[0083] The adaptive error correction parameters for the transmission environment are obtained. The initial forward error correction code redundancy and automatic repeat request triggering threshold are determined by analyzing the channel noise level and packet loss rate. If the channel noise level exceeds the preset threshold, the forward error correction code redundancy is increased, and an error correction code is generated using a Reed-Solomon code (where the variable n represents the total number of symbols, k represents the number of data symbols, and nk represents the number of redundant symbols), resulting in a data packet with enhanced error correction capability. If the packet loss rate exceeds the preset threshold, the automatic repeat request triggering threshold is lowered, and error packets are marked using the Selective Repeat Protocol (SRP) to generate a retransmission request sequence. Based on the data packet with enhanced error correction capability, the damaged data packet is repaired using a forward error correction decoding algorithm (Berlekamp-Massey algorithm, which inputs error symbol positions and outputs an error correction polynomial) to obtain a preliminary repaired data packet. Residual errors in the preliminary repaired data packet are detected using a cyclic redundancy check (CRC) (which inputs the data packet bit stream and outputs a checksum) to obtain an error detection result. If the error detection result indicates residual errors, the corresponding data packet is retransmitted using the automatic repeat request mechanism according to the retransmission request sequence to obtain the final repaired data packet. By analyzing the error correction efficiency and transmission reliability of the final repaired data packet, the adaptive error correction parameters are dynamically adjusted, and the forward error correction code redundancy and automatic repeat request triggering threshold are updated to obtain an optimized error correction scheme.

[0084] For example, in the adaptive error correction parameter adjustment process, firstly, by real-time monitoring of the channel quality index (such as bit error rate BER = 1.2 × 10 -4, signal-to-noise ratio (SNR=18dB) to dynamically calculate the redundancy of the forward error correction code. Using the Reed-Solomon (255, 223) code as the basis, the number of redundant symbols is adaptively expanded according to the channel status. When the packet loss rate exceeds 0.5%, the coding block is automatically adjusted to (255, 215), increasing the redundant data by 12.8%. At the same time, the number of consecutive error frames is counted based on the sliding window algorithm. When three CRC check failures are detected within 10ms, the automatic repeat request (ARQ) threshold is adjusted, shortening the retransmission timeout from 200ms to 150ms. By constructing a Markov chain model to predict the channel degradation trend, when the probability of the bit error rate rising in the next cycle is predicted to exceed 65%, the constraint length of the convolutional code is increased from 7 to 9 in advance, and the soft decision mode of the Viterbi decoder is enabled. The packet repair phase utilizes an iterative erasure decoding algorithm. A syndrome is first calculated for the received (255, 215) codeword. If the syndrome is non-zero, a qian search algorithm is initiated to locate up to 20 error locations. Error correction is then performed in conjunction with the Berlekamp-Massey algorithm. For any remaining error packets, the Hamming distance between the ARQ retransmitted version and the original packet is compared, and the version with the smallest difference is weightedly combined to produce the final repaired packet. Throughout this process, the channel parameter database is continuously updated, and historical measurements are smoothed using an exponentially weighted moving average (α = 0.3) to ensure the stability of parameter adjustments.

[0085] Step S105: extract network egress traffic from the repaired data packet, and parse the header and content of the data packet through deep packet inspection technology combined with content-aware analysis and context analysis to obtain traffic feature data.

[0086] Using deep packet inspection technology, network egress traffic is extracted from repaired data packets, and the packet headers and content are parsed to obtain raw traffic data. Using content-aware analysis technology, the raw traffic data is parsed to extract the protocol type, source address, and destination address from the packets, obtaining content feature data. Using context analysis technology, the content feature data is correlated and analyzed, combining the packet timestamp and session ID to obtain context feature data. If the protocol type in the context feature data is a preset protocol of interest, the support vector machine algorithm is used to classify the context feature data to determine whether the traffic is abnormal, obtaining an abnormal traffic identifier. Based on the abnormal traffic identifier, the context feature data is filtered, and the packet content corresponding to the abnormal traffic is extracted to obtain abnormal traffic feature data. Using a random forest algorithm, the abnormal traffic feature data is ranked by feature importance, key features are identified, and key traffic feature data is obtained. Using a preset threshold judgment method, if the frequency of the source address in the key traffic feature data exceeds the threshold, the traffic is marked as high-risk, obtaining high-risk traffic feature data.

[0087] For example, in the repaired data packets, deep packet inspection technology is first used to extract network egress traffic. Regular expressions are then used to match the header information of specific protocols, such as HTTP, for example, matching "GET / index.htmlHTTP / 1.1" to identify the request type. Next, content-aware analysis is performed using the TF-IDF algorithm to calculate the weight of keywords in the data packets. For example, if "login" appears 50 times in 1,000 data packets, its weight is 0.05. Combined with contextual analysis, this traffic is identified as possibly involving user login behavior. The data packet header is further parsed to extract the source IP address, such as 192.168.1.1, and the destination IP address, such as 203.0.113.1. Combined with the port number, such as 80, the traffic direction is determined. By analyzing the data packet content, the K-means clustering algorithm is used to classify the traffic into different categories. For example, 1,000 data packets can be divided into five categories, each containing 200 data packets, to identify abnormal traffic such as DDoS attacks. Finally, combined with traffic characteristic data, such as the average packet size of 1500 bytes and the average transmission rate of 1Mbps, a traffic report is generated to provide a basis for network security management.

[0088] In step S106, an abnormal traffic pattern recognition algorithm is used for the traffic characteristic data, combined with pre-established historical baseline data, to calculate the deviation values of traffic size, time, and destination, determine abnormal traffic behavior, and obtain an abnormal traffic identification.

[0089] Obtain traffic characteristic data, clean and standardize the data through the pre-processing module to obtain a normalized traffic data set. Use the random forest algorithm, combined with pre-established historical baseline data, to perform feature analysis on the normalized traffic data set to generate traffic size deviation, time deviation value, and destination deviation. Use the deviation value generation module to calculate the difference between each deviation value and the historical baseline data to obtain a deviation feature vector. If the modulus of the deviation feature vector is greater than the preset threshold, the abnormal traffic identification algorithm is used to classify the traffic data and determine abnormal traffic behavior. Based on the classification results of abnormal traffic behavior, an abnormal traffic identifier is generated and stored in the abnormal behavior database. Extract the abnormal traffic identifier from the abnormal behavior database, and use the time series analysis tool to generate the frequency and change trend of abnormal behavior. Based on the change trend, a dynamic distribution map of abnormal traffic behavior is generated through the visualization module to determine the abnormal traffic monitoring results.

[0090] For example, in abnormal traffic detection, network traffic data is first collected at intervals of 5 minutes using a sliding window algorithm. For example, the traffic average of the destination IP 192.168.1.100 in a certain period is 120Mbps, and the standard deviation is 15Mbps. The exponentially weighted moving average (EWMA) algorithm is used to establish a baseline model, and the smoothing coefficient α is set to 0.3. When the real-time traffic reaches 180Mbps, its Z-score value is calculated to be (180-120) / 15=4, which exceeds the 3σ threshold. For time dimension anomalies, time series decomposition (STL) is used to detect periodic deviations. If the morning peak traffic on weekdays drops by 40% compared to the baseline, an alarm is triggered. For destination anomalies, a clustering algorithm (such as DBSCAN, setting eps=0.5, min_samples=10) is used to find that a certain IP suddenly connects to 50 new destination ports, while the historical baseline is only 3-5. Finally, a multi-dimensional decision tree model was used. When traffic Z-score exceeds 3, time deviation exceeds 30%, and destination dispersion exceeds 20, it is marked as abnormal traffic with the ID: ALERT_202. The traffic is then linked to the threat intelligence database to match known attack signatures. For example, if the similarity with the exploit pattern of the CVE-2023-1234 vulnerability reaches 85%, a defense rule is automatically generated to block the traffic. Kalman filtering is used to dynamically update baseline data during this process, ensuring the model adapts to changes in the network environment.

[0091] Step S107: If the abnormal traffic indicator indicates potential data leakage, the data content is extracted from the traffic feature data, and by matching it with the sensitive data fingerprint library, it is determined whether it contains sensitive information to obtain a data leakage detection result.

[0092] If the traffic monitoring system detects a traffic anomaly indicator, it obtains traffic feature data from the network traffic, extracts the data content through packet parsing technology, and obtains the first data content. A feature extraction algorithm is used from the first data content to obtain the structured features of the data content and generate a first feature set. The first feature set is matched with a pre-established sensitive data fingerprint library. If the matching degree exceeds a preset threshold, it is determined that sensitive information is contained and a first matching result is obtained. Based on the first matching result, a decision tree algorithm is used to classify the possibility of data leakage and generate a data leakage detection result. The context information of the leakage event is extracted from the data leakage detection result, and the associated traffic source information is obtained through log analysis technology to generate a first traffic source set. For the first traffic source set, abnormal traffic analysis technology is used to determine whether the traffic source is continuously abnormal and obtain an abnormal traffic confirmation result. The final data leakage detection report is generated through a joint analysis of the abnormal traffic confirmation result and the first matching result.

[0093] For example, when a network traffic monitoring system detects that a server suddenly generates over 1GB of abnormal outbound traffic within 10 minutes (the baseline is 100MB / 10 minutes), the traffic analysis module first extracts the raw data packets from the TCP / UDP payload and uses Wireshark to parse the application layer protocol content. For HTTP traffic, a regular expression (such as `\d{18}\|\d{4}[-\s]\d{4}[-\s]\d{4}`) is used to match the credit card number format; for database traffic, a SQL parser is used to extract the field values in the WHERE condition. The extracted text content is hashed with SHA-256 and then compared with a predefined sensitive data fingerprint library (containing 500,000 hash values, such as a combined hash of the first 6 digits of an ID number, date of birth, and last 4 digits). The SimHash algorithm is used to calculate the Hamming distance, and data is identified as sensitive when the distance is less than 3. For example, `3e9d7a5f2b.. is detected. If the hash value exactly matches the customer information hash value stored in the fingerprint database (3e9d7a5f2b...), and the data is sent through an unauthorized port (such as 6667), a data leak alert is triggered. The system automatically generates a security incident report containing the leaked data type (such as bank card number), data volume (128 records), and destination IP address (192.168.1.100). The entire process is handled in real time via the Kafka message queue, with average latency under 200 milliseconds.

[0094] In step S108, based on the data leakage detection result, an alarm log including abnormal traffic identification and sensitive information matching details is generated and transmitted to the security management platform through a preset encrypted channel to obtain an alarm processing instruction.

[0095] The data leakage detection module performs real-time analysis of network traffic, employs an anomaly detection algorithm to identify abnormal traffic patterns, and generates an abnormal traffic identifier. If the abnormal traffic identifier exceeds the preset threshold, sensitive information matching details are extracted from the traffic data in combination with sensitive information matching rules to generate an alarm log. Alarm logs are structured using log formatting tools to generate standardized alarm log data. Through a preset encrypted channel, the standardized alarm log data is encrypted using an encrypted transmission protocol and transmitted to the security management platform. On the security management platform, a classification algorithm is used to prioritize the received alarm log data and determine the urgency of the alarm. Based on the urgency of the alarm, the preset command response mechanism is triggered, generating alarm processing instructions. The alarm processing instructions are then transmitted back to the data leakage detection module via an encrypted channel to update the detection rules and optimize real-time threat detection.

[0096] For example, during data leak detection, the system monitors network traffic in real time through its traffic analysis module. Using an entropy-based anomaly detection algorithm, if a specific IP address sends more than 5,000 requests within 10 minutes, and the requests contain a large number of duplicate packets, the system flags the traffic as anomalous and generates an anomalous traffic identifier: "ABN-20231012-001." The system then uses a regular expression matching algorithm to screen the data within the anomalous traffic for sensitive information. It detects the string "110105199003071234," which matches the format of an ID card number, and the string "6222020200001234567," which matches the format of a bank card number. These matches are successfully matched against a pre-defined sensitive information rule library. The system encapsulates the detection results into a JSON-formatted alert log, containing the anomalous traffic identifier, the matching sensitive information type, and the specific content. The log is encrypted using the AES-256 algorithm and transmitted to the security management platform via a pre-defined SSL / TLS encrypted channel. After receiving the alarm log, the security management platform uses the pre-shared key to decrypt it and automatically generates the processing instruction "BLOCK-IP-192.168.1.100" based on the preset alarm processing rules, instructing the firewall to immediately block access from the abnormal IP address, trigger an alarm notification, and send relevant information to the security administrator for further investigation.

[0097] Step S109 , based on the alarm processing instruction, automatically adjust the network traffic control strategy, limit the transmission bandwidth of the abnormal destination, update the sensitive data fingerprint library, and obtain the optimized protection configuration.

[0098] By parsing the alarm processing instructions, the abnormal destination information is obtained and the characteristics of the abnormal traffic are determined. If the abnormal destination matches the preset sensitive data fingerprint, a traffic control policy is generated, the corresponding bandwidth is limited, and a preliminary network adjustment plan is obtained. According to the preliminary network adjustment plan, the sensitive data fingerprint library is updated, new abnormal traffic characteristics are added, and an updated fingerprint data set is generated. Using the updated fingerprint data set, potential anomalies in network traffic are detected to determine whether further adjustment of bandwidth allocation is required to obtain an optimized traffic policy. Based on the optimized traffic policy, a protection configuration is generated, and the security configuration management module is updated to obtain the final protection configuration. If the final protection configuration triggers a new alarm processing instruction, the alarm information is cyclically parsed, the traffic policy is adjusted, and a continuously optimized network protection plan is obtained. According to the continuously optimized network protection plan, the bandwidth allocation rules are regularly updated to generate a dynamic security configuration management policy.

[0099] For example, after detecting an abnormal alarm, the system first identifies the IP address of the abnormal destination, such as 192.168.1.100, through a traffic analysis algorithm, and calculates the traffic peak of the IP based on historical traffic data to reach 500Mbps, which is much higher than the normal threshold of 200Mbps. Subsequently, the system automatically calls the traffic control strategy and uses the token bucket algorithm to limit the transmission bandwidth of the abnormal destination to 100Mbps to ensure the rational allocation of network resources. At the same time, the system analyzes the characteristics of the abnormal traffic through a machine learning model, extracts new sensitive data fingerprints, such as the hash value "a1b2c3d4" of a specific data packet, and updates it to the sensitive data fingerprint library. Finally, the system redeploys the firewall rules based on the optimized protection configuration to ensure that all traffic undergoes strict fingerprint matching and bandwidth control when passing through, thereby effectively improving network security.

[0100] On the other hand, this embodiment further provides an electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the computing program.

[0101] On the other hand, this embodiment further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the method when executed by a processor.

[0102] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for detecting data security in a dedicated network based on big data, characterized in that: include: Obtaining a real-time data packet from dedicated network traffic, encoding the real-time data packet using a forward error correction code technology, generating redundant check information, and obtaining an encoded data packet; If a bit error is detected during the transmission of the encoded data packet, the transmitting end is requested to retransmit the damaged data packet through an automatic retransmission request mechanism to obtain a retransmitted data packet; For the encoded data packet and the retransmitted data packet, a hybrid error correction strategy is adopted to obtain adaptive error correction parameters by analyzing historical error correction patterns and calculating current network interference intensity; Adjusting the redundancy of the forward error correction code and the triggering threshold of the automatic repeat request according to the adaptive error correction parameters to generate an optimized error correction scheme and obtain a repaired data packet; Extracting network egress traffic from the repaired data packet, parsing the header and content of the data packet using deep packet inspection technology combined with content-aware analysis and context analysis to obtain traffic feature data; Based on the traffic characteristic data, an abnormal traffic pattern recognition algorithm is used, combined with pre-established historical baseline data, to calculate the traffic deviation value, determine abnormal traffic behavior, and obtain an abnormal traffic identification; If the abnormal traffic indicator indicates a potential data leak, the data content is extracted from the traffic feature data, and by matching it with the sensitive data fingerprint library, it is determined whether it contains sensitive information to obtain a data leak detection result; Based on the data leakage detection results, an alarm log containing abnormal traffic identification and sensitive information matching details is generated and transmitted to the security management platform through a preset encrypted channel to obtain alarm processing instructions; In response to the alarm processing instructions, the network traffic control strategy is automatically adjusted, the transmission bandwidth of the abnormal destination is limited, the sensitive data fingerprint library is updated, and an optimized protection configuration is obtained.

2. The method according to claim 1, characterized in that The process of obtaining the encoded data packet includes: Acquire real-time data packets through a dedicated network interface, generate initial error correction information using forward error correction code technology, and obtain coded data packets; Verify the redundant check information in the coded data packet using the preset error correction code rules to determine the verified data packet; If packet loss is detected in the verified data packet, the data is reconstructed using the redundant verification information to obtain a reconstructed data packet; Based on the integrity of the reconstructed data packet, traffic analysis tools are used to detect data consistency and determine the consistency results; Adjust the encoding parameters based on the consistency results, generate optimized error correction codes, and obtain optimized data packets; For optimized data packets, obtain the traffic characteristics during transmission and determine the changing trend of traffic characteristics; According to the changing trend of traffic characteristics, the encoding strategy is adjusted through machine learning algorithms to obtain the final encoded data packet.

3. The method according to claim 1, characterized in that The process of obtaining the retransmitted data packet includes: Analyze bit errors in data packet transmission through an error detection mechanism to determine whether there are error bits and obtain error detection results; If the error detection result indicates that there is a bit error, a retransmission request is generated and sent to the sender through the request mechanism to obtain the response status of the sender; According to the response status of the sending end, the retransmitted data packet is received, the integrity of the data packet is determined, and the retransmitted data packet is obtained; Use the data integrity verification algorithm to verify the retransmitted data packet to determine whether there are still bit errors and obtain the verification result; If the check result shows no bit errors, the retransmitted data packet is integrated into the transmission sequence to confirm the continuity of the transmission process and obtain the updated data stream; The updated data stream is verified twice using a cyclic redundancy check algorithm to determine the integrity of the data stream and obtain the final verification result; According to the final verification result, a complete data packet sequence is output to obtain the retransmitted data packet.

4. The method according to claim 1, wherein The process of obtaining the adaptive error correction parameters includes: The error correction pattern distribution is obtained through the stored historical error correction data, and the pattern analysis results are obtained. The pattern analysis results are used to extract data packet analysis features from the encoded data and retransmitted data to obtain feature acquisition output; Based on the feature acquisition output, the statistical value of network interference is calculated to obtain the interference intensity value. If the interference intensity value exceeds the preset threshold, the calculation strategy is adjusted through intensity analysis to obtain the optimized intensity analysis result; Based on the optimized strength analysis results, the linear regression algorithm is used to calculate the adaptive parameters, and the parameter calculation output is obtained. The parameter calculation output is obtained and combined with the current data packet analysis characteristics to determine the final adaptive error correction parameters.

5. The method according to claim 1, wherein The process of obtaining the repaired data packet includes: Obtain the adaptive error correction parameters of the transmission environment and determine the initial forward error correction code redundancy and automatic repeat request triggering threshold by analyzing the channel noise level and packet loss rate; If the channel noise level is higher than a preset threshold, the forward error correction code redundancy is increased and the error correction code is generated through the Reed-Solomon code to obtain a data packet with enhanced error correction capability; If the packet loss rate is higher than the preset threshold, the automatic retransmission request trigger threshold is lowered, and the error data packets are marked through the selective repeat protocol to obtain a retransmission request sequence; According to the data packets with enhanced error correction capability, a forward error correction decoding algorithm is used to repair the damaged data packets to obtain preliminary repaired data packets; For the preliminary repair data packet, residual errors are detected by cyclic redundancy check to obtain error detection results; If the error detection result shows residual errors, the automatic retransmission request mechanism is used to retransmit the corresponding data packet according to the retransmission request sequence to obtain the final repaired data packet.

6. The method according to claim 1, wherein The process of obtaining traffic characteristic data includes: Through deep packet inspection technology, network egress traffic is extracted from the repaired data packets, and the packet header and content are analyzed to obtain the original traffic data; Using content-aware analysis technology, the original traffic data is parsed to extract the protocol type, source address, and destination address in the data packet to obtain content feature data; Through context analysis technology, the content feature data is correlated and analyzed, and the context feature data is obtained by combining the timestamp and session identifier of the data packet; If the protocol type in the context feature data is the preset protocol of interest, the support vector machine algorithm is used to classify the context feature data to determine whether the traffic is abnormal and obtain an abnormal traffic identifier; According to the abnormal traffic identifier, the context feature data is filtered, the data packet content corresponding to the abnormal traffic is extracted, and the abnormal traffic feature data is obtained; The random forest algorithm is used to sort the abnormal traffic feature data by feature importance, determine the key features, and obtain the key traffic feature data; A preset threshold judgment method is used. If the frequency of source address appearance in key traffic feature data exceeds the threshold, it is marked as high-risk traffic and high-risk traffic feature data is obtained.

7. The method according to claim 1, characterized in that The process of obtaining the abnormal traffic identifier includes: Obtain traffic characteristic data, clean and standardize the data through the preprocessing module to obtain a normalized traffic data set; The random forest algorithm is used to analyze the characteristics of the normalized traffic data set in combination with pre-established historical baseline data to generate traffic size deviation, time deviation value and destination deviation; Through the deviation value generation module, the difference between each deviation value and the historical baseline data is calculated to obtain the deviation feature vector; If the modulus of the deviation feature vector is greater than the preset threshold, the abnormal traffic identification algorithm is used to classify the traffic data and determine the abnormal traffic behavior; Based on the classification results of abnormal traffic behavior, an abnormal traffic identification is generated and stored in the abnormal behavior database.

8. The method according to claim 1, characterized in that The process of obtaining the data leakage detection result includes: If the traffic monitoring system detects a traffic anomaly indicator, traffic characteristic data is obtained from the network traffic, and data content is extracted using a data packet parsing technology to obtain first data content; Using a feature extraction algorithm from the first data content, obtaining structured features of the data content, and generating a first feature set; Matching the first feature set with a pre-established sensitive data fingerprint library; if the matching degree exceeds a preset threshold, it is determined that sensitive information is contained, and a first matching result is obtained; Based on the first matching result, a decision tree algorithm is used to classify the possibility of data leakage and generate a data leakage detection result; Extracting context information of the leakage event from the data leakage detection result, obtaining associated traffic source information through log analysis technology, and generating a first traffic source set; For the first traffic source set, an abnormal traffic analysis technology is used to determine whether the traffic source is continuously abnormal, and an abnormal traffic confirmation result is obtained; The final data leakage detection report is generated by jointly analyzing the abnormal traffic confirmation results and the first matching results.

9. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein: When the processor executes the computing program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images