Message decoding method and device based on statistical characteristic analysis
By performing multi-dimensional statistical feature analysis on network packets, calculating the matching confidence of encoding types, and determining decoding strategies, the efficiency and accuracy problems of existing decoding methods in complex encoding scenarios are solved, and intelligent decoding decision-making is realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU DPTECH TECH
- Filing Date
- 2026-02-14
- Publication Date
- 2026-05-26
Smart Images
Figure CN122093129A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer information processing, and more specifically, to a message decoding method and apparatus based on statistical feature analysis. Background Technology
[0002] In today's digital age, the network communication environment is becoming increasingly complex and dangerous. During the communication and interaction of network services, attackers often use various encoding techniques to disguise malicious message payloads, thereby evading security detection and protection. Common text encoding methods include URL encoding, Base64 encoding, and HTML entity encoding, etc. These encoding techniques can be used individually or in multiple nested combinations to form complex encoding structures.
[0003] Current mainstream decoding methods mainly face the technical challenge of encoding uncertainty, which is a core pain point in network communication and data parsing. Traditional decoding methods either rely on metadata information such as Content-Type, but this metadata may be missing, incorrect, or maliciously tampered with, leading to failure in the message decoding process or producing incorrect results; or they are "trial and error" decoding methods, which try known decoding methods one by one until a successful decoding method is found. Although this method is simple and direct, it has obvious performance bottlenecks when facing complex encoding scenarios. When the message undergoes multiple layers of encoding or mixed encoding, the brute-force method will generate a lot of computational overhead, seriously affecting the system's processing efficiency and real-time performance.
[0004] Therefore, a new message decoding method and device based on statistical feature analysis is needed.
[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] In view of this, this application provides a message decoding method and apparatus based on statistical feature analysis, which can effectively deal with encoding variants and deliberate circumvention techniques, improve encoding recognition accuracy, and effectively improve decoding efficiency in complex encoding scenarios.
[0007] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.
[0008] According to one aspect of this application, a message decoding method based on statistical feature analysis is proposed. The method includes: acquiring the original character stream of a network message; performing multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; calculating the matching confidence scores of the original character stream with various preset encoding types based on the multi-dimensional feature set; determining a target decoding strategy based on the matching confidence scores of each encoding type; and performing decoding processing on the original character stream according to the target decoding strategy.
[0009] In one exemplary embodiment of this application, obtaining the original character stream of a network packet includes: performing protocol parsing on the network packet to determine the target transmission protocol; and extracting the original character stream to be decoded from the network packet according to the target transmission protocol.
[0010] In one exemplary embodiment of this application, multi-dimensional statistical feature analysis is performed on the original character stream, including: basic character distribution statistical feature analysis of the original character stream; and / or encoding pattern statistical feature analysis of the original character stream; and / or spatial distribution feature calculation statistical feature analysis of the original character stream.
[0011] In one exemplary embodiment of this application, basic character distribution statistical feature analysis is performed on the original character stream, including: counting the occurrence frequency of each character in the original character stream to generate a character frequency distribution; calculating the occupancy ratio of at least one character category in the original character stream, wherein the character category includes at least one of the following: hexadecimal characters, Base64 character set characters, special symbol characters, printable ASCII characters, and control characters or non-printable characters.
[0012] In one exemplary embodiment of this application, the encoding pattern statistical feature analysis of the original character stream includes: statistically analyzing the occurrence ratio and distribution pattern of feature characters or feature symbols related to the encoding type in the original character stream; analyzing the encoding pattern integrity corresponding to the feature characters or feature symbols; and calculating the encoding segment length distribution feature corresponding to the encoding type, wherein the length distribution feature includes at least the statistical distribution or coefficient of variation of the segment length.
[0013] In one exemplary embodiment of this application, the spatial distribution feature calculation and statistical feature analysis of the original character stream includes: obtaining the position information of the feature characters in the original character stream; calculating the distribution uniformity of the feature characters in the character stream based on the position information; performing segmented statistics on the original character stream and calculating the dispersion of the number of occurrences of the feature characters in each segment; and calculating the statistical quantity of the interval between adjacent feature characters.
[0014] In one exemplary embodiment of this application, calculating the matching confidence of the original character stream with multiple preset encoding types further includes: for each preset encoding type, obtaining the matching confidence of the encoding type by weighted calculation based on at least one statistical feature value corresponding to the encoding type; wherein the weight coefficient corresponding to the statistical feature value is preset according to the encoding type or obtained by machine learning training.
[0015] In one exemplary embodiment of this application, determining a target decoding strategy based on the matching confidence level corresponding to each of the encoding types includes: when a single matching confidence level exceeds a first threshold, selecting the decoding method of the encoding type corresponding to that matching confidence level as the decoding strategy; or when multiple matching confidence levels exceed a second threshold, establishing a decoding attempt priority queue based on the high and low levels of the multiple matching confidence levels; or determining whether it is a mixed encoding based on the spatial distribution characteristics and segmenting the character stream, applying different decoding methods to different segments.
[0016] In one exemplary embodiment of this application, the method further includes: re-performing statistical feature analysis and matching confidence calculation on the decoded character stream; and repeating the decoding process when a new encoded feature is detected until a preset termination condition is met.
[0017] According to one aspect of this application, a message decoding device based on statistical feature analysis is proposed. The device includes: a message module for acquiring the original character stream of network messages; a statistics module for performing multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; a matching module for calculating the matching confidence scores of the original character stream with various preset encoding types based on the multi-dimensional feature set; a strategy module for determining a target decoding strategy according to the matching confidence scores of each encoding type; and a decoding module for performing decoding processing on the original character stream according to the target decoding strategy.
[0018] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.
[0019] According to one aspect of this application, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described above.
[0020] According to the message decoding method and apparatus based on statistical feature analysis of this application, the method involves: acquiring the original character stream of a network message; performing multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; calculating the matching confidence scores of the original character stream with various preset encoding types based on the multi-dimensional feature set; determining a target decoding strategy based on the matching confidence scores of each encoding type; and performing decoding processing on the original character stream according to the target decoding strategy. This approach can effectively address encoding variations and deliberate circumvention techniques, improve encoding recognition accuracy, and effectively enhance decoding efficiency in complex encoding scenarios.
[0021] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description
[0022] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0023] Figure 1 This is a flowchart illustrating a message decoding method based on statistical feature analysis according to an exemplary embodiment.
[0024] Figure 2 This is a flowchart illustrating a message decoding method based on statistical feature analysis according to another exemplary embodiment.
[0025] Figure 3 This is a schematic diagram illustrating a message decoding method based on statistical feature analysis according to another exemplary embodiment.
[0026] Figure 4 This is a block diagram illustrating a message decoding apparatus based on statistical feature analysis according to an exemplary embodiment.
[0027] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment.
[0028] Figure 6 This is a block diagram illustrating a computer-readable medium according to an exemplary embodiment. Detailed Implementation
[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.
[0030] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0033] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.
[0034] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.
[0035] In the existing technology, there is a network packet decoding method that mainly relies on rule-based feature matching. The basic process of this method includes: first, performing protocol parsing on the character stream to be decoded to determine the character stream to be decoded; then, matching the character stream to be decoded with a set of feature characters for multiple preset encoding methods, where each character set corresponds to one encoding method; finally, determining the encoding features of the character stream to be decoded based on the matching results. If the character stream to be decoded corresponds to multiple encoding features, then determining the appropriate target decoding method based on a preset prior decoding order or HTTP context.
[0036] Existing technology is a feature matching method based on a specific, local feature character set. It is static and fixed, with poor adaptability. Attackers can easily use non-standard character sets or insert interference characters to make the message unable to match the preset rule features, thereby evading detection. In complex situations with multiple possible encodings, this technology either relies on preset decoding priorities or requires additional context information, making it difficult to adapt to all the changing actual encoding combinations. This may result in the decoding path selection not being optimal, requiring multiple attempts and affecting decoding efficiency.
[0037] In view of the problems of low decoding efficiency, insufficient accuracy of encoding recognition, and lack of quantitative basis for decoding path decision-making in existing technologies, this application provides a message decoding method based on statistical feature analysis. The core technical idea is to analyze the character statistical features of message data, calculate and evaluate the matching confidence of each encoding type based on the statistical features, perform a comprehensive evaluation, realize intelligent decoding decision-making, and finally execute decoding.
[0038] The content of this application will be described in detail below with the aid of specific embodiments.
[0039] Figure 1 This is a flowchart illustrating a message decoding method based on statistical feature analysis according to an exemplary embodiment. The message decoding method 10 based on statistical feature analysis includes at least steps S102 to S110.
[0040] like Figure 1 As shown, in S102, the raw character stream of the network packet is obtained. For example, the network packet can be parsed to determine the target transmission protocol; and the raw character stream to be decoded can be extracted from the network packet according to the target transmission protocol.
[0041] First, the received network packets are parsed to extract the character stream to be decoded based on the transport protocol used in the packets (such as HTTP, DNS, and IMAP). This step includes parsing the protocol header, separating metadata and packet payload, and finally obtaining the raw string that needs to be decoded and analyzed.
[0042] In S104, a multi-dimensional statistical feature analysis is performed on the original character stream to obtain a multi-dimensional feature set. For example, a basic character distribution statistical feature analysis can be performed on the original character stream; an encoding pattern statistical feature analysis can be performed on the original character stream; or a spatial distribution feature calculation statistical feature analysis can be performed on the original character stream.
[0043] In S106, based on the multi-dimensional feature set, the matching confidence scores of the original character stream and various preset encoding types are calculated respectively.
[0044] In one embodiment, the method further includes: for each preset encoding type, obtaining the matching confidence of the encoding type by weighted calculation based on at least one statistical feature value corresponding to the encoding type; wherein the weight coefficient corresponding to the statistical feature value is preset according to the encoding type or obtained through machine learning training.
[0045] In S108, the target decoding strategy is determined based on the matching confidence level corresponding to each of the encoding types.
[0046] For example, when a single match confidence exceeds the first threshold, the decoding method corresponding to that match confidence is selected as the decoding strategy; or when multiple match confidences exceed the second threshold, a decoding attempt priority queue is established based on the level of the multiple match confidences; or the spatial distribution characteristics are used to determine whether it is a mixed encoding and the character stream is segmented, and different decoding methods are applied to different segments.
[0047] More specifically, for each coding type, a matching confidence score can be calculated based on its unique statistical feature pattern. The confidence score calculation adopts a weighted comprehensive evaluation method, which comprehensively considers the contribution of multiple statistical features.
[0048] Taking URL encoding as an example, considering factors such as % character density, URL pattern completeness, hexadecimal proportion, and the length distribution characteristics of URL encoded fragments, the confidence score is calculated as follows: URL confidence score = w1 * %character density + w2 * URL pattern completeness + w3 * hexadecimal character ratio + w4 * length distribution characteristics Where w1, w2, w3, and w4 are the weight coefficients of each feature, and their optimal values can be obtained through machine learning training; % character density: the frequency of the % character in a character stream; URL pattern completeness: the proportion of each % symbol followed by two hexadecimal characters; Hexadecimal ratio: The hexadecimal ratio after the % character; Length distribution characteristics: the proportion of segments whose length modulo 3 has a remainder of 0 and the coefficient of variation of segment length (standard deviation / mean).
[0049] In S110, the original character stream is decoded according to the target decoding strategy.
[0050] More specifically, this could include: a single high-confidence scenario. When the confidence of a certain encoding type is significantly higher than that of other types and exceeds a preset threshold, the decoding method corresponding to that encoding type is directly selected.
[0051] More specifically, this can include scenarios with varying confidence levels. When the confidence levels of multiple encoding types all exceed a threshold, the following strategy is adopted: The encoding type with the highest confidence level is selected for priority testing; based on spatial distribution characteristics, it is determined whether it is a mixed encoding; if so, character segmentation is performed, and corresponding decoding strategies are applied to different character segments. A priority queue for decoding attempts is established, and attempts are made in descending order of confidence level. More specifically, this could include low-confidence scenarios. When the confidence of all encoding types is below the threshold, the following strategy is adopted: prioritize trying the most common encoding type, combine protocol context information to assist decision-making, and perform limited decoding attempts when performance allows.
[0052] In one embodiment, the method further includes: re-performing statistical feature analysis and matching confidence calculation on the decoded character stream; and repeating the decoding process when a new encoded feature is detected until a preset termination condition is met.
[0053] The decoded character stream is then subjected to statistical feature extraction and confidence assessment again to check whether any identifiable encoded features still exist. If new encoded features are detected, the above statistical analysis and decoding decision process is repeated, forming an iterative decoding loop.
[0054] Termination conditions (termination occurs when one of the following conditions is met): no more encoded features are detected in the decoded character stream; the preset maximum number of iterations is reached; the decoded content passes format verification (such as valid JSON, XML, etc.).
[0055] According to the message decoding method based on statistical feature analysis of this application, the original character stream of network messages is obtained; multi-dimensional statistical feature analysis is performed on the original character stream to obtain a multi-dimensional feature set; based on the multi-dimensional feature set, the matching confidence scores of the original character stream and various preset encoding types are calculated respectively; a target decoding strategy is determined according to the matching confidence scores of each encoding type; and the original character stream is decoded according to the target decoding strategy. This method can effectively cope with encoding variants and deliberate circumvention techniques, improve the accuracy of encoding recognition, and effectively improve decoding efficiency in complex encoding scenarios.
[0056] It should be clearly understood that this application describes how specific examples are formed and used, but the principles of this application are not limited to any details of these examples. Rather, based on the teachings of the disclosure of this application, these principles can be applied to many other embodiments.
[0057] Figure 2 This is a flowchart illustrating a message decoding method based on statistical feature analysis according to another exemplary embodiment. Figure 2 The process shown in step 20 is... Figure 1 The flowchart shown describes in detail S104 "Perform multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set".
[0058] like Figure 2 As shown, in S202, basic character distribution statistical feature analysis is performed on the original character stream. For example, the frequency of occurrence of each character in the original character stream is counted to generate a character frequency distribution; the proportion of at least one character category in the original character stream is calculated, and the character category includes at least one of the following: hexadecimal characters, Base64 character set characters, special symbol characters, printable ASCII characters, and control characters or non-printable characters.
[0059] For example, one could statistically analyze the frequency of each character in the entire character stream and establish a character frequency distribution model. This could involve calculating the proportion of specific character categories within the total character set, including: the proportion of hexadecimal characters (0-9, AF, af); the proportion of Base64 encoded characters (AZ, az, 0-9, +, / , =); the proportion of special symbols; the proportion of printable ASCII characters; and the proportion of control characters and non-printable characters.
[0060] In S204, the original character stream is subjected to statistical feature analysis of the encoding pattern. For example, the occurrence ratio and distribution pattern of feature characters or feature symbols related to the encoding type in the original character stream are statistically analyzed; the integrity of the encoding pattern corresponding to the feature characters or feature symbols is analyzed; and the length distribution feature of the encoded segment corresponding to the encoding type is calculated, wherein the length distribution feature includes at least the statistical distribution or coefficient of variation of the segment length.
[0061] For example, one could calculate the distribution characteristics of various coding patterns, such as common coding methods: URL encoding: Statistically analyze the proportion and distribution patterns of the % character; analyze the frequency of occurrence of the complete URL with two hexadecimal characters following the % character; calculate the proportion of hexadecimal characters following the % character; statistically analyze the length distribution characteristics of URL-encoded segments (calculate the proportion of segments with a remainder of 0 modulo 3 and the coefficient of variation of segment length). Base64 encoding: Statistically determine the proportion of Base64 characters in the overall character stream; analyze the proportion and distribution of padding characters (=); calculate the length distribution of Base64 encoded blocks (standard Base64 encoding uses 4 characters per block); evaluate the entropy analysis of character distribution (Base64 encoding usually has a high entropy value, approaching the theoretical maximum entropy of 64 symbols log2(64) = 6 bits, and the character distribution is extremely uniform; the entropy calculation formula is H = -Σ(p_i * log2(p_i)), where p_i is the probability of the i-th symbol appearing, log2 is the logarithm to the base 2, and the unit of entropy is bits). HTML entity encoding: Statistics on character usage ratios and distribution patterns; analysis of the ratio of numeric entities (decimal and hexadecimal) to named entities (two methods for safely writing reserved characters, invisible characters, and arbitrary Unicode characters into HTML source code); statistics on entity terminators; the occurrence ratio of certain characters; calculation of the distribution characteristics of HTML entity lengths. In S206, spatial distribution feature calculation and statistical feature analysis are performed on the original character stream. For example, the position information of feature characters in the original character stream can be obtained; the uniformity of the distribution of feature characters in the character stream can be calculated based on the position information; the original character stream can be segmented and statistically analyzed to calculate the dispersion of the frequency of occurrence of feature characters in each segment; and the statistical measure of the interval between adjacent feature characters can be calculated.
[0062] For example, to calculate the uniformity of the location distribution of a feature: calculate the intervals between adjacent locations in the location sequence, and then calculate the coefficient of variation (i.e., the ratio of the standard deviation to the mean) of these intervals. The smaller the coefficient of variation, the more uniform the distribution; conversely, the larger the coefficient of variation, the more uneven the distribution.
[0063] Analyze the spatial statistical characteristics of the clustering of characteristic characters: Divide the message into segments of fixed length, count the occurrences of characteristic characters in each segment, and then calculate the standard deviation of these occurrences. The larger the standard deviation, the more concentrated the distribution of characteristic characters; conversely, the smaller the standard deviation, the more dispersed the distribution.
[0064] Calculate the distribution pattern of intervals between adjacent characteristic characters: Calculate the minimum, maximum, mean, median, and standard deviation of the intervals. These statistics can be used to determine the distribution characteristics of the intervals. Figure 3 This is a schematic diagram illustrating a message decoding method based on statistical feature analysis according to another exemplary embodiment. Figure 3 The process shown in step 30 is... Figure 1 A detailed description of the process shown.
[0065] like Figure 3As shown, in S302, the raw character stream is extracted. Specifically, the received network packet is parsed to identify the corresponding transmission protocol, and the raw character stream to be decoded is extracted from the parsed packet payload as the input object for subsequent statistical analysis and decoding processing.
[0066] In S304, statistical feature analysis is performed. By conducting multi-dimensional statistical analysis on the original character stream, statistical feature information characterizing its encoding properties is extracted.
[0067] More specifically, this could include: basic character distribution statistics.
[0068] For example, the frequency of occurrence of each character in the original character stream is statistically analyzed, and the proportion of different character categories in the overall character stream is calculated to reflect the overall characteristics of character distribution.
[0069] More specifically, this could include: encoding pattern statistics.
[0070] For example, for a variety of preset encoding types, the occurrence ratio, distribution pattern, and length distribution characteristics of the characteristic characters or symbols corresponding to each encoding type are statistically analyzed to characterize the statistical characteristics of different encoding modes.
[0071] More specifically, this may include: spatial distribution characteristic calculation.
[0072] For example, obtain the position information of feature characters in the character stream, and analyze the uniformity of the distribution of feature characters in the character stream, the degree of clustering, and the distribution characteristics of the interval between adjacent feature characters.
[0073] In S306, confidence assessment is performed. Based on the above statistical feature analysis results, the matching confidence scores of the original character stream and various preset encoding types are calculated respectively. The matching confidence scores are used to characterize the probability that the original character stream adopts the corresponding encoding type.
[0074] In S308, the decoding strategy is as follows: Based on the matching confidence level corresponding to each encoding type, a target decoding strategy is selected for the current raw character stream.
[0075] More specifically, this could include: a single high-confidence scenario.
[0076] When the matching confidence of a certain encoding type is significantly higher than that of other encoding types and exceeds a preset threshold, the decoding method corresponding to that encoding type is directly selected as the decoding strategy.
[0077] More specifically, this could include scenarios with varying levels of confidence.
[0078] When the matching confidence of multiple encoding types exceeds the preset threshold, the decoding priority is determined according to the order of matching confidence, or the spatial distribution characteristics are combined to determine whether it is a mixed encoding, and the original character stream is segmented when it is determined to be a mixed encoding.
[0079] More specifically, this could include low-confidence scenarios.
[0080] When the matching confidence of each encoding type is lower than the preset threshold, a preset default decoding strategy or a limited trial decoding strategy can be used for processing.
[0081] In S310, decoding is performed. A decoding operation is executed on the original character stream or the segmented character segments to obtain a decoded character stream; if necessary, statistical feature analysis and decoding processing can be performed again on the decoded character stream.
[0082] The beneficial effects of this application are: by using statistical feature analysis and confidence assessment, the number of decoding attempts is reduced, effectively improving decoding efficiency in complex encoding scenarios.
[0083] Identification based on statistical feature distribution rather than fixed rules can effectively deal with code variants and deliberate circumvention techniques, thereby improving the accuracy of code identification.
[0084] A probabilistic confidence assessment mechanism is introduced to provide a decision-making basis for the decoding strategy based on the characteristics of the message itself.
[0085] Those skilled in the art will understand that all or part of the steps of the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, it performs the functions defined by the method provided in this application. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.
[0086] Furthermore, it should be noted that the above figures are merely illustrative representations of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.
[0087] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0088] Figure 4 This is a block diagram illustrating a message decoding apparatus based on statistical feature analysis according to an exemplary embodiment. Figure 4As shown, the message decoding device 40 based on statistical feature analysis includes: a message module 402, a statistics module 404, a matching module 406, a strategy module 408, and a decoding module 410.
[0089] The message module 402 is used to obtain the raw character stream of the network message; the message module 402 is also used to perform protocol parsing on the network message to determine the target transmission protocol; and to extract the raw character stream to be decoded from the network message according to the target transmission protocol.
[0090] The statistics module 404 is used to perform multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; the statistics module 404 is also used to perform basic character distribution statistical feature analysis on the original character stream; the statistics module 404 is also used to perform encoding pattern statistical feature analysis on the original character stream; the statistics module 404 is also used to perform spatial distribution feature calculation statistical feature analysis on the original character stream.
[0091] The matching module 406 is used to calculate the matching confidence of the original character stream with various preset encoding types based on the multi-dimensional feature set; the matching module 406 is also used to obtain the matching confidence of each preset encoding type by weighted calculation based on at least one statistical feature value corresponding to the encoding type; wherein the weight coefficient corresponding to the statistical feature value is preset according to the encoding type or obtained by machine learning training.
[0092] The strategy module 408 is used to determine the target decoding strategy based on the matching confidence level corresponding to each of the encoding types; the strategy module 408 is also used to select the decoding method of the encoding type corresponding to the matching confidence level as the decoding strategy when a single matching confidence level exceeds a first threshold; or to establish a decoding attempt priority queue based on the high and low levels of the multiple matching confidence levels when multiple matching confidence levels exceed a second threshold; or to determine whether it is a mixed encoding based on the spatial distribution characteristics and segment the character stream, and apply different decoding methods to different segments.
[0093] The decoding module 410 is used to perform decoding processing on the original character stream according to the target decoding strategy. The decoding module 410 is also used to re-perform statistical feature analysis and matching confidence calculation on the decoded character stream; when a new encoded feature is detected, the decoding process is repeated until a preset termination condition is met.
[0094] According to the message decoding device based on statistical feature analysis of this application, the device acquires the original character stream of network packets; performs multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; calculates the matching confidence scores of the original character stream with various preset encoding types based on the multi-dimensional feature set; determines a target decoding strategy based on the matching confidence scores of each encoding type; and performs decoding processing on the original character stream according to the target decoding strategy. This method can effectively cope with encoding variants and deliberate circumvention techniques, improve encoding recognition accuracy, and effectively improve decoding efficiency in complex encoding scenarios.
[0095] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment.
[0096] The following reference Figure 5 To describe an electronic device 500 according to this embodiment of the present application. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0097] like Figure 5 As shown, the electronic device 500 is presented in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), a display unit 540, etc.
[0098] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in this specification according to various exemplary embodiments of this application. For example, the processing unit 510 can perform actions such as... Figure 1 , Figure 2 , Figure 3 The steps are shown in the figure.
[0099] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 5201 and / or a cache storage unit 5202, and may further include a read-only memory unit (ROM) 5203.
[0100] The storage unit 520 may also include a program / utility 5204 having a set (at least one) program module 5205, such program module 5205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0101] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0102] Electronic device 500 can also communicate with one or more external devices 500' (e.g., keyboard, pointing device, Bluetooth device, etc.), enabling users to communicate with devices that interact with electronic device 500, and / or any device (e.g., router, modem, etc.) that allows electronic device 500 to communicate with one or more other computing devices. This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. Network adapter 560 can communicate with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0103] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 6 As shown, the technical solution according to the embodiments of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of this application.
[0104] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0105] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0106] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0107] The aforementioned computer-readable medium carries one or more programs. When these programs are executed by a device, the computer-readable medium performs the following functions: acquiring the raw character stream of network packets; performing multi-dimensional statistical feature analysis on the raw character stream to obtain a multi-dimensional feature set; calculating the matching confidence scores of the raw character stream with various preset encoding types based on the multi-dimensional feature set; determining a target decoding strategy based on the matching confidence scores of each encoding type; and performing decoding processing on the raw character stream according to the target decoding strategy.
[0108] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0109] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.
[0110] Exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.
Claims
1. A message decoding method based on statistical feature analysis, characterized in that, include: Obtain the raw character stream of network packets; Perform multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; Based on the multi-dimensional feature set, the matching confidence scores of the original character stream and various preset encoding types are calculated respectively; The target decoding strategy is determined based on the matching confidence level corresponding to each of the aforementioned encoding types; Decoding is performed on the original character stream according to the target decoding strategy.
2. The method as described in claim 1, characterized in that, Obtain the raw character stream of network packets, including: Perform protocol parsing on network packets to determine the target transmission protocol; The original character stream to be decoded is extracted from the network message according to the target transmission protocol.
3. The method as described in claim 1, characterized in that, Perform multi-dimensional statistical feature analysis on the original character stream, including: Perform basic character distribution statistical feature analysis on the original character stream; and / or Perform statistical feature analysis of the encoding pattern on the original character stream; and / or Spatial distribution characteristics are calculated and statistical features are analyzed on the original character stream.
4. The method as described in claim 3, characterized in that, The original character stream is subjected to basic character distribution statistical feature analysis, including: The frequency of each character in the original character stream is counted to generate a character frequency distribution; Calculate the proportion of at least one character category in the original character stream, wherein the character category includes at least one of the following: hexadecimal characters, Base64 character set characters, special symbol characters, printable ASCII characters, and control characters or non-printable characters.
5. The method as described in claim 3, characterized in that, The original character stream is subjected to statistical feature analysis of encoding patterns, including: The proportion and distribution pattern of characteristic characters or symbols related to the encoding type in the original character stream were statistically analyzed. Analyze the integrity of the encoding pattern corresponding to the feature character or feature symbol; Calculate the length distribution characteristics of the encoded segments corresponding to the encoding type, wherein the length distribution characteristics include at least the statistical distribution or coefficient of variation of the segment length.
6. The method as described in claim 3, characterized in that, The original character stream is subjected to spatial distribution feature calculation and statistical feature analysis, including: Obtain the position information of the feature characters in the original character stream; The uniformity of the distribution of the feature characters in the character stream is calculated based on the location information; The original character stream is segmented and statistically analyzed to calculate the dispersion of the frequency of occurrence of characteristic characters in each segment; Calculate the statistics of the interval between adjacent feature characters.
7. The method as described in claim 1, characterized in that, The calculation of the matching confidence scores between the original character stream and various preset encoding types also includes: For each preset coding type, the matching confidence of the coding type is obtained by weighted calculation based on at least one statistical feature value corresponding to the coding type. The weight coefficients corresponding to the statistical feature values are preset according to the encoding type or obtained through machine learning training.
8. The method as described in claim 1, characterized in that, Based on the matching confidence corresponding to each of the aforementioned encoding types, the target decoding strategy is determined, including: When a single match confidence score exceeds the first threshold, the decoding method corresponding to that match confidence score is selected as the decoding strategy; or When multiple matching confidence levels exceed the second threshold, a decoding attempt priority queue is established based on the relative confidence levels of these multiple matching levels; or Based on the spatial distribution characteristics, it is determined whether it is a mixed encoding and the character stream is segmented, and different decoding methods are applied to different segments.
9. The method as described in claim 1, characterized in that, Also includes: Statistical feature analysis and matching confidence calculation are re-performed on the decoded character stream; When a new encoded feature is detected, the decoding process is repeated until the preset termination condition is met.
10. A message decoding method based on statistical feature analysis, characterized in that, include: The message module is used to obtain the raw character stream of network messages; The statistics module is used to perform multi-dimensional statistical feature analysis on the original character stream to obtain a multi-dimensional feature set; The matching module is used to calculate the matching confidence of the original character stream with various preset encoding types based on the multi-dimensional feature set. The strategy module is used to determine the target decoding strategy based on the matching confidence corresponding to each of the encoding types. The decoding module is used to perform decoding processing on the original character stream according to the target decoding strategy.