A safety warning generation method, apparatus, device, and medium

Abstract

The application discloses a security alarm generation method and device, equipment and medium, and relates to the technical field of network security, comprising: preprocessing the network resource access address of the network security log to obtain a target path segment; performing subword segmentation on the target path segment by using a target vocabulary to obtain a subword sequence; the target vocabulary is constructed based on the subword pair with the highest frequency of occurrence in the historical path segment, which is composed of adjacent subwords; matching the subword sequence with the feature representation of each existing group, and determining the group to which the network security log belongs according to the matching result; and aggregating the network security logs in the same group to obtain an aggregated security alarm. The application solves the problem that the network resource access addresses of the same business scenario present differentiated characteristics due to the random content contained in the path, which leads to the inability to effectively identify the inherent business correlation, and at the same time, the analysis burden of the security operation personnel is reduced.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method, apparatus, device and medium for generating security alerts. Background Technology

[0002] HTTP (Hypertext Transfer Protocol) is the mainstream interaction method for web services, and also a major entry point for attackers to launch network attacks. Security devices such as Web Application Firewalls (WAFs) generate massive amounts of network security alerts by monitoring HTTP traffic in real time. To reduce the workload of security operations personnel, security operations platforms typically aggregate and display similar security alerts.

[0003] In traditional alert aggregation solutions, security systems often rely on fixed URL (Uniform Resource Locator) characteristics, static rules, or simple string matching methods to identify similar business scenarios. However, modern web applications, in order to improve their dynamism, security, or optimize user experience, frequently introduce randomly generated content into URL paths, such as dynamic session IDs, randomly named resource files, and anti-crawler random parameters. This random content causes URLs belonging to the same business scenario to exhibit differentiated characteristics, making it impossible for traditional solutions to effectively identify their inherent business relationships. This leads to alert fragmentation: a large number of essentially similar security alerts are split and displayed, significantly increasing the analytical burden on security operations personnel. Therefore, these problems urgently need to be addressed by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a security alarm generation method, apparatus, device, and medium to solve the problem that network resource access addresses in the same business scenario exhibit differentiated characteristics due to the inclusion of random content in the path, making it impossible to effectively identify their inherent business correlation, while also reducing the analysis burden on security operations personnel.

[0005] To address the above problems, this application provides a method for generating security alarms, including: The network resource access addresses of newly generated network security logs are obtained in real time, and the network resource access addresses are preprocessed to obtain target path fragments; The target path segment is segmented into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The feature representations of each existing group are determined, and the word sequence is matched with each of the feature representations to obtain the target matching result. Then, the group to which the network security log belongs is determined based on the target matching result. Aggregate the network security logs of the same group to obtain aggregated security alerts.

[0006] Optionally, the preprocessing of the network resource access address to obtain the target path segment includes: The protocol header field used to identify the transmission protocol, the domain name field used to locate the server, and the query parameter field used to transmit request information are removed from the network resource access address to obtain the path part; The path portion is segmented to obtain the target path segment.

[0007] Optionally, matching the sub-word sequence with each of the feature representations to obtain the target matching result includes: Calculate the similarity between the word sequence and each of the feature representations; The similarity between the sub-word sequence and each of the feature representations is taken as the target matching result between the sub-word sequence and each of the feature representations.

[0008] Optionally, determining the group to which the network security log belongs based on the target matching result includes: If there is only one feature representation with a similarity greater than a preset similarity threshold, then the network security log is divided into the group corresponding to the feature representation; If there are multiple feature representations with a similarity greater than the preset similarity threshold, then the network security logs are divided into the group corresponding to the feature representation with the highest similarity. If no feature representation has a similarity greater than the preset similarity threshold, a new group is created for the network security log, and the sub-word sequence is used as the feature representation of the new group.

[0009] Optionally, determining the feature representation of each existing group includes: For each existing group, the sequence of sub-words corresponding to the first network security log assigned to that group is used as the feature representation of that group.

[0010] Optionally, calculating the similarity between the sub-word sequence and each of the feature representations includes: For each of the aforementioned feature representations, calculate the length of the longest common substring between the word sequence and the feature representation; Determine the length of the shorter sequence in the feature representation compared to the sub-word sequence; The similarity between the word sequence and the feature representation is determined based on the ratio of the length of the longest common substring to the length of the shorter sequence.

[0011] Optionally, the security alarm generation method further includes: Add all sub-words from the historical path segment to the basic vocabulary; The frequency of occurrence of word pairs consisting of adjacent words in the historical path segment is counted, and the word pairs with the highest frequency are merged into new words. Then, the basic vocabulary is updated using the new words. The process proceeds to the step of counting the occurrence frequency of sub-word pairs until the size of the current vocabulary reaches a preset size threshold, or the highest occurrence frequency of a sub-word pair is lower than a preset frequency threshold, at which point the target vocabulary is obtained.

[0012] This application also provides a security alarm generation device, including: The path fragment extraction module is used to obtain the network resource access address of newly generated network security logs in real time, preprocess the network resource access address, and obtain the target path fragment. The sub-word sequence generation module is used to segment the target path segment into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The log grouping determination module is used to determine the feature representation of each existing group, match the word sequence with each feature representation to obtain the target matching result, and then determine the group to which the network security log belongs based on the target matching result; The aggregated alarm generation module is used to aggregate the network security logs of the same group to obtain aggregated security alarms.

[0013] This application also provides a security alarm generation device, including: Memory, used to store computer programs; A processor is configured to implement the steps of the security alert generation method described above when executing the computer program.

[0014] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the security alarm generation method described above.

[0015] This application provides a method for generating security alerts, comprising: acquiring network resource access addresses of newly generated network security logs in real time; preprocessing the network resource access addresses to obtain target path segments; segmenting the target path segments into sub-words using a target vocabulary to obtain corresponding sub-word sequences; wherein the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; determining the feature representations of existing groups; matching the sub-word sequences with each of the feature representations to obtain target matching results; determining the group to which the network security log belongs based on the target matching results; and aggregating the network security logs in the same group to obtain aggregated security alerts.

[0016] In summary, this application first preprocesses network resource access addresses to remove invalid information, retaining only the core path portion. Then, using a target vocabulary built based on historical path fragments, it segments the target path fragments into sub-word sequences. This vocabulary specifically selects high-frequency sub-word pairs composed of adjacent sub-words, enabling accurate extraction of business-related sub-word sequences from paths containing random content. This overcomes the limitations of traditional solutions, where network resource access addresses within the same business scenario exhibit differentiated characteristics due to the presence of random content in the paths, making it difficult to effectively identify their inherent business relevance. Furthermore, after determining the feature representations of existing groups, the sub-word sequences are matched with these feature representations, and network security logs are grouped according to the target matching results. This ensures that network security logs corresponding to essentially similar network resource access addresses are grouped together, significantly improving the accuracy of security alert aggregation. Finally, network security logs in the same group are aggregated to obtain aggregated security alerts, greatly reducing the number of fragmented alerts, effectively lowering the workload and analytical complexity of security operations, and alleviating the analytical burden on security operations personnel.

[0017] This application also provides a security alarm generation device, equipment, and medium, which have the above-mentioned effects, and will not be elaborated here. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a security alarm generation method provided in this application embodiment; Figure 2A schematic diagram of a security alarm generation device provided in an embodiment of this application; Figure 3 This is a schematic diagram of a security alarm generation device provided in an embodiment of this application. Detailed Implementation

[0020] The core of this application is to provide a security alarm generation method, device, equipment, and medium, which solves the problem that network resource access addresses in the same business scenario exhibit differentiated characteristics due to the inclusion of random content in the path, making it impossible to effectively identify their inherent business correlations, while also reducing the analysis burden on security operations personnel.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In traditional alert aggregation solutions, security systems often rely on fixed URL characteristics, static rules, or simple string matching methods to identify similar business scenarios. However, modern web applications, in order to enhance their dynamism, security, or optimize user experience, frequently introduce randomly generated content into URL paths, such as dynamic session IDs, randomly named resource files, and anti-crawler random parameters. This random content causes URLs that originally belong to the same business scenario to exhibit differentiated characteristics, making it impossible for traditional solutions to effectively identify their inherent business relationships. This leads to alert fragmentation: a large number of essentially similar security alerts are split and displayed, significantly increasing the analytical burden on security operations personnel.

[0023] Therefore, this application first preprocesses the network resource access addresses to remove invalid information, retaining only the core path portion. Then, using a target vocabulary constructed based on historical path fragments, the path portion is segmented into sub-word sequences. This vocabulary specifically selects high-frequency sub-word pairs composed of adjacent sub-words, which can accurately extract business-related sub-word sequences from paths containing random content. This solves the limitation of traditional solutions, namely, that network resource access addresses in the same business scenario exhibit differentiated characteristics due to the inclusion of random content in the paths, making it impossible to effectively identify their inherent business relevance. Furthermore, after determining the feature representations of existing groups, the sub-word sequences are matched with these feature representations, and then the network security logs are grouped according to the target matching results. This ensures that network security logs corresponding to essentially similar network resource access addresses are grouped into the same group, significantly improving the accuracy of security alert aggregation. Finally, network security logs in the same group are aggregated to obtain aggregated security alerts, greatly reducing the number of fragmented alerts, effectively reducing the workload and analysis complexity of security operations, and alleviating the analytical burden on security operations personnel.

[0024] The following embodiment illustrates a security alarm generation method provided in this application.

[0025] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a security alarm generation method provided in an embodiment of this application.

[0026] In this embodiment, the method may include: S101: Obtain the network resource access address of the newly generated network security log in real time, preprocess the network resource access address, and obtain the target path fragment.

[0027] In this embodiment, network security logs refer to the continuous records of various security-related events during the operation of network devices, systems, applications, etc. They record in detail the time of the event, source IP (Internet Protocol) address, destination IP address, port number, user account, operation type (such as login attempt, file access, permission change, traffic anomaly, etc.), execution result (success or failure), and other key information. They are an important data source for analyzing network behavior, tracing security events, and identifying potential threats (such as hacker attacks, malware infections, etc.).

[0028] In this embodiment, the protocol header field used to identify the transmission protocol, the domain name field used to locate the server, and the query parameter field used to transmit request information are removed from the network resource access address to obtain the path portion. This removes redundant information unrelated to the core path portion, focusing on the core path portion of the network resource access address. Further, the path portion is segmented based on a path separator (such as " / ") to obtain the corresponding target path segments. For example, if the network resource access address is "https: / / domain.com / abc / defxyz?p=1", after removing the protocol header field "https: / / ", the domain name field "domain.com", and the query parameter field "p=1", the path portion " / abc / defxyz" is obtained. This is then segmented based on the path separator " / " to obtain the target path segments "abc" and "defxyz".

[0029] S102: The target path segment is segmented into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in the historical path segment.

[0030] In this embodiment, all sub-words in the historical path segment are added to the basic vocabulary; the frequency of occurrence of sub-word pairs composed of adjacent sub-words in the historical path segment is counted, and the sub-word pair with the highest frequency is merged into a new sub-word, and then the basic vocabulary is updated using the new sub-word; the process jumps to the step of counting the frequency of occurrence of sub-word pairs until the size of the current vocabulary reaches a preset size threshold, or the highest frequency of occurrence of a sub-word pair is lower than a preset frequency threshold, then the target vocabulary is obtained.

[0031] In summary, the construction of the target vocabulary is based on the BPE (Byte Pair Encoding) algorithm, a statistical unsupervised word segmentation algorithm. Its construction process strictly follows this logic: First, all words in the historical path segments are added to the basic vocabulary; then, the frequency of occurrence of word pairs composed of adjacent words in the historical path segments is counted, and the word pair with the highest frequency is merged into a new word, which is then used to update the basic vocabulary; subsequently, the step of counting the frequency of word pairs is repeated until the size of the current vocabulary reaches a preset threshold, or the highest frequency of a word pair is lower than the preset frequency threshold, at which point the iteration stops and the target vocabulary is obtained. It should be noted that the historical path segments are obtained as follows: the network resource access addresses of historical network security logs are determined, these addresses are preprocessed to obtain the path portion, and the path portion is segmented to obtain the corresponding historical path segments; the preprocessing is described in the aforementioned disclosures and will not be repeated here.

[0032] In this way, by leveraging the core mechanism of the BPE algorithm to iteratively merge high-frequency adjacent sub-word pairs, the target vocabulary can accurately extract the sub-word sequences that reflect business associations in the path. This process breaks through the limitations of traditional solutions that rely on fixed URL features, static rules, or simple string matching, and solves the problem that network resource access addresses in the same business scenario exhibit differentiated characteristics due to the presence of random content in the path, resulting in the inability to effectively identify their inherent business associations.

[0033] In this embodiment, the target path segment is segmented into sub-words using a target vocabulary to obtain the corresponding sub-word sequence. For each target path segment, the target vocabulary is used to segment the sub-words one by one to obtain semantically continuous sub-word units. Then, sub-word units with a length less than a preset threshold (mostly random interference information without actual business semantics) are filtered out, and the longest sub-word unit in each target path segment is selected as the core sub-word unit of that segment. Finally, the core sub-word units corresponding to all target path segments are arranged in order of their sequence in the original path to form a complete sub-word sequence.

[0034] S103: Determine the feature representations of each existing group, and match the word sequence with each feature representation to obtain the target matching result. Determine the group to which the network security log belongs based on the target matching result.

[0035] In this embodiment, the similarity between the sub-word sequence and each feature representation is calculated; the similarity between the sub-word sequence and each feature representation is used as the target matching result between the sub-word sequence and each feature representation. Further, if only one feature representation has a similarity greater than a preset similarity threshold, the network security log is assigned to the group corresponding to that feature representation; if multiple feature representations have a similarity greater than the preset similarity threshold, the network security log is assigned to the group corresponding to the feature representation with the highest similarity; if no feature representation has a similarity greater than the preset similarity threshold, a new group is created for the network security log, and the sub-word sequence is used as the feature representation of the new group.

[0036] In one specific implementation, for each existing group, the word sequence corresponding to the first network security log assigned to that group is used as the feature representation of that group. Calculating the similarity between the word sequence and each feature representation then includes: for each feature representation, calculating the length of the longest common substring between the word sequence and the feature representation; determining the length of the shorter sequence between the word sequence and the feature representation; and determining the similarity between the word sequence and the feature representation based on the ratio of the longest common substring length to the shorter sequence length.

[0037] This can be expressed by the formula: ; Where len represents the length of the sequence to be solved, lcs represents the longest common substring of the two sequences; s1 represents the sub-word sequence, and s2 represents the feature representation of the group (i.e., the sub-word sequence corresponding to the first network security log assigned to the group). This indicates the similarity between the word sequence and the feature representation.

[0038] In other words, existing groups all use the word sequence corresponding to the first cybersecurity log to be assigned to that group as their feature representation (i.e., the group cluster head). Newly generated cybersecurity logs only need to be compared for similarity with the feature representations of each group cluster head; there's no need to traverse all logs within a group. Specifically, this decision logic achieves accurate and efficient grouping through three layers of rules: if only one feature representation has a similarity greater than a preset similarity threshold, the cybersecurity log is assigned to the group corresponding to that feature representation; if multiple feature representations have similarities greater than the preset similarity threshold, the cybersecurity log is assigned to the group corresponding to the feature representation with the highest similarity; if no feature representation has a similarity greater than the preset similarity threshold, a new group is created for the cybersecurity log, and the word sequence is used as the feature representation of the new group. In this way, this embodiment significantly reduces the computational load of similarity comparison and improves the real-time processing efficiency of cybersecurity log grouping.

[0039] Assuming that the preset similarity threshold of Group 1 already exists is 0.6, and a newly generated network security log A is generated, the similarity between the word sequence of network security log A and the feature representation of Group 1 is calculated. If the similarity is 0.5, then Group 2 is created for network security log A. Later, a newly generated network security log B is generated. The similarity between the word sequence of network security log B and the feature representation of Group 1 is calculated. If the similarity is 1, then network security log B is assigned to Group 1.

[0040] S104: Aggregate the network security logs of the same group to obtain aggregated security alerts.

[0041] In this embodiment, security alerts are threats detected by the security system (e.g., a cloud security center) that exist in the network or system. These are typically malicious attacks, intrusions, or abnormal activities targeting assets. Such alerts usually include detailed information about the threat, such as the malicious IP address, the attack result (success, attempt, etc.), and the alert level (critical, high-risk, etc.), so that security personnel can quickly identify and respond to these security events. Based on this, the specific logic of the aggregation operation is as follows: First, search for historically generated aggregated security alerts and verify whether all of the following conditions are met: First, the grouping is consistent; that is, the group to which the current network security log belongs, determined by the sub-word sequence similarity judgment rule described above, is the same group as the group corresponding to the historical aggregated security alert (network resource access addresses within the same group all correspond to similar business scenarios); second, the threat type is consistent; and third, the corresponding source IP address or target IP address determined according to the threat type is consistent. If a historical aggregated security alert that meets all the above conditions exists, the current network security log will be merged into that historical aggregated security alert, and relevant statistical information such as the number of alert triggers and the most recent occurrence time will be updated synchronously. If no historical aggregated security alert that meets the conditions is found, a new aggregated security alert will be generated based on key information such as the group to which the current network security log belongs, the threat type, the corresponding source / destination IP address, and the core description of the event. Through this aggregation logic, security alerts with the same business scenario and the same threat attribute can be accurately aggregated, reducing the fragmentation problem of alerts caused by random content in the path of network resource access addresses.

[0042] The following points will explain this application: I. Overall Overview of the Technical Solution of this Application This application discloses a security alert aggregation scheme based on network resource access addresses (i.e., URLs) containing random paths, using the BPE algorithm. The core logic of this scheme is as follows: First, a baseline is established for the network resource access address characteristics of user services. Then, a suitable BPE vocabulary (i.e., a target vocabulary) is periodically generated through BPE algorithm learning. For newly received HTTP network security logs, target logs are first filtered, and their network resource access addresses are obtained in real time. These addresses are then preprocessed by removing protocol header fields used to identify the transmission protocol, domain name fields used to locate the server, and query parameter fields used to transmit request information, resulting in a path portion that focuses on core features. The path portion is then segmented based on path separators (such as " / ") to obtain corresponding target path segments. Subsequently, each target path segment is segmented using the trained BPE vocabulary, filtering out sub-word units shorter than a preset threshold, and selecting the longest sub-word unit in each segment as the core sub-word unit. These sub-word units are arranged in the original path order to form a sub-word sequence. Next, combining dimensions such as IP address and threat type, the similarity between different logs is calculated using the longest common substring algorithm. Logs with similarity higher than a preset threshold are grouped, and finally, security alerts are generated based on the grouping results. By leveraging the word-level feature extraction capability of the BPE algorithm, this application can effectively separate valid semantic words from random interference words in network resource access address paths, accurately identify alarms corresponding to similar network resource access addresses containing random paths, solve the problem of alarm dispersion and fragmentation, improve the efficiency and accuracy of alarm aggregation, reduce the analysis pressure on security operation and maintenance personnel, and is applicable to scenarios such as security operation and maintenance centers and scalable detection and response platforms. Taking "https: / / abc.xyz / task / uploadResult / 123456 / jt18digshell0.jsp?p1=x&p2=z" as an example, after preprocessing, the path part " / task / uploadresult / 000000 / jt00digshell0.jsp" is obtained. After segmentation, the target path fragments "task", "uploadresult", "000000", and "jt00digshell0.jsp" are obtained. Further, after word segmentation based on the target vocabulary, "[task,uploadresult,000000,jt00,digshell0,jsp]" are obtained. Finally, the sub-word sequence is formed by arranging them in the original path order: "[task,uploadresult,000000,digshell,jsp]".

[0043] II. The technical solution of this application is achieved through the coordinated operation of four functional modules.

[0044] (1) BPE vocabulary learning module: continuously collect network resource access addresses from user network security logs, perform operations consistent with subsequent log preprocessing, and periodically train the BPE vocabulary using the processed path parts: initially add all characters in the path to the basic vocabulary, divide the path based on the path delimiter to obtain historical path segments, count the frequency of sub-words in the historical path segments, iteratively merge the most frequent consecutive sub-word pairs and update the vocabulary until the vocabulary size reaches the preset scale threshold or the highest frequency of sub-word pairs is lower than the preset threshold, and output the trained BPE vocabulary (i.e., the target vocabulary). (2) Similar URL determination module: The similarity of the network resource access address corresponding to the new log is calculated with the help of the trained BPE vocabulary. The network resource access address of the new log is preprocessed and path segmented in the same way as the vocabulary training to obtain the target path fragment. Each target path fragment is segmented with the BPE vocabulary, and the longest sub-word unit of each fragment is taken as the core sub-word and formed into a sub-word sequence in order. The similarity is obtained by the similarity calculation formula mentioned above. (3) Similar URL clustering module: Real-time stream processing is used to meet the real-time requirements of log grouping. The feature of each group is represented by the sub-word sequence (i.e., the group cluster head) corresponding to the first network security log assigned to the group. If there is no group, a new group is created and the sub-word sequence of the current log is set as the cluster head. When processing new logs, only the sub-word sequences of each group cluster head are compared: If there is only one cluster head with a similarity greater than the preset threshold, the new log is assigned to the group corresponding to that cluster head; if there are multiple cluster heads with a similarity greater than the threshold, the new log is assigned to the group with the highest similarity; if there is no cluster head with a similarity greater than the threshold, a group is created for the new log and its own sub-word sequence is set as the cluster head. (4) Alarm aggregation generation module: Search for historically generated aggregated security alarms and verify whether they meet the three conditions of "consistent grouping, consistent threat type, and consistent source IP address or target IP address determined according to the threat type". If all conditions are met, the current log is merged into the historical alarm and the statistical information such as alarm trigger count and most recent occurrence time is updated synchronously. If no historical alarm that meets the conditions is found, a new aggregated security alarm is generated based on the key information such as the group to which the current log belongs, threat type, corresponding source / destination IP address, and core event description.

[0045] Furthermore, this application also includes the following modified designs or alternative solutions: those similar to the architecture or method of this application, but applied to aggregation scenarios of different types of security logs, including but not limited to file path security logs and command line security logs; those similar to the architecture or method of this application, but using different network resource access address vocabulary generation algorithms based on word frequency analysis; and those similar to the architecture or method of this application, using the same network resource access address similarity determination mechanism, but employing different grouping mechanisms. In other words, the modified designs or alternative solutions of this application mainly include the following: First, maintaining the core architecture and method logic unchanged, extending its application to aggregation scenarios of other types of security logs such as file path and command line logs; second, maintaining the overall architecture and business process consistency, but replacing the existing network resource access address vocabulary generation algorithm based on word frequency analysis with other similar word frequency analysis algorithms; and third, retaining the same network resource access address similarity determination mechanism, only adjusting the specific implementation method of log grouping. It is understood that the core architecture and method of this solution, as well as the aforementioned modified designs or alternative solutions, are all within the scope of protection of this solution.

[0046] This application provides a method for generating security alerts, comprising: acquiring network resource access addresses of newly generated network security logs in real time; preprocessing the network resource access addresses to obtain target path segments; segmenting the target path segments into sub-words using a target vocabulary to obtain corresponding sub-word sequences; wherein the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; determining the feature representations of existing groups; matching the sub-word sequences with each of the feature representations to obtain target matching results; determining the group to which the network security log belongs based on the target matching results; and aggregating the network security logs in the same group to obtain aggregated security alerts.

[0047] In summary, this application first preprocesses network resource access addresses to remove invalid information, retaining only the core path portion. Then, using a target vocabulary built based on historical path fragments, it segments the target path fragments into sub-word sequences. This vocabulary specifically selects high-frequency sub-word pairs composed of adjacent sub-words, enabling accurate extraction of business-related sub-word sequences from paths containing random content. This overcomes the limitations of traditional solutions, where network resource access addresses within the same business scenario exhibit differentiated characteristics due to the presence of random content in the paths, making it difficult to effectively identify their inherent business relevance. Furthermore, after determining the feature representations of existing groups, the sub-word sequences are matched with these feature representations, and network security logs are grouped according to the target matching results. This ensures that network security logs corresponding to essentially similar network resource access addresses are grouped together, significantly improving the accuracy of security alert aggregation. Finally, network security logs in the same group are aggregated to obtain aggregated security alerts, greatly reducing the number of fragmented alerts, effectively lowering the workload and analytical complexity of security operations, and alleviating the analytical burden on security operations personnel.

[0048] This application also provides a security alarm generation device, equipment, and medium, which have the above-mentioned effects, and will not be elaborated here.

[0049] The security alarm generation device provided in the embodiments of this application is described below.

[0050] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a security alarm generation device provided in an embodiment of this application.

[0051] In this embodiment, the device may include: The path fragment extraction module 100 is used to obtain the network resource access address of newly generated network security logs in real time, preprocess the network resource access address, and obtain the target path fragment. The sub-word sequence generation module 200 is used to segment the target path segment into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The log grouping determination module 300 is used to determine the feature representation of each existing group, match the sub-word sequence with each feature representation to obtain the target matching result, and then determine the group to which the network security log belongs based on the target matching result; The aggregated alarm generation module 400 is used to aggregate the network security logs of the same group to obtain aggregated security alarms.

[0052] Figure 3This is a schematic diagram of a security alarm generation device provided in an embodiment of this application.

[0053] This application embodiment also provides a security alarm generation device, which may include: Memory, used to store computer programs; A processor is configured to implement the steps of the security alarm generation method as described in the above embodiments when executing the computer program.

[0054] like Figure 3 The diagram shows the structural composition of a security alarm generation device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 communicate with each other via the communication bus 13.

[0055] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0056] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiments of the security alarm generation method.

[0057] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions: The network resource access addresses of newly generated network security logs are obtained in real time, and the network resource access addresses are preprocessed to obtain target path fragments; The target path segment is segmented into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The feature representations of each existing group are determined, and the word sequence is matched with each of the feature representations to obtain the target matching result. Then, the group to which the network security log belongs is determined based on the target matching result. Aggregate the network security logs of the same group to obtain aggregated security alerts.

[0058] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0059] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0060] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0061] Of course, it should be noted that, Figure 3 The structure shown does not constitute a limitation on the security alarm generation device in the embodiments of this application. In practical applications, the security alarm generation device may include devices that are more advanced than those described above. Figure 3 More or fewer components as shown, or combinations of certain components.

[0062] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the security alarm generation method described in the above embodiments.

[0063] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0065] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0066] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0067] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0068] The foregoing has provided a detailed description of a security alarm generation method, apparatus, device, and medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for generating security alarms, characterized in that, include: The network resource access addresses of newly generated network security logs are obtained in real time, and the network resource access addresses are preprocessed to obtain target path fragments; The target path segment is segmented into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The feature representations of each existing group are determined, and the word sequence is matched with each of the feature representations to obtain the target matching result. Then, the group to which the network security log belongs is determined based on the target matching result. Aggregate the network security logs of the same group to obtain aggregated security alerts.

2. The security alarm generation method according to claim 1, characterized in that, The preprocessing of the network resource access address to obtain the target path segment includes: The protocol header field used to identify the transmission protocol, the domain name field used to locate the server, and the query parameter field used to transmit request information are removed from the network resource access address to obtain the path part; The path portion is segmented to obtain the target path segment.

3. The security alarm generation method according to claim 1, characterized in that, The step of matching the sub-word sequence with each of the feature representations to obtain the target matching result includes: Calculate the similarity between the word sequence and each of the feature representations; The similarity between the sub-word sequence and each of the feature representations is taken as the target matching result between the sub-word sequence and each of the feature representations.

4. The security alarm generation method according to claim 3, characterized in that, The step of determining the group to which the network security log belongs based on the target matching result includes: If there is only one feature representation with a similarity greater than a preset similarity threshold, then the network security log is divided into the group corresponding to the feature representation; If there are multiple feature representations with a similarity greater than the preset similarity threshold, then the network security logs are divided into the group corresponding to the feature representation with the highest similarity. If no feature representation has a similarity greater than the preset similarity threshold, a new group is created for the network security log, and the sub-word sequence is used as the feature representation of the new group.

5. The security alarm generation method according to claim 4, characterized in that, The determination of the feature representations of each existing group includes: For each existing group, the sequence of sub-words corresponding to the first network security log assigned to that group is used as the feature representation of that group.

6. The security alarm generation method according to claim 5, characterized in that, The calculation of the similarity between the sub-word sequence and each of the feature representations includes: For each of the aforementioned feature representations, calculate the length of the longest common substring between the word sequence and the feature representation; Determine the length of the shorter sequence in the feature representation compared to the sub-word sequence; The similarity between the word sequence and the feature representation is determined based on the ratio of the length of the longest common substring to the length of the shorter sequence.

7. The security alarm generation method according to claim 1, characterized in that, Also includes: Add all sub-words from the historical path segment to the basic vocabulary; The frequency of occurrence of word pairs consisting of adjacent words in the historical path segment is counted, and the word pairs with the highest frequency are merged into new words. Then, the basic vocabulary is updated using the new words. The process proceeds to the step of counting the occurrence frequency of sub-word pairs until the size of the current vocabulary reaches a preset size threshold, or the highest occurrence frequency of a sub-word pair is lower than a preset frequency threshold, at which point the target vocabulary is obtained.

8. A safety alarm generation device, characterized in that, include: The path fragment extraction module is used to obtain the network resource access address of newly generated network security logs in real time, preprocess the network resource access address, and obtain the target path fragment. The sub-word sequence generation module is used to segment the target path segment into sub-words using a target vocabulary to obtain the corresponding sub-word sequence; wherein, the target vocabulary is constructed based on the most frequently occurring sub-word pairs composed of adjacent sub-words in historical path segments; The log grouping determination module is used to determine the feature representation of each existing group, match the word sequence with each feature representation to obtain the target matching result, and then determine the group to which the network security log belongs based on the target matching result; The aggregated alarm generation module is used to aggregate the network security logs of the same group to obtain aggregated security alarms.

9. A security alarm generation device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the security alarm generation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the security alarm generation method as described in any one of claims 1 to 7.

Images