Network data anomaly identification method and computer equipment

Abstract

The embodiments of the present application relate to the field of network data monitoring, and specifically to a network data anomaly identification method and computer equipment. A specific implementation of the method includes: obtaining an initial network link information set; performing a first anomaly identification process; deleting the initial network link information corresponding to the abnormal network link identification result in the initial network link information set from the initial network link information set; obtaining an initial web page information set; sensitive information extraction process; sensitive information classification model; determining the initial web page information corresponding to the sensitive category information group as abnormal web page information; category identification process to generate web page abnormal category information, and obtain a web page abnormal category information set; based on the web page abnormal category information set, generating a web page abnormal alarm information set; sending the web page abnormal alarm information set to the alarm terminal for alarm processing. This implementation improves the security of network services.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network data monitoring, and specifically to a method and computer device for identifying anomalies in network data. Background Art

[0002] For websites related to the broadcasting industry, the website security monitoring platform can monitor and identify abnormal network data based on the three core security requirements of the website: website availability, website content (e.g., sensitive information), and website vulnerability discovery, so as to monitor the website security. At present, when identifying abnormal network data in website content, the commonly used method is to identify sensitive information in web page content by extracting sensitive words.

[0003] However, in practice, it is found that when the above method is used to identify network data anomalies, the following technical problems often occur:

[0004] Only sensitive information in web page content is identified, but it is difficult to identify dark links and Trojan programs that appear in web page links, resulting in reduced accuracy of identified network anomaly data, thereby reducing the security of network services. Summary of the invention

[0005] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present application propose a network data anomaly identification method, a computer device, and a computer-readable storage medium to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present application provide a method for identifying anomalies in network data, the method comprising: in response to receiving network data anomaly identification request information sent by a server, obtaining an initial network link information set from the above-mentioned server; performing a first anomaly identification process on each initial network link information in the above-mentioned initial network link information set to generate an abnormal network link identification result, and obtaining an abnormal network link identification result set; for each abnormal network link identification result in the above-mentioned abnormal network link identification result set, in response to determining that the above-mentioned abnormal network link identification result meets a preset link anomaly condition, deleting the initial network link information in the above-mentioned initial network link information set corresponding to the above-mentioned abnormal network link identification result from the above-mentioned initial network link information set; obtaining initial web page information corresponding to each initial network link information in the deleted initial network link information set, and obtaining an initial web page information set; based on the preset A sensitive information database is provided, sensitive information extraction processing is performed on each initial web page information in the above-mentioned initial web page information set to generate a web page sensitive information group, and a web page sensitive information group set is obtained; each web page sensitive information in the above-mentioned web page sensitive information group set is input into a pre-trained sensitive information classification model to generate sensitive category information, and a sensitive category information group set is obtained; for each sensitive category information group in the above-mentioned sensitive category information group set, in response to determining that the above-mentioned sensitive category information group contains sensitive category information that meets a preset category condition, the initial web page information corresponding to the above-mentioned sensitive category information group is determined as abnormal web page information; category identification processing is performed on each determined abnormal web page information to generate web page abnormal category information, and a web page abnormal category information set is obtained; based on the above-mentioned web page abnormal category information set, a web page abnormal alarm information set is generated; and the above-mentioned web page abnormal alarm information set is sent to an alarm terminal for alarm processing.

[0008] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method described in any implementation of the first aspect is implemented.

[0009] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0010] The above-mentioned embodiments of the present application have the following beneficial effects: through the network data anomaly identification method of some embodiments of the present application, the security of network services can be improved. Specifically, the reason for the reduced security of network services is that only sensitive information in the web page content is identified, and it is difficult to identify the dark links and Trojan programs appearing in the web page links, resulting in a reduction in the accuracy of the identified network abnormal data. Based on this, the network data anomaly identification method of some embodiments of the present application, first, in response to receiving the network data anomaly identification request information sent by the server, obtains the initial network link information set from the above server. Secondly, each initial network link information in the above initial network link information set is subjected to a first abnormal identification process to generate an abnormal network link identification result, and obtains an abnormal network link identification result set. Thus, abnormal network links can be identified, and web page dark links and Trojan programs can be identified. Again, for each abnormal network link identification result in the above abnormal network link identification result set, in response to determining that the above abnormal network link identification result meets the preset link abnormality condition, the initial network link information corresponding to the above abnormal network link identification result in the above initial network link information set is deleted from the above initial network link information set. Thus, the identified abnormal web page links can be screened out from the initial web page link set. Then, the initial web page information corresponding to each initial network link information in the deleted initial network link information set is obtained to obtain the initial web page information set. Then, based on the preset sensitive information library, sensitive information extraction processing is performed on each initial web page information in the above initial web page information set to generate a web page sensitive information group to obtain a web page sensitive information group set. Thus, sensitive information in the web page content can be extracted. Then, each web page sensitive information in the above web page sensitive information group set is input into a pre-trained sensitive information classification model to generate sensitive category information to obtain a sensitive category information group set. Thus, sensitive information can be classified so as to warn sensitive information of the target category. Then, for each sensitive category information group in the above sensitive category information group set, in response to determining that there is sensitive category information that meets the preset category condition in the above sensitive category information group, the initial web page information corresponding to the above sensitive category information group is determined as abnormal web page information. Thus, the web page where the sensitive information of the target category appears can be determined as an abnormal web page. Subsequently, category identification processing is performed on each abnormal web page information determined to generate abnormal web page category information to obtain an abnormal web page category information set. Thus, the abnormal category and level information of the abnormal web page information can be obtained, so as to give an alarm for each category of abnormal web page information. Then, based on the above web page abnormal category information set, a web page abnormal alarm information set is generated. Thus, the alarm information corresponding to each category of abnormal web pages can be obtained. Finally, the above web page abnormal alarm information set is sent to the alarm terminal for alarm processing. Thus, an alarm can be given for abnormal web pages and network data.Therefore, some network data anomaly identification methods of the present application can first identify anomalies on the network links obtained from the server (for example, bad link scanning, dark link scanning, or Trojan scanning). Then, the web page content corresponding to the network link without abnormality is obtained. Next, the sensitive information in the web page information is extracted. Subsequently, the abnormal level and category of the web page are determined based on different categories of sensitive information. In this way, the accuracy and comprehensiveness of the identified network anomaly data can be improved, thereby improving the security of network services. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a flow chart of some embodiments of the network data anomaly identification method according to the present application;

[0013] Figure 2 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0014] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0015] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0020] Figure 1 The process 100 of some embodiments of the network data anomaly identification method according to the present application is shown. The network data anomaly identification method comprises the following steps:

[0021] Step 101: in response to receiving network data anomaly identification request information sent by a server, an initial network link information set is obtained from the server.

[0022] In some embodiments, the execution subject of the network data anomaly identification method may obtain the initial network link information set from the above-mentioned server through a wired connection or a wireless connection in response to receiving the network data anomaly identification request information sent by the server. Among them, the above-mentioned server may forward the above-mentioned network data anomaly identification request information to the above-mentioned execution subject in response to receiving the network data anomaly identification request information sent by the target user terminal. The above-mentioned server may be a server for storing and sending the initial network link information set. The above-mentioned target user terminal may be a terminal that wants to perform network data anomaly identification. The above-mentioned network data anomaly identification request information may indicate that the above-mentioned target user terminal wants to perform network data anomaly identification. The initial network link information in the above-mentioned initial network link information set may represent a web page link. The web page link may uniquely identify a web page. The above-mentioned web page may be a web page related to the radio and television industry.

[0023] It should be noted that the above wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0024] Step 102: Perform a first abnormality identification process on each initial network link information in the initial network link information set to generate an abnormal network link identification result, thereby obtaining an abnormal network link identification result set.

[0025] In some embodiments, the execution subject may perform a first abnormality identification process on each initial network link information in the initial network link information set to generate an abnormal network link identification result, and obtain an abnormal network link identification result set. Among them, the first abnormality identification process may be performed on each initial network link information in the initial network link information set to generate an abnormal network link identification result through a preset abnormality identification algorithm. The abnormal network link identification result may be information indicating that "an abnormality has occurred in the network link" or information indicating that "no abnormality has occurred in the network link".

[0026] As an example, the above-mentioned preset anomaly identification algorithm may include but is not limited to at least one of the following: SearchConsole algorithm, Link Checker algorithm, Libra dark link detection algorithm, AWCS (Acunetix Web Vulnerability Scanner, web vulnerability scanning) algorithm.

[0027] Step 103, for each abnormal network link identification result in the abnormal network link identification result set, in response to determining that the abnormal network link identification result meets a preset link abnormality condition, the initial network link information corresponding to the abnormal network link identification result in the initial network link information set is deleted from the initial network link information set.

[0028] In some embodiments, the execution subject may delete the initial network link information corresponding to each abnormal network link identification result in the abnormal network link identification result set from the initial network link information set in response to determining that the abnormal network link identification result meets a preset link abnormality condition. The preset link abnormality condition may be: the abnormal network link identification result is information indicating that "an abnormal network link has occurred".

[0029] Step 104 , obtaining the initial web page information corresponding to each initial network link information in the deleted initial network link information set, to obtain the initial web page information set.

[0030] In some embodiments, the execution subject may obtain the initial web page information corresponding to each initial network link information in the deleted initial network link information set to obtain the initial web page information set. The initial web page information corresponding to each initial network link information in the deleted initial network link information set may be obtained from the server. The initial web page information may represent the content of a web page corresponding to a web page link (including broadcast information of broadcast content).

[0031] Step 105 , based on a preset sensitive information library, sensitive information extraction processing is performed on each initial web page information in the initial web page information set to generate a web page sensitive information group, thereby obtaining a web page sensitive information group set.

[0032] In some embodiments, the execution subject may perform sensitive information extraction processing on each initial web page information in the initial web page information set based on a preset sensitive information library to generate a web page sensitive information group to obtain a web page sensitive information group set. The preset sensitive information library may be a pre-set database for storing sensitive information.

[0033] As an example, the above-mentioned sensitive information may be, but is not limited to, at least one of the following: “criminal” or “most famous”.

[0034] In some optional implementations of some embodiments, the execution subject performs sensitive information extraction processing on each initial web page information in the initial web page information set based on a preset sensitive information library to generate a web page sensitive information group, which may include the following steps:

[0035] The first step is to generate a sensitive information tree corresponding to the above-mentioned sensitive information library. The above-mentioned sensitive information tree may include: a sensitive node information set. The sensitive information tree corresponding to the above-mentioned sensitive information library may be generated by a preset structure tree generation algorithm. The sensitive node information in the above-mentioned sensitive node information set may include but is not limited to: sensitive characters, node identifiers and next sensitive node identifier sets. The sensitive characters included in the above-mentioned sensitive node information may represent a character in the sensitive information. The node identifier included in the above-mentioned sensitive node information may uniquely identify a sensitive node information. Specifically, when the next sensitive node identifier set included in the sensitive node information is empty, it can be represented that the sensitive node information does not have a child node in the above-mentioned sensitive information tree. When the next sensitive node identifier set included in the sensitive node information is not empty, the next sensitive node identifier in the next sensitive node identifier set included in the sensitive node information may represent the sensitive node information corresponding to a child node in the above-mentioned sensitive information tree.

[0036] As an example, the above-mentioned preset structure tree generation algorithm may be but is not limited to at least one of the following: a hash tree algorithm or a prefix tree algorithm.

[0037] The second step is to perform word segmentation processing on the initial web page information to obtain a web page text character sequence. The initial web page information can be processed by a preset word segmentation algorithm to obtain a web page text character sequence.

[0038] As an example, the above-mentioned preset word segmentation algorithm may be but is not limited to at least one of the following: a hidden Markov model algorithm, a conditional random field algorithm or an LSTM (Long short-term memory) model algorithm.

[0039] Step 3: For each webpage text character in the webpage text character sequence, perform the following extraction sub-steps:

[0040] The first extraction sub-step, in response to determining that the sensitive information tree includes sensitive node information that matches the web page text character, adds the web page text character to the web page sensitive information sequence. The web page sensitive information sequence is initially empty. Specifically, the sensitive node information that matches the web page text character may be: sensitive node information that includes the same sensitive character as the web page text character.

[0041] In the second extraction sub-step, in response to determining that the sensitive node information matching the web page text characters satisfies the preset node condition, each sensitive node information corresponding to the sensitive node information matching the web page text characters in the sensitive node information set is determined as a sensitive sub-node information set. The above-mentioned determination of each sensitive node information corresponding to the sensitive node information matching the web page text characters in the sensitive node information set as a sensitive sub-node information set may be: first, the sensitive node information matching the web page text characters is determined as the target sensitive node information. Then, each sensitive node information corresponding to the target sensitive node information in the sensitive node information set is determined as a sensitive sub-node information set. Specifically, the above-mentioned satisfaction of the preset node condition may be: the next sensitive node identifier set included in the sensitive node information matching the web page text characters is not empty. Each sensitive node information corresponding to the target sensitive node information in the sensitive node information set may be: sensitive node information represented by each next sensitive node identifier in the next sensitive node identifier set included in the sensitive node information.

[0042] The third extraction sub-step is to perform the following matching sub-steps on the next webpage text character of the webpage text character based on the sensitive sub-node information set:

[0043] The first matching sub-step is to match the sensitive sub-node information set with the next web page text character of the web page text character to obtain a matching result. The matching process between the sensitive sub-node information set and the next web page text character of the web page text character to obtain a matching result may be: in response to determining that among the sensitive characters included in each sensitive sub-node information in the sensitive sub-node information set, there is a sensitive character that is the same as the next web page text character of the web page text character, the first preset matching information is determined as the matching result. In response to determining that among the sensitive characters included in each sensitive sub-node information in the sensitive sub-node information set, there is no sensitive character that is the same as the next web page text character of the web page text character, the second preset matching information is determined as the matching result.

[0044] As an example, the first preset matching information may be information indicating “matching success”. The second preset matching information may be information indicating “matching failure”.

[0045] The second matching sub-step is to add the next web page text character of the web page text character to the web page sensitive information sequence in response to determining that the matching result satisfies the preset matching condition. The preset matching condition may be that the matching result is information indicating "match failure".

[0046] In the third matching sub-step, the sensitive sub-node information that matches the next web page text character of the web page text character is determined as the target sensitive sub-node information.

[0047] In the fourth extraction sub-step, in response to determining that the target sensitive sub-node information satisfies a preset sub-node condition, the web page sensitive information sequence is determined as a web page sensitive information group. The preset sub-node condition may be: the next sensitive node identifier set included in the target sensitive sub-node information is empty.

[0048] Optionally, the above execution entity may further perform the following steps:

[0049] In the first step, in response to determining that the sensitive node information matching the web page text characters does not satisfy a preset node information condition, a web page sensitive information sequence is determined as a web page sensitive information group.

[0050] In the second step, in response to determining that the target sensitive sub-node information does not meet the preset sub-node information condition, each sub-node information corresponding to the target sensitive sub-node information is determined as a sensitive sub-node information set to perform the matching processing sub-step again.

[0051] Step 106: Input each piece of sensitive information of the web page in the sensitive information set into a pre-trained sensitive information classification model to generate sensitive category information, thereby obtaining a sensitive category information set.

[0052] In some embodiments, the execution subject may input each webpage sensitive information in the webpage sensitive information set into a pre-trained sensitive information classification model to generate sensitive category information, thereby obtaining a sensitive category information set. The pre-trained sensitive information classification model may be a pre-trained neural network model that takes webpage sensitive information as input and takes sensitive category information as output.

[0053] As an example, the above sensitive category information may be, but is not limited to, at least one of the following: "social life", "laws and regulations", "ethnic and religious" or "international relations".

[0054] In some optional implementations of some embodiments, the execution subject may further perform the following steps before inputting each webpage sensitive information in the webpage sensitive information group set into a pre-trained sensitive information classification model to generate sensitive category information:

[0055] The first step is to obtain the first sample sensitive information set and the first sample sensitive category information set. The first sample sensitive information in the first sample sensitive information set corresponds one-to-one to the first sample sensitive category information in the first sample sensitive category information set. The first sample sensitive information set and the first sample sensitive category information set can be obtained from a storage terminal. The storage terminal can be a terminal for storing the first sample sensitive information set and the first sample sensitive category information set. Each first sample sensitive information in the first sample sensitive information set can represent a sensitive information. The first sample sensitive category information in the first sample sensitive category information set can be but not limited to at least one of the following: "social life category", "laws and regulations category", "ethnic and religious category" or "international relations category".

[0056] In the second step, a preset number of first sample sensitive information is selected from the first sample sensitive information set as the first random sample information set. A preset number of first sample sensitive information can be randomly selected from the first sample sensitive information set as the first random sample information set. The preset number can be the product value of the number of first sample sensitive information in the first sample sensitive information set and a preset ratio.

[0057] As an example, the preset ratio may be ten percent.

[0058] In the third step, each first random sample information in the first random sample information set is input into the initial sensitive information classification model to generate first category information, thereby obtaining the first category information set. The initial sensitive information classification model may be an untrained neural network model that takes the first random sample information as input and the first category information as output.

[0059] As an example, the initial sensitive information classification model may be, but is not limited to, at least one of the following: a naive Bayes model, a decision tree model, a support vector machine model, or a random forest model.

[0060] The fourth step is to generate second random sample information corresponding to each first category information in the above first category information set to obtain the second random sample information set.

[0061] Step 5: construct a sensitive information classification sub-model set based on the second random sample information set. The sensitive information classification sub-model set constructed based on the second random sample information set may be: based on the second random sample information set, at least one untrained initial sensitive information classification sub-model is trained by a preset model training algorithm to obtain the sensitive information classification sub-model set.

[0062] As an example, the preset model training algorithm may be, but is not limited to, at least one of the following: a back propagation algorithm or a stochastic gradient algorithm. The initial sensitive information classification sub-model may be a BERT (Bidirectional Encoder Representation from Transformers) model.

[0063] In the sixth step, based on the above sensitive information classification sub-model set, the first sample sensitive information set is filtered to obtain a second sample sensitive information set.

[0064] In the seventh step, for each second sample sensitive information in the second sample sensitive information set, the first sample sensitive category information corresponding to the second sample sensitive information in the first sample sensitive category information set is determined as the second sample sensitive category information to obtain the second sample sensitive category information set.

[0065] In the eighth step, based on the second sample sensitive information set and the second sample sensitive category information set, the initial sensitive information classification model is trained to obtain a sensitive information classification model.

[0066] In some optional implementations of some embodiments, the execution subject generates the second random sample information corresponding to each first category information in the first category information set, which may include the following steps:

[0067] In the first step, the first category information is input into a pre-trained prompt word model to obtain first category feature information. The pre-trained prompt word model may be a pre-trained large language model that takes the first category information as input and the first category feature information as output.

[0068] As an example, the prompt word model may be a large language model based on a preset instruction. The preset instruction may be information indicating “generating a feature vector corresponding to the category information”.

[0069] The second step is to generate first category positive sample information and first category negative sample information corresponding to the first category feature information. The first category positive sample information and first category negative sample information corresponding to the first category feature information can be generated by a preset generation algorithm.

[0070] As an example, the above-mentioned preset generation algorithm can be a generative adversarial network model algorithm.

[0071] The third step is to fuse the first category positive sample information, the first category negative sample information, and the first random sample information corresponding to the first category information to obtain the second random sample information. The above-mentioned fusing the first category positive sample information, the first category negative sample information, and the first random sample information corresponding to the first category information to obtain the second random sample information may be to determine the first category positive sample information, the first category negative sample information, and the first random sample information corresponding to the first category information as the first category positive sample information, the first category negative sample information, and the first random sample information included in the second random sample information.

[0072] In some optional implementations of some embodiments, the execution subject generates the second random sample information corresponding to each first category information in the first category information set, which may include the following steps:

[0073] In the first step, for each first sample sensitive information in the first sample sensitive information set, perform the following deletion sub-steps:

[0074] In a first sub-step, based on the first sample sensitive information, a first sample classification probability value corresponding to each sensitive information classification sub-model in the sensitive information classification sub-model set is generated to obtain a first sample classification probability value set.

[0075] In the second sub-step, the average of the first sample classification probability values ​​in the first sample classification probability value set is determined as the second sample classification probability value.

[0076] The third sub-step is, in response to determining that the second sample classification probability value is less than a preset probability value, deleting the first sample sensitive information from the first sample sensitive information set.

[0077] As an example, the preset probability value may be 0.

[0078] In the second step, the deleted first sample sensitive information set is determined as the second sample sensitive information set.

[0079] In some optional implementations of some embodiments, the execution subject generates, based on the first sample sensitive information, a first sample classification probability value corresponding to each sensitive information classification sub-model in the sensitive information classification sub-model set, which may include the following steps:

[0080] The first step is to input the above-mentioned first sample sensitive information into the sensitive information classification sub-model to obtain the first classification information.

[0081] The second step is to generate positive sample category information and negative sample category information corresponding to the first classification information. The positive sample category information and negative sample category information corresponding to the first classification information can be generated by the preset generation algorithm.

[0082] The third step is to determine the positive sample classification probability value corresponding to the positive sample category information and the first sample sensitive information based on a preset classification loss function.

[0083] As an example, the above-mentioned preset classification loss function can be but is not limited to at least one of the following: a cross entropy loss function, a hinge loss function or an exponential loss function.

[0084] The fourth step is to determine the negative sample classification probability value corresponding to the negative sample category information and the first sample sensitive information based on a preset classification loss function.

[0085] In the fifth step, the difference between the positive sample classification probability value and the negative sample classification probability value is determined as the first sample classification probability value.

[0086] In some optional implementations of some embodiments, the execution subject performs training processing on the initial sensitive information classification model based on the second sample sensitive information set and the second sample sensitive category information set to obtain the sensitive information classification model, and the second sample sensitive information may be selected from the second sample sensitive information set to perform the following training steps:

[0087] In the first step, the second sample sensitive information is input into the initial sensitive information classification model to obtain initial sensitive category information, wherein the second sample sensitive information can be randomly selected from the second sample sensitive information set.

[0088] In the second step, based on a preset classification loss function, the classification loss values ​​of the second sample sensitive category information set and the second sample sensitive category information, the second sample sensitive category information and the initial sensitive category information are determined.

[0089] As an example, the above-mentioned preset classification loss function can be but is not limited to one of the following: a cross entropy loss function, a least square function or a classification cross entropy loss function.

[0090] In the third step, in response to determining that the classification loss value is less than the target threshold, the initial sensitive information classification model is determined as the sensitive information classification model.

[0091] As an example, the target threshold mentioned above may be 0.01.

[0092] Optionally, the execution subject may also adjust relevant parameters in the initial sensitive information classification model based on the sensitive information classification sub-model set in response to determining that the classification loss value is greater than or equal to the target threshold, determine the adjusted initial sensitive information classification model as the initial sensitive information classification model, and select second sample sensitive information from each second sample sensitive information that has not been selected in the second sample sensitive information set, so as to perform the training step again. The relevant parameters in the initial sensitive information classification model may be adjusted by a preset adjustment algorithm.

[0093] As an example, the above-mentioned preset adjustment algorithm may be a knowledge distillation algorithm.

[0094] Thus, at least one classification sub-model can be trained through part of the sample data. Then, the sample data can be screened through the trained classification sub-model. Then, the classification model can be further trained and adjusted based on the sample data and each trained classification sub-model. Thus, the accuracy of the classification model can be improved. Therefore, the accuracy of sensitive information classification can be improved, thereby improving the accuracy of network data anomaly identification, and further, the security of network services can be improved.

[0095] Step 107: for each sensitive category information group in the sensitive category information group set, in response to determining that sensitive category information that meets a preset category condition exists in the sensitive category information group, initial web page information corresponding to the sensitive category information group is determined as abnormal web page information.

[0096] In some embodiments, the execution subject may, for each sensitive category information group in the sensitive category information group set, determine the initial web page information corresponding to the sensitive category information group as abnormal web page information in response to determining that the sensitive category information group contains sensitive category information that meets a preset category condition. The preset category condition may be target category information represented by the sensitive category information.

[0097] As an example, the target category information may be, but is not limited to, at least one of the following: “laws and regulations” or “international relations”.

[0098] Step 108, performing category identification processing on each determined abnormal web page information to generate web page abnormal category information, and obtaining a web page abnormal category information set.

[0099] In some embodiments, the execution subject may perform category identification processing on each abnormal web page information determined to generate web page abnormality category information, and obtain a web page abnormality category information set. Among them, a preset category identification algorithm may be used to perform category identification processing on each abnormal web page information determined to generate web page abnormality category information, and obtain a web page abnormality category information set. The web page abnormality category information may be, but is not limited to, at least one of the following: information representing a "first-level abnormality", information representing a "second-level abnormality", or information representing a "third-level abnormality".

[0100] As an example, the above-mentioned preset category recognition algorithm can be but is not limited to at least one of the following: K nearest neighbor algorithm, decision tree algorithm, naive Bayes algorithm, logistic regression algorithm or support vector machine algorithm.

[0101] Step 109: Generate a web page abnormality warning information set based on the web page abnormality category information set.

[0102] In some embodiments, the execution subject may generate a web page abnormality warning information set based on the web page abnormality category information set. The generation of the web page abnormality warning information set based on the web page abnormality category information set may be: for each web page abnormality category information in the web page abnormality category information set, the warning text or prompt sound representing the web page abnormality category information is determined as the web page abnormality warning information corresponding to the web page abnormality category information. Then, each determined web page abnormality warning information may be determined as a web page abnormality warning information set.

[0103] Step 110, sending the webpage abnormality alarm information set to the alarm terminal for alarm processing.

[0104] In some embodiments, the execution subject may send the webpage abnormality alarm information set to an alarm terminal for alarm processing. The alarm terminal may be a terminal for responding to display or playing the webpage abnormality alarm information set.

[0105] Optionally, the execution subject may delete each abnormal web page information determined from the initial web page information set, and send the deleted initial web page information set to the server. Thus, the server may send the deleted initial web page information set to the target user terminal.

[0106] The above-mentioned embodiments of the present application have the following beneficial effects: through the network data anomaly identification method of some embodiments of the present application, the security of network services can be improved. Specifically, the reason for the reduced security of network services is that only sensitive information in the web page content is identified, and it is difficult to identify the dark links and Trojan programs appearing in the web page links, resulting in a reduction in the accuracy of the identified network abnormal data. Based on this, the network data anomaly identification method of some embodiments of the present application, first, in response to receiving the network data anomaly identification request information sent by the server, obtains the initial network link information set from the above server. Secondly, each initial network link information in the above initial network link information set is subjected to a first abnormal identification process to generate an abnormal network link identification result, and obtains an abnormal network link identification result set. Thus, abnormal network links can be identified, and web page dark links and Trojan programs can be identified. Again, for each abnormal network link identification result in the above abnormal network link identification result set, in response to determining that the above abnormal network link identification result meets the preset link abnormality condition, the initial network link information corresponding to the above abnormal network link identification result in the above initial network link information set is deleted from the above initial network link information set. Thus, the identified abnormal web page links can be screened out from the initial web page link set. Then, the initial web page information corresponding to each initial network link information in the deleted initial network link information set is obtained to obtain the initial web page information set. Then, based on the preset sensitive information library, sensitive information extraction processing is performed on each initial web page information in the above initial web page information set to generate a web page sensitive information group to obtain a web page sensitive information group set. Thus, sensitive information in the web page content can be extracted. Then, each web page sensitive information in the above web page sensitive information group set is input into a pre-trained sensitive information classification model to generate sensitive category information to obtain a sensitive category information group set. Thus, sensitive information can be classified so as to warn sensitive information of the target category. Then, for each sensitive category information group in the above sensitive category information group set, in response to determining that there is sensitive category information that meets the preset category condition in the above sensitive category information group, the initial web page information corresponding to the above sensitive category information group is determined as abnormal web page information. Thus, the web page where the sensitive information of the target category appears can be determined as an abnormal web page. Subsequently, category identification processing is performed on each abnormal web page information determined to generate abnormal web page category information to obtain an abnormal web page category information set. Thus, the abnormal category and level information of the abnormal web page information can be obtained, so as to give an alarm for each category of abnormal web page information. Then, based on the above web page abnormal category information set, a web page abnormal alarm information set is generated. Thus, the alarm information corresponding to each category of abnormal web pages can be obtained. Finally, the above web page abnormal alarm information set is sent to the alarm terminal for alarm processing. Thus, an alarm can be given for abnormal web pages and network data.Therefore, some network data anomaly identification methods of the present application can first identify anomalies on the network links obtained from the server (for example, bad link scanning, dark link scanning, or Trojan scanning). Then, the web page content corresponding to the network link without abnormality is obtained. Next, the sensitive information in the web page information is extracted. Subsequently, the abnormal level and category of the web page are determined based on different categories of sensitive information. In this way, the accuracy and comprehensiveness of the identified network anomaly data can be improved, thereby improving the security of network services.

[0107] The present application also provides a computer device 200. Figure 2 As shown, the computer device 200 includes: a bus 201, a processor 202, a memory 203 and a communication interface 204. The processor 202, the memory 203 and the communication interface 204 communicate with each other through the bus 201. The computer device 200 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computer device 200.

[0108] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 The bus 201 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 201 may include a path for transmitting information between various components of the computer device 200 (for example, the memory 203, the processor 202, and the communication interface 204).

[0109] The processor 202 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0110] The memory 203 may include a volatile memory, such as a random access memory (RAM). The memory 203 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0111] The memory 203 stores executable program codes, and the processor 202 executes the executable program codes to respectively implement the functions of the acquisition module, sampling module, determination module, and mixing module, thereby implementing the above-mentioned network data anomaly identification method. That is, the memory 203 stores instructions for executing the above-mentioned network data anomaly identification method.

[0112] The communication interface 204 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computer device 200 and other devices or a communication network.

[0113] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned network data anomaly identification method.

[0114] The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The above-mentioned available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk). The computer-readable storage medium includes instructions, and the above-mentioned instructions instruct the computing device to execute the above-mentioned network data anomaly identification method.

[0115] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying abnormal network data, comprising: In response to receiving a network data anomaly identification request message sent by a server, acquiring an initial network link information set from the server; Performing a first abnormality identification process on each initial network link information in the initial network link information set to generate an abnormal network link identification result, thereby obtaining an abnormal network link identification result set; For each abnormal network link identification result in the abnormal network link identification result set, in response to determining that the abnormal network link identification result meets a preset link abnormality condition, deleting the initial network link information corresponding to the abnormal network link identification result in the initial network link information set from the initial network link information set; Acquire the initial web page information corresponding to each initial network link information in the deleted initial network link information set to obtain the initial web page information set; Based on a preset sensitive information database, performing sensitive information extraction processing on each initial web page information in the initial web page information set to generate a web page sensitive information group, thereby obtaining a web page sensitive information group set; Inputting each webpage sensitive information in the webpage sensitive information set into a pre-trained sensitive information classification model to generate sensitive category information, thereby obtaining a sensitive category information set; For each sensitive category information group in the sensitive category information group set, in response to determining that sensitive category information that meets a preset category condition exists in the sensitive category information group, determining initial web page information corresponding to the sensitive category information group as abnormal web page information; Performing category identification processing on each determined abnormal web page information to generate web page abnormal category information, and obtaining a web page abnormal category information set; Based on the webpage abnormality category information set, generating a webpage abnormality warning information set; Sending the webpage abnormality alarm information set to an alarm terminal for alarm processing; The step of performing sensitive information extraction processing on each initial web page information in the initial web page information set based on a preset sensitive information library to generate a web page sensitive information group includes: Generate a sensitive information tree corresponding to the sensitive information library, wherein the sensitive information tree includes: a sensitive node information set; Performing word segmentation processing on the initial web page information to obtain a web page text character sequence; For each webpage text character in the webpage text character sequence, the following extraction steps are performed: In response to determining that the sensitive information tree includes sensitive node information that matches the web page text character, adding the web page text character to a web page sensitive information sequence, wherein the web page sensitive information sequence is initially empty; In response to determining that the sensitive node information matching the webpage text character satisfies a preset node condition, determining each sensitive node information corresponding to the sensitive node information matching the webpage text character in the sensitive node information set as a sensitive sub-node information set; Based on the sensitive sub-node information set, the following matching processing steps are performed on the next webpage text character of the webpage text character: Matching the sensitive sub-node information set with the next web page text character of the web page text character to obtain a matching result; In response to determining that the matching result satisfies the preset matching condition, adding the next web page text character of the web page text character to the web page sensitive information sequence; Determine the sensitive sub-node information that matches the next web page text character of the web page text character as the target sensitive sub-node information; In response to determining that the target sensitive sub-node information satisfies a preset sub-node condition, the web page sensitive information sequence is determined as a web page sensitive information group.

2. The network data anomaly identification method according to claim 1, wherein: The method further comprises: The determined abnormal web page information is deleted from the initial web page information set, and the deleted initial web page information set is sent to the server.

3. The network data anomaly identification method according to claim 1, wherein: The method further comprises: In response to determining that the sensitive node information matching the webpage text characters does not satisfy a preset node information condition, determining the webpage sensitive information sequence as a webpage sensitive information group; In response to determining that the target sensitive sub-node information does not meet the preset sub-node information condition, each sub-node information corresponding to the target sensitive sub-node information is determined as a sensitive sub-node information set to perform the matching processing step again.

4. The network data anomaly identification method according to claim 1, wherein: Before inputting each webpage sensitive information in the webpage sensitive information set into a pre-trained sensitive information classification model to generate sensitive category information, the method further includes: Acquire a first sample sensitive information set and a first sample sensitive category information set, wherein the first sample sensitive information in the first sample sensitive information set corresponds one-to-one to the first sample sensitive category information in the first sample sensitive category information set; Selecting a preset number of first sample sensitive information from the first sample sensitive information set as a first random sample information set; Inputting each first random sample information in the first random sample information set into the initial sensitive information classification model to generate first category information, thereby obtaining a first category information set; Generate second random sample information corresponding to each first category information in the first category information set to obtain a second random sample information set; Based on the second random sample information set, construct a sensitive information classification sub-model set; Based on the sensitive information classification sub-model set, filtering the first sample sensitive information set to obtain a second sample sensitive information set; For each second sample sensitive information in the second sample sensitive information set, determine the first sample sensitive category information corresponding to the second sample sensitive information in the first sample sensitive category information set as the second sample sensitive category information, to obtain the second sample sensitive category information set; Based on the second sample sensitive information set and the second sample sensitive category information set, the initial sensitive information classification model is trained to obtain a sensitive information classification model.

5. The network data anomaly identification method according to claim 4, wherein: The generating of second random sample information corresponding to each first category information in the first category information set includes: Inputting the first category information into a pre-trained prompt word model to obtain first category feature information; Generate first category positive sample information and first category negative sample information corresponding to the first category feature information; The first category positive sample information, the first category negative sample information and the first random sample information corresponding to the first category information are fused to obtain second random sample information.

6. A computer device, wherein: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images