Internet portal website abnormity monitoring method and system

Abstract

The invention provides an internet portal website abnormity monitoring method and system, and the method comprises the steps: carrying out the sensitive text recognition and matching of page behavior information, so as to obtain a sensitive text proportion, and determining a first recognition result based on the sensitive text proportion; performing feature extraction and malicious link identification on the website link information to output a second identification result; performing feature description and malicious image recognition on the page picture information to output a third recognition result; and outputting a final identification result and an abnormal type of influence based on the first identification result, the second identification result and the third identification result. The method can effectively manage the access condition, the page content, the internal and external links and the link content of the target portal website, and carries out monitoring and system warning on the three abnormal conditions; the link security and the content security of the website are improved, and the anomaly detection precision is high.

Description

Technical Field

[0001] The invention belongs to the technical field of website monitoring, and in particular relates to an abnormality monitoring method and system for an Internet portal website. Background Art

[0002] At present, with the development of the Internet, while the Internet brings convenience, it also brings the following network security problems, such as: leakage of personal information, virus attacks, spread of bad false information, etc. Therefore, in order to improve the network security of the Internet, it is very necessary to monitor the abnormalities of Internet portals.

[0003] At present, it is impossible to achieve effective and accurate monitoring of Internet portal websites. The usual monitoring method is to obtain content information from the Internet and determine whether there are corresponding abnormal situations on the Internet based on the information obtained. However, the above monitoring method has the following problems: the monitoring is not comprehensive enough and it is impossible to monitor abnormalities from aspects such as links, pages, and pictures on the Internet; the existing monitoring accuracy is not accurate enough and it can only roughly monitor the existence of abnormalities on the Internet, but cannot accurately output the corresponding abnormality type. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an Internet portal website abnormality monitoring method and system, which are used to solve the technical problems in the prior art.

[0005] In a first aspect, the present invention provides the following technical solution, a method for monitoring abnormalities of an Internet portal website, comprising: Obtaining website information of the target website, the website information including page behavior information, website link information and page image information; Performing sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determining a first recognition result based on the sensitive text ratio; Extracting features and identifying malicious links on the website link information to output a second identification result; Performing feature description and malicious image recognition on the page image information to output a third recognition result; Outputting a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result, and the third recognition result; The step of performing sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determining a first recognition result based on the sensitive text ratio includes: Extracting specific tag content from the page behavior information, and removing special characters from the specific tag content to obtain uncluttered tag content, and segmenting the uncluttered tag content to obtain a number of page text information; Establishing a vertex table based on the plurality of page text information, wherein the vertex table is used to record the header address of the single linked list corresponding to the page text information; Starting from state 0, the finite state automaton established by breadth-first search establishes corresponding nodes for the next state that can be jumped to, and links the corresponding nodes to the singly linked list of state 0; Calculate the hash value for the first byte of the character of each node in the single linked list of state 0, and link the corresponding node to the hash table according to the hash value to obtain a hash linked list; Calculate the state transition function and corresponding failure function of the next arbitrary state A starting from state 0; Determine the output function of state A based on the state transition function and the failure function, and add the keyword pattern string to the output function of state A after processing each keyword pattern string; Initialize state A to state 0 and point the text string pointer to the first character of the text string. If the text string pointer is not empty, output the pointed character K. A sensitive text ratio is determined based on the character K, and a first recognition result is determined according to the sensitive text ratio.

[0006] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention first obtains the website information of the target website, and the website information includes page behavior information, website link information and page image information; then the page behavior information is subjected to sensitive text recognition and matching to obtain the sensitive text ratio, and a first recognition result is determined based on the sensitive text ratio; then the website link information is subjected to feature extraction and malicious link recognition to output a second recognition result; then the page image information is subjected to feature description and malicious image recognition to output a third recognition result; finally, based on the first recognition result, the second recognition result and the third recognition result, a final recognition result and the affected abnormal type are output. The present invention can effectively manage the access situation of the target portal website, the page content, and the internal and external links and link content, and monitor and issue system alarms for the three abnormal situations, thereby improving the link security and content security of the website, and the anomaly detection accuracy is high.

[0007] Preferably, the step of determining the sensitive text ratio based on the character K and determining the first recognition result according to the sensitive text ratio includes: Calculate the state transition function of state A and character K, and determine state B based on the state transition function of state A and character K. If state B is 0, search the corresponding single-column table in the hash list according to the hash value of the character and determine whether state B is fail; If state B is fail, state A is the cutoff state and the corresponding output function is output. If state B is not fail, state A is different from state B and the corresponding output function is output. It is determined whether the corresponding output function is empty. If the corresponding output function is empty, initialize state B to state 0 and repeat the calculation and judgment process of the state transfer function. If the corresponding output function is not empty, directly output the corresponding output function. Determine the matching sensitive text based on the output function, and match the matching sensitive text with the sensitive text library to determine the sensitive text ratio; If the sensitive text ratio exceeds the preset sensitive text ratio, the target website has page behavior abnormalities; if the sensitive text ratio does not exceed the preset sensitive text ratio, the target website does not have page behavior abnormalities, so as to obtain a first identification result.

[0008] Preferably, the step of extracting features and identifying malicious links from the website link information to output a second identification result includes: Performing data cleaning on the website link information to remove abnormal link data to obtain cleaned link data, wherein the abnormal link data includes random domain names with abnormal vowel ratios, data with abnormal character set discreteness, data with abnormal domain name length, and link data with excessively high frequency of switching between numbers and letters; The cleaned link data is input into a preset encoder and passed through a self-attention module in the preset encoder to obtain a weighted feature vector : ; In the formula, , , Represent the query vector, key vector, and value vector respectively. represents the dimension of the key vector; The weighted feature vector is input into the fully connected layer in the preset encoder to output the activation feature : ; In the formula, , , Represent the first and second encoder coefficients respectively, , Respectively represent the first and second encoder constants; Get the first relative position code of the word in the clean link data Or the second relative position code , encode the first relative position Relative position code with the second Add to the activation feature To obtain the encoding features: ; ; In the formula, represents the position of the words in the clean link data in the clean link data, represents the position encoding dimension, Represents dimension index; Obtain manually annotated training malicious link data, input the training malicious link data into a preset recognition model for training, input the encoding features into the trained preset recognition model for malicious link recognition, and output a second recognition result.

[0009] Preferably, the step of performing feature description and malicious image recognition on the page image information to output a third recognition result includes: Define the spatial scale of the page image information: ; In the formula, represents a two-dimensional Gaussian function, Indicates the page image in the page image information at position The pixel value at Represents scale space; A Gaussian pyramid is constructed based on the spatial scale, and the page image information is downsampled based on the Gaussian pyramid and using different spatial scales, and Gaussian difference space extreme point detection is used to obtain a plurality of detection extreme points, wherein the Gaussian difference space for: ; In the formula, Respectively represent the group number index and the layer number index within the group; Determine the Gaussian shift value for each extreme point : ; In the formula, represents the Gaussian difference space for detecting extreme points, represents the coordinates of the detected extreme points, Represents the scale space corresponding to the detected extreme point; The extreme points whose Gaussian offset value is not greater than the preset offset value are eliminated, and the extreme points whose Gaussian offset value is greater than the preset offset value are retained to obtain feature points; An image feature is determined based on the feature point, and a third recognition result is determined according to the image feature.

[0010] Preferably, the step of determining the image features based on the feature points and determining the third recognition result according to the image features includes: Determine a pixel area with the feature point as the center, determine the horizontal gradient and the vertical gradient of each pixel in the pixel area, and determine a gradient matrix based on the horizontal gradient and the vertical gradient; Determine the covariance matrix of the gradient matrix and calculate the eigenvalues ​​and responses of the covariance matrix to obtain eigenvectors; Arrange the eigenvalues ​​of the covariance matrix in descending order, select the first several eigenvalues ​​as matrix eigenvalues, and determine the characteristic matrix according to the eigenvectors corresponding to the matrix eigenvalues; Multiplying the gradient matrix by the feature matrix to obtain a descriptor, and fusing the descriptor with the feature points to obtain image features; Obtain manually annotated training malicious image data, input the training malicious image data into a preset image recognition model for training, input image features into the trained preset image recognition model for malicious image recognition, and output a third recognition result.

[0011] Preferably, the step of outputting the final recognition result and the affected abnormality type based on the first recognition result, the second recognition result and the third recognition result comprises: If the first recognition result is that there is no page behavior anomaly, the second recognition result is that there is no link anomaly, and the third recognition result is that there is no image anomaly, then the final recognition result is that there is no anomaly; If the first recognition result is that there is a page behavior anomaly, or the second recognition result is that there is a link anomaly, or the third recognition result is that there is an image anomaly, then the final recognition result is that there is an anomaly, and the corresponding anomaly type is output.

[0012] In a second aspect, the present invention provides the following technical solution: an Internet portal website abnormality monitoring system, the system comprising: An acquisition module is used to acquire website information of a target website, wherein the website information includes page behavior information, website link information, and page image information; A first recognition module is used to perform sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determine a first recognition result based on the sensitive text ratio; A second identification module, used for extracting features and identifying malicious links from the website link information to output a second identification result; A third recognition module, used for performing feature description and malicious image recognition on the page image information to output a third recognition result; An output module is used to output a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result and the third recognition result.

[0013] In a third aspect, the present invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for monitoring abnormalities of an Internet portal website when executing the computer program.

[0014] In a fourth aspect, the present invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned method for monitoring abnormalities of an Internet portal website. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of an Internet portal website abnormality monitoring method provided in Embodiment 1 of the present invention; Figure 2 A structural block diagram of an Internet portal website anomaly monitoring system provided in Embodiment 2 of the present invention; Figure 3 A schematic diagram of the hardware structure of a computer provided in another embodiment of the present invention.

[0017] The embodiments of the present invention will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0018] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0019] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0020] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0021] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0022] Embodiment 1 In the first embodiment of the present invention, Figure 1 As shown, a method for monitoring abnormalities of an Internet portal website comprises: S1. Obtain website information of the target website, wherein the website information includes page behavior information, website link information, and page image information; Specifically, in the process of obtaining the target website information, several crawler nodes can crawl the target site content and data, adopt a breadth-first website crawling strategy, crawl all pages within the third-level page depth of the website, and in the crawling process, it can automatically determine whether the target website has an anti-crawling mechanism. If it is determined that an anti-crawling mechanism exists, the link crawling strategy is automatically switched, and Selenium WebDriver technology is used to obtain the target site content and data by calling the browser request method and simulating human mouse operation behavior, thereby obtaining the website information.

[0023] S2. performing sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determining a first recognition result based on the sensitive text ratio; Wherein, the step S2 comprises: S21, extracting specific tag content from the page behavior information, and removing special characters from the specific tag content to obtain uncluttered tag content, and segmenting the uncluttered tag content to obtain a number of page text information; Specifically, special characters may include characters such as title, a, h, span, etc. in the HTML code of the page.

[0024] S22, establishing a vertex table based on the plurality of page text information, wherein the vertex table is used to record the header address of the single linked list corresponding to the page text information; S23, starting from state 0, using a finite state automaton established by breadth-first search to establish a corresponding node for the next state that can be jumped to, and linking the corresponding node to the singly linked list of state 0; Specifically, several nodes are gradually established starting from the node in state 0, and each node can be represented as a byte or a letter or other minimum identification unit of text; For example, in this application, assuming that the sensitive text is represented as abcdefg, then a, b, c, d, e, f, and g can all represent a state.

[0025] S24. Calculate a hash value for the first byte of the character of each node in the single linked list of state 0, and link the corresponding node to the hash table according to the hash value to obtain a hash linked list.

[0026] S25, calculating the state transition function and the corresponding failure function of the next arbitrary state A starting from state 0; Specifically, the invalidation function is represented as a cutoff state. When the invalidation function of the invalidation function is matched, the matching process can be stopped. Specifically, the invalidation function calculates the state to be transferred when the target character is not matched. The calculation method is: if the parent state r of state s is 0, the parent state is the previous state, then the invalidation function , if the parent state r is not 0, for the target character " ”, its failure function , among which Represents the state transfer function. For the state transfer function, it is used to calculate the next state of the matched target character. The calculation method is: starting from the current state r, find the character " "The next state s to jump from state r. If there is no next state, the failure state fail is returned.

[0027] S26. Determine the output function of state A based on the state transfer function and the failure function, and add the keyword pattern string to the output function of state A after processing each keyword pattern string.

[0028] S27, initializing state A to state 0 and pointing the text string pointer to the first character of the text string, and if the text string pointer is not empty, outputting the pointed character K; Specifically, the character K here is similar to the character " ", and for the processed keyword pattern string, it specifically refers to each individual text information contained in the page text information. When processing the keyword pattern string, each state represents a character, and when processing characters, the text string pointer can represent the output direction, such as a b c d e f g, if the text string pointer is not empty, just output the character it points to.

[0029] S28, determining a sensitive text ratio based on the character K, and determining a first recognition result according to the sensitive text ratio; Wherein, the step S28 comprises: S281, calculate the state transition function of state A and character K, and determine state B based on the state transition function of state A and character K. If state B is 0, search the corresponding single-column table in the hash linked list according to the hash value of the character and determine whether state B is fail; Specifically, if state B is not 0, the singly linked list corresponding to state B is searched.

[0030] S282. If state B is fail, state A is the cutoff state and the corresponding output function is output. If state B is not fail, state A is different from state B and the corresponding output function is output. It is determined whether the corresponding output function is empty.

[0031] S283. If the corresponding output function is empty, initialize state B to state 0 and repeat the calculation and judgment process of the state transfer function. If the corresponding output function is not empty, directly output the corresponding output function. S284, determining matching sensitive text based on the output function, and matching the matching sensitive text with a sensitive text library to determine a sensitive text ratio; Specifically, in the above steps, the output function is specifically a word or sentence text, so the matching sensitive text can be obtained according to the output function, and then by matching with the pre-established sensitive text library, the corresponding sensitive text ratio can be obtained.

[0032] S285. If the sensitive text ratio exceeds the preset sensitive text ratio, the target website has page behavior abnormality; if the sensitive text ratio does not exceed the preset sensitive text ratio, the target website does not have page behavior abnormality, so as to obtain a first recognition result; Specifically, if the sensitive text ratio exceeds the preset sensitive text ratio, it means that the website has abnormal page behavior problems, and vice versa.

[0033] S3, extracting features and identifying malicious links on the website link information to output a second identification result; Wherein, the step S3 comprises: S31. Clean the website link information to remove abnormal link data to obtain cleaned link data, wherein the abnormal link data includes random domain names with abnormal vowel ratios, data with abnormal character set discreteness, data with abnormal domain name lengths, and link data with excessively high frequency of switching between numbers and letters.

[0034] S32, inputting the cleaned link data into a preset encoder, and passing through the self-attention module in the preset encoder to obtain a weighted feature vector : ; In the formula, , , Represent the query vector, key vector, and value vector respectively. Indicates the dimension of the key vector.

[0035] S33, inputting the weighted feature vector into the fully connected layer in the preset encoder to output the activation feature : ; In the formula, , , Represent the first and second encoder coefficients respectively, , Represent the first and second encoder constants respectively.

[0036] S34, obtaining the first relative position code of the word in the cleaning link data in the cleaning link data Or the second relative position code , encode the first relative position Relative position code with the second Add to the activation feature To obtain the encoding features: ; ; In the formula, represents the position of the words in the clean link data in the clean link data, represents the position encoding dimension, Represents a dimension index.

[0037] S35, obtaining manually annotated training malicious link data, inputting the training malicious link data into a preset recognition model for training, inputting the encoding features into the trained preset recognition model for malicious link recognition, and outputting a second recognition result; Specifically, in this application, the preset recognition model is a CNN model.

[0038] S4, performing feature description and malicious image recognition on the page image information to output a third recognition result; Wherein, the step S4 comprises: S41, defining the spatial scale of the page image information: ; In the formula, represents a two-dimensional Gaussian function, Indicates the page image in the page image information at position The pixel value at Represents the scale space.

[0039] S42, constructing a Gaussian pyramid based on the spatial scale, downsampling the page image information based on the Gaussian pyramid and using different spatial scales, and using Gaussian difference space extreme point detection to obtain a plurality of detection extreme points, wherein the Gaussian difference space for: ; In the formula, They represent the group number index and the layer number index within the group respectively.

[0040] S43, determine the Gaussian offset value of each extreme point : ; In the formula, represents the Gaussian difference space for detecting extreme points, represents the coordinates of the detected extreme points, Represents the scale space corresponding to the detected extreme point; S44, eliminating extreme value points whose Gaussian offset values ​​are not greater than the preset offset value, and retaining extreme value points whose Gaussian offset values ​​are greater than the preset offset value, so as to obtain feature points; Specifically, in step S44, some extreme points with strong edge effects and low contrast may be eliminated.

[0041] S45, determining image features based on the feature points, and determining a third recognition result according to the image features; Wherein, the step S45 comprises: S451, determining a pixel area with the feature point as the center, and determining the horizontal gradient and the vertical gradient of each pixel in the pixel area, and determining a gradient matrix based on the horizontal gradient and the vertical gradient.

[0042] S452, determining the covariance matrix of the gradient matrix and calculating the eigenvalues ​​and responses of the covariance matrix to obtain eigenvectors.

[0043] S453, arranging the eigenvalues ​​of the covariance matrix in descending order, selecting the first several eigenvalues ​​as matrix eigenvalues ​​and determining a characteristic matrix according to the eigenvectors corresponding to the matrix eigenvalues.

[0044] S454: multiply the gradient matrix by the feature matrix to obtain a descriptor, and fuse the descriptor with the feature point to obtain an image feature.

[0045] S455, obtaining manually annotated training malicious image data, inputting the training malicious image data into a preset image recognition model for training, and inputting image features into the trained preset image recognition model for malicious image recognition, so as to output a third recognition result; Specifically, the preset image recognition model here is an LSTM model.

[0046] S5. Output a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result, and the third recognition result.

[0047] Wherein, the step S5 comprises: S51. If the first recognition result is that there is no page behavior anomaly, the second recognition result is that there is no link anomaly, and the third recognition result is that there is no image anomaly, then the final recognition result is that there is no anomaly.

[0048] S52: If the first recognition result is that there is a page behavior anomaly, or the second recognition result is that there is a link anomaly, or the third recognition result is that there is an image anomaly, then the final recognition result is that there is an anomaly, and the corresponding anomaly type is output; Specifically, the exception type here is the exception in the above-mentioned recognition result. If there is only an image exception, the exception type is an image exception. Similarly, the same is true for page behavior exceptions and link exceptions. If there are two exceptions, the corresponding exception type can be output. If there are three exceptions, the exception type is page behavior exception, link exception, and image exception.

[0049] The method for monitoring abnormalities of an Internet portal website provided in the first embodiment of the present invention first obtains website information of a target website, the website information includes page behavior information, website link information and page image information; then, sensitive text recognition and matching are performed on the page behavior information to obtain a sensitive text ratio, and a first recognition result is determined based on the sensitive text ratio; then, feature extraction and malicious link recognition are performed on the website link information to output a second recognition result; then, feature description and malicious image recognition are performed on the page image information to output a third recognition result; finally, a final recognition result and an affected abnormality type are output based on the first recognition result, the second recognition result and the third recognition result. The present invention can effectively manage the access situation of the target portal website, the page content, and the internal and external links and link content, and monitor and issue system alarms for the three abnormal situations, thereby improving the link security and content security of the website, and the anomaly detection accuracy is high.

[0050] Embodiment 2 like Figure 2 As shown, in the second embodiment of the present invention, a system for monitoring abnormalities of an Internet portal website is provided, and the system includes: Acquisition module 1, used to acquire website information of the target website, the website information includes page behavior information, website link information and page image information; A first recognition module 2 is used to perform sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determine a first recognition result based on the sensitive text ratio; The second identification module 3 is used to extract features and identify malicious links from the website link information to output a second identification result; The third recognition module 4 is used to perform feature description and malicious image recognition on the page image information to output a third recognition result; An output module 5, configured to output a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result and the third recognition result; The first identification module 2 includes: A segmentation submodule is used to extract specific tag content from the page behavior information, remove special characters from the specific tag content to obtain clean tag content, and segment the clean tag content to obtain a number of page text information; A vertex table submodule, used for establishing a vertex table based on a plurality of the page text information, wherein the vertex table is used for recording a header address of a single linked list corresponding to the page text information; The node submodule is used to establish a corresponding node for the next state that can be jumped to using the finite state automaton established by breadth-first search starting from state 0, and link the corresponding node to the single linked list of state 0; A hash submodule is used to calculate a hash value for the first byte of each node character in the single linked list of state 0, and link the corresponding node into the hash table according to the hash value to obtain a hash linked list; A calculation submodule is used to calculate the state transition function of the next arbitrary state A starting from state 0 and the corresponding failure function; An output submodule, used to determine the output function of state A based on the state transfer function and the failure function, and add the keyword pattern string to the output function of state A after processing each keyword pattern string; The pointing submodule is used to initialize the state A to state 0 and point the text string pointer to the first character of the text string. If the text string pointer is not empty, the pointed character K is output; The ratio determination submodule is used to determine the sensitive text ratio based on the character K, and determine the first recognition result according to the sensitive text ratio.

[0051] The ratio determination submodule includes: A calculation unit is used to calculate the state transition function of state A and character K, and determine state B based on the state transition function of state A and character K. If state B is 0, search the corresponding single-column table in the hash linked list according to the hash value of the character and determine whether state B is fail; A cut-off unit, used for, if state B is fail, then state A is the cut-off state, and outputs the corresponding output function; if state B is not fail, then state A is different from state B, and outputs the corresponding output function, and determines whether the corresponding output function is empty; An initialization unit, used for initializing state B to state 0 and repeating the calculation and judgment process of the state transfer function if the corresponding output function is empty, and directly outputting the corresponding output function if the corresponding output function is not empty; A matching unit, used to determine the matched sensitive text based on the output function, and match the matched sensitive text with the sensitive text library to determine the sensitive text ratio; The first identification unit is used to determine if the sensitive text ratio exceeds a preset sensitive text ratio, that the target website has page behavior abnormalities; if the sensitive text ratio does not exceed the preset sensitive text ratio, that the target website does not have page behavior abnormalities, so as to obtain a first identification result.

[0052] The second identification module 3 includes: A cleaning submodule is used to clean the website link information to remove abnormal link data to obtain clean link data, wherein the abnormal link data includes random domain names with abnormal vowel ratio, data with abnormal character set discreteness, data with abnormal domain name length, and link data with excessively high frequency of switching between numbers and letters; The encoding submodule is used to input the cleaned link data into a preset encoder, and pass it through the self-attention module in the preset encoder to obtain a weighted feature vector : ; In the formula, , , Represent the query vector, key vector, and value vector respectively. represents the dimension of the key vector; A connection submodule is used to input the weighted feature vector into a fully connected layer in a preset encoder to output an activation feature : ; In the formula, , , Represent the first and second encoder coefficients respectively, , Respectively represent the first and second encoder constants; The position encoding submodule is used to obtain the first relative position encoding of the word in the cleaned link data in the cleaned link data. Or the second relative position code , encode the first relative position Relative position code with the second Add to the activation feature To obtain the encoding features: ; ; In the formula, represents the position of the words in the clean link data in the clean link data, represents the position encoding dimension, Represents dimension index; The link identification submodule is used to obtain the manually annotated training malicious link data, input the training malicious link data into a preset identification model for training, input the encoding features into the trained preset identification model for malicious link identification, and output a second identification result.

[0053] The third identification module 4 includes: The scale submodule is used to define the spatial scale of the page image information: ; In the formula, represents a two-dimensional Gaussian function, Indicates the page image in the page image information at position The pixel value at Represents scale space; The sampling submodule is used to construct a Gaussian pyramid based on the spatial scale, perform downsampling processing on the page image information based on the Gaussian pyramid and using different spatial scales, and use Gaussian difference space extreme point detection to obtain a number of detection extreme points, wherein the Gaussian difference space for: ; In the formula, Respectively represent the group number index and the layer number index within the group; The offset submodule is used to determine the Gaussian offset value of each extreme point : ; In the formula, represents the Gaussian difference space for detecting extreme points, represents the coordinates of the detected extreme points, Represents the scale space corresponding to the detected extreme point; A removal submodule is used to remove extreme points whose Gaussian offset value is not greater than a preset offset value, and to retain extreme points whose Gaussian offset value is greater than the preset offset value, so as to obtain feature points; The feature submodule is used to determine the image feature based on the feature point, and determine the third recognition result according to the image feature.

[0054] The feature submodule includes: A gradient unit, used to determine a pixel area with the feature point as the center, and determine a horizontal gradient and a vertical gradient of each pixel in the pixel area, and determine a gradient matrix based on the horizontal gradient and the vertical gradient; An eigenvalue unit, used to determine the covariance matrix of the gradient matrix and calculate the eigenvalues ​​and responses of the covariance matrix to obtain an eigenvector; An arrangement unit, used for arranging the eigenvalues ​​of the covariance matrix in descending order, selecting the first several eigenvalues ​​as matrix eigenvalues ​​and determining a characteristic matrix according to eigenvectors corresponding to the matrix eigenvalues; A fusion unit, used for multiplying the gradient matrix with the feature matrix to obtain a descriptor, and fusing the descriptor with the feature point to obtain an image feature; The image recognition unit is used to obtain manually annotated training malicious image data, input the training malicious image data into a preset image recognition model for training, input image features into the trained preset image recognition model for malicious image recognition, and output a third recognition result.

[0055] The output module 5 comprises: A first abnormality output module, configured to output a final recognition result of no abnormality if the first recognition result is no page behavior abnormality, the second recognition result is no link abnormality, and the third recognition result is no image abnormality; The second abnormality output module is used to output the corresponding abnormality type if the first recognition result is that the page behavior is abnormal, the second recognition result is that the link is abnormal, or the third recognition result is that the image is abnormal.

[0056] In some other embodiments of the present invention, the embodiments of the present invention provide the following technical solutions: a computer, comprising a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101, wherein the processor 101 implements the above-mentioned Internet portal website abnormality monitoring method when executing the computer program.

[0057] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0058] Among them, the memory 102 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 102 may be inside or outside the data processing device. In a specific embodiment, the memory 102 is a non-volatile memory. In a specific embodiment, the memory 102 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0059] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0060] The processor 101 implements the above-mentioned Internet portal website abnormality monitoring method by reading and executing the computer program instructions stored in the memory 102.

[0061] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 3 As shown, the processor 101, the memory 102, and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0062] The communication interface 103 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present invention. The communication interface 103 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0063] The bus 100 includes hardware, software or both, and couples the components of the computer device to each other. The bus 100 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.

[0064] The computer can execute the Internet portal website anomaly monitoring method of the present invention based on the acquired Internet portal website anomaly monitoring system, thereby realizing Internet portal website anomaly monitoring.

[0065] In some further embodiments of the present invention, in combination with the above-mentioned Internet portal website anomaly monitoring method, the embodiments of the present invention provide the following technical solutions: a storage medium having a computer program stored thereon, and the computer program implements the above-mentioned Internet portal website anomaly monitoring method when executed by a processor.

[0066] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0067] More specific examples of readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0068] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0069] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for monitoring abnormalities of an Internet portal website, characterized in that: include: Obtaining website information of the target website, the website information including page behavior information, website link information and page image information; Performing sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determining a first recognition result based on the sensitive text ratio; Extracting features and identifying malicious links on the website link information to output a second identification result; Performing feature description and malicious image recognition on the page image information to output a third recognition result; Outputting a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result, and the third recognition result; The step of performing sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determining a first recognition result based on the sensitive text ratio includes: Extracting specific tag content from the page behavior information, and removing special characters from the specific tag content to obtain uncluttered tag content, and segmenting the uncluttered tag content to obtain a number of page text information; Establishing a vertex table based on the plurality of page text information, wherein the vertex table is used to record the header address of the single linked list corresponding to the page text information; Starting from state 0, the finite state automaton established by breadth-first search establishes corresponding nodes for the next state that can be jumped to, and links the corresponding nodes to the singly linked list of state 0; Calculate the hash value for the first byte of the character of each node in the single linked list of state 0, and link the corresponding node to the hash table according to the hash value to obtain a hash linked list; Calculate the state transition function and corresponding failure function of the next arbitrary state A starting from state 0; Determine the output function of state A based on the state transition function and the failure function, and add the keyword pattern string to the output function of state A after processing each keyword pattern string; Initialize state A to state 0 and point the text string pointer to the first character of the text string. If the text string pointer is not empty, output the pointed character K. A sensitive text ratio is determined based on the character K, and a first recognition result is determined according to the sensitive text ratio.

2. The method for monitoring abnormalities of an Internet portal website according to claim 1, characterized in that: The step of determining the sensitive text ratio based on the character K and determining the first recognition result according to the sensitive text ratio includes: Calculate the state transition function of state A and character K, and determine state B based on the state transition function of state A and character K. If state B is 0, search the corresponding single-column table in the hash list according to the hash value of the character and determine whether state B is fail; If state B is fail, state A is the cutoff state and the corresponding output function is output. If state B is not fail, state A is different from state B and the corresponding output function is output. It is determined whether the corresponding output function is empty. If the corresponding output function is empty, initialize state B to state 0 and repeat the calculation and judgment process of the state transfer function. If the corresponding output function is not empty, directly output the corresponding output function. Determine the matching sensitive text based on the output function, and match the matching sensitive text with the sensitive text library to determine the sensitive text ratio; If the sensitive text ratio exceeds the preset sensitive text ratio, the target website has page behavior abnormalities; if the sensitive text ratio does not exceed the preset sensitive text ratio, the target website does not have page behavior abnormalities, so as to obtain a first identification result.

3. The method for monitoring abnormalities of an Internet portal website according to claim 1, characterized in that: The step of extracting features and identifying malicious links from the website link information to output a second identification result includes: Performing data cleaning on the website link information to remove abnormal link data to obtain cleaned link data, wherein the abnormal link data includes random domain names with abnormal vowel ratios, data with abnormal character set discreteness, data with abnormal domain name length, and link data with excessively high frequency of switching between numbers and letters; The cleaned link data is input into a preset encoder and passed through a self-attention module in the preset encoder to obtain a weighted feature vector : ； In the formula, , , Represent the query vector, key vector, and value vector respectively. represents the dimension of the key vector; The weighted feature vector is input into the fully connected layer in the preset encoder to output the activation feature : ； In the formula, , , Represent the first and second encoder coefficients respectively, , Respectively represent the first and second encoder constants; Get the first relative position code of the word in the clean link data Or the second relative position code , encode the first relative position Relative position code with the second Add to the activation feature To obtain the encoding features: ； ； In the formula, represents the position of the words in the clean link data in the clean link data, represents the position encoding dimension, Represents dimension index; Obtain manually annotated training malicious link data, input the training malicious link data into a preset recognition model for training, input the encoding features into the trained preset recognition model for malicious link recognition, and output a second recognition result.

4. The method for monitoring abnormalities of an Internet portal website according to claim 1, characterized in that: The step of performing feature description and malicious image recognition on the page image information to output a third recognition result includes: Define the spatial scale of the page image information: ； In the formula, represents a two-dimensional Gaussian function, Indicates the page image in the page image information at position The pixel value at Represents scale space; A Gaussian pyramid is constructed based on the spatial scale, and the page image information is downsampled based on the Gaussian pyramid and using different spatial scales, and Gaussian difference space extreme point detection is used to obtain a plurality of detection extreme points, wherein the Gaussian difference space for: ； In the formula, Respectively represent the group number index and the layer number index within the group; Determine the Gaussian shift value for each extreme point : ； In the formula, represents the Gaussian difference space for detecting extreme points, represents the coordinates of the detected extreme points, Represents the scale space corresponding to the detected extreme point; The extreme points whose Gaussian offset value is not greater than the preset offset value are eliminated, and the extreme points whose Gaussian offset value is greater than the preset offset value are retained to obtain feature points; An image feature is determined based on the feature point, and a third recognition result is determined according to the image feature.

5. The method for monitoring abnormalities of an Internet portal website according to claim 4, characterized in that: The step of determining the image features based on the feature points and determining the third recognition result according to the image features comprises: Determine a pixel area with the feature point as the center, determine the horizontal gradient and the vertical gradient of each pixel in the pixel area, and determine a gradient matrix based on the horizontal gradient and the vertical gradient; Determine the covariance matrix of the gradient matrix and calculate the eigenvalues ​​and responses of the covariance matrix to obtain eigenvectors; Arrange the eigenvalues ​​of the covariance matrix in descending order, select the first several eigenvalues ​​as matrix eigenvalues, and determine the characteristic matrix according to the eigenvectors corresponding to the matrix eigenvalues; Multiplying the gradient matrix by the feature matrix to obtain a descriptor, and fusing the descriptor with the feature points to obtain image features; Obtain manually annotated training malicious image data, input the training malicious image data into a preset image recognition model for training, input image features into the trained preset image recognition model for malicious image recognition, and output a third recognition result.

6. The method for monitoring abnormalities of an Internet portal website according to claim 1, characterized in that: The step of outputting the final recognition result and the affected abnormality type based on the first recognition result, the second recognition result and the third recognition result comprises: If the first recognition result is that there is no page behavior anomaly, the second recognition result is that there is no link anomaly, and the third recognition result is that there is no image anomaly, then the final recognition result is that there is no anomaly; If the first recognition result is that there is a page behavior anomaly, or the second recognition result is that there is a link anomaly, or the third recognition result is that there is a picture anomaly, then the final recognition result is that there is an anomaly, and the corresponding anomaly type is output.

7. An Internet portal website abnormality monitoring system, characterized in that: The system comprises: An acquisition module is used to acquire website information of a target website, wherein the website information includes page behavior information, website link information, and page image information; A first recognition module is used to perform sensitive text recognition and matching on the page behavior information to obtain a sensitive text ratio, and determine a first recognition result based on the sensitive text ratio; A second identification module, used for extracting features and identifying malicious links from the website link information to output a second identification result; A third recognition module, used for performing feature description and malicious image recognition on the page image information to output a third recognition result; An output module is used to output a final recognition result and an affected abnormality type based on the first recognition result, the second recognition result and the third recognition result.

8. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for monitoring abnormalities of an Internet portal website according to any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for monitoring abnormalities of an Internet portal website according to any one of claims 1 to 6 is implemented.

Images