Method, device and electronic equipment for detecting website access request

Abstract

The application provides a website access request detection method and device and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: obtaining a target access sequence of a target user, the target access sequence comprising a plurality of target URLs corresponding to a target website and sorted according to access time; obtaining a target weight corresponding to each of the plurality of target URLs; obtaining a plurality of target transition probabilities corresponding to the target access sequence according to a stored corresponding relationship between a website and a page state transition probability matrix, the page state transition probability matrix comprising transition probabilities between pages of a website when the pages of the website are normally accessed; and determining whether an access request corresponding to the target access sequence is an abnormal access request according to the plurality of target weights and the plurality of target transition probabilities. In this way, abnormal access can be identified according to a page state transition probability matrix corresponding to normal user access and a target access sequence of a target user, thereby reducing the situation of missing abnormal access.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, and electronic device for detecting website access requests. Background Technology

[0002] With the widespread adoption of the internet, websites have gradually become bridges for various industries to showcase themselves and communicate information. However, websites receive a variety of abnormal requests, such as CC attacks, website scans, and DDoS attacks. Existing protection methods mainly use web firewalls, but these primarily rely on coarse-grained rule blocking and are ineffective at intercepting fine-grained illegal access requests. Current detection methods generally analyze the request information of a single access request to determine whether it is abnormal; however, this approach can miss some abnormal access requests. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and readable storage medium for detecting website access requests. It can identify abnormal access based on the page state transition probability matrix corresponding to normal user access and the target user's target access sequence, thereby reducing the situation where access requests from some illegal crawlers or scanning scenarios are identified as normal accesses when judging only a single access.

[0004] The embodiments of this application can be implemented as follows:

[0005] In a first aspect, embodiments of this application provide a method for detecting website access requests, the method comprising:

[0006] Obtain the target access sequence of the target user, wherein the target access sequence includes multiple target URLs of the corresponding target website sorted by access time;

[0007] Obtain the target weight corresponding to each of the multiple target URLs;

[0008] Based on the correspondence between the stored website and page state transition probability matrix, multiple target transition probabilities corresponding to the target access sequence are obtained, wherein the page state transition probability matrix includes the transition probabilities between pages of the website when accessing the website normally.

[0009] Based on the multiple target weights and the multiple target transition probabilities, it is determined whether the access request corresponding to the target access sequence is an abnormal access request.

[0010] Secondly, embodiments of this application provide a website access request detection device, the device comprising:

[0011] The sequence acquisition module is used to obtain the target access sequence of the target user, wherein the target access sequence includes multiple target URLs of the corresponding target website sorted by access time;

[0012] The weight acquisition module is used to obtain the target weight corresponding to each of the multiple target URLs;

[0013] The probability acquisition module is used to obtain multiple target transition probabilities corresponding to the target access sequence based on the correspondence between the stored website and page state transition probability matrix, wherein the page state transition probability matrix includes the transition probabilities between pages of the website when accessing the website normally.

[0014] The judgment module is used to determine whether the access request corresponding to the target access sequence is an abnormal access request based on the multiple target weights and the multiple target transition probabilities.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the website access request detection method described in the foregoing embodiments.

[0016] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the website access request detection method as described in the foregoing embodiments.

[0017] The website access request detection method, apparatus, electronic device, and readable storage medium provided in this application first obtain a target user's target access sequence. Then, based on the stored correspondence between website and page state transition probability matrices, multiple target transition probabilities corresponding to the target access sequence are obtained, as well as target weights corresponding to multiple target URLs. The target access sequence includes multiple target URLs of the corresponding target website, sorted by access time. The page state transition probability matrix includes transition probabilities between pages of the website when accessing pages normally. Finally, based on the multiple target weights and multiple target transition probabilities, it is determined whether the access request corresponding to the target access sequence is an abnormal access request. In this way, abnormal access can be identified based on the page state transition probability matrix corresponding to normal user access and the target user's target access sequence, thereby reducing the possibility of misidentifying access requests from illegal crawlers or scanners as normal access when judging only a single access. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;

[0020] Figure 2 This is one of the flowcharts illustrating a method for detecting website access requests provided in an embodiment of this application.

[0021] Figure 3 A second schematic flowchart illustrating the method for detecting website access requests provided in this application embodiment;

[0022] Figure 4 for Figure 3 A flowchart illustrating the sub-steps included in step S110;

[0023] Figure 5 The third flowchart illustrating the method for detecting website access requests provided in this application embodiment;

[0024] Figure 6 for Figure 2 A flowchart illustrating the sub-steps included in step S140;

[0025] Figure 7 for Figure 2 A flowchart illustrating the sub-steps included in step S150;

[0026] Figure 8 for Figure 2 A flowchart illustrating the sub-steps included in step S160;

[0027] Figure 9 A fourth flowchart illustrating the method for detecting website access requests provided in this application embodiment;

[0028] Figure 10 One of the block diagrams of a website access request detection device provided in an embodiment of this application;

[0029] Figure 11 This is a second block diagram of a website access request detection device provided in an embodiment of this application.

[0030] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication unit; 200 - Website access request detection device; 210 - Matrix acquisition module; 220 - Probability setting module; 230 - Sequence acquisition module; 240 - Weight acquisition module; 250 - Probability acquisition module; 260 - Judgment module. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0032] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0033] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0034] Existing website access detection technologies primarily rely on request information from a single URL to construct a feature vector. This feature vector is then used to identify whether the request information is illegitimate, thus determining abnormal access. However, because this approach only considers a single URL request and does not integrate the user's entire access sequence, it may mistakenly identify requests from illegal crawlers or scanners as legitimate accesses. Consequently, it may miss users whose individual URLs appear legitimate but whose overall behavior is abnormal.

[0035] For example, from the perspective of a single visit, a user's access to a webpage may seem normal. However, if the user continuously accesses the same webpage, it could be a DDoS attack, thus constituting abnormal access behavior. Another example is a user visiting both url_1 and url_3. Normal users wouldn't normally be redirected directly from url_1 to url_3; the redirection logic is illogical, and if multiple such instances occur, this group of users also constitutes abnormal access, meaning their access is considered anomaly.

[0036] To address the above issues, this application provides a method, apparatus, electronic device, and readable storage medium for detecting website access requests. These methods can identify abnormal access based on the page state transition probability matrix corresponding to normal user access and the target user's target access sequence, thereby reducing the possibility of misidentifying access requests from illegal crawlers or scanners as normal accesses when judging only a single access.

[0037] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0038] Please refer to Figure 1 , Figure 1 This is a block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may be, but is not limited to, a computer, a server, etc. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, processor 120, and communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0039] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0040] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, the memory 110 stores a website access request detection device 200, which includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running the software programs and modules stored in the memory 110, such as the website access request detection device 200 in this embodiment, thereby implementing the website access request detection method in this embodiment.

[0041] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0042] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0043] Please refer to Figure 2 , Figure 2 This is one of the flowcharts illustrating a method for detecting website access requests provided in this application. The method can be applied to the aforementioned electronic device 100. The specific flow of the method for detecting website access requests is described in detail below. In this embodiment, the method may include steps S130 to S160.

[0044] Step S130: Obtain the target access sequence of the target user.

[0045] In this embodiment, the target user is the user whose website access request needs to be detected. The specific target user is determined by actual requirements; for example, every user who accesses the website can be considered a target user. The access sequence of the target user to the target website can be obtained based on the target user's website access records, and this access sequence is used as the target user's target access sequence. The target access sequence includes multiple target URLs (Universal Resource Locators) corresponding to the target website, ordered by access time. Each target URL corresponds to a webpage on the target website.

[0046] It is worth noting that the target access sequence corresponds to a single website; that is, all accesses represented by the target access sequence are for accessing a specific website. In other words, the target website is the website corresponding to the obtained target access sequence for the same website, and this specific target website is determined by the actual situation. In this embodiment, the method for detecting website access requests is illustrated using the example of a user accessing a single website. However, it is understood that this method can be used to detect accesses to any website by any user.

[0047] Step S140: Obtain the target weights corresponding to each of the multiple target URLs.

[0048] Given the target access sequence, the target weights corresponding to each of the multiple target URLs in the target access sequence can be obtained in any way. For example, the target weights corresponding to each target URL can be pre-set, or they can be determined based on the access information corresponding to each target URL, etc., depending on the actual needs.

[0049] Step S150: Based on the correspondence between the stored website and page state transition probability matrices, obtain multiple target transition probabilities corresponding to the target access sequence.

[0050] The transition probability matrix was proposed by the mathematician Markov. He discovered that in the transition of certain factors in a system, the result of the nth transition is only affected by the result of the (n-1)th transition, meaning it is only related to the current state and not to past states. Markov analysis introduces the concept of state transition. A state refers to the possible or existing state of an objective thing; the state transition probability is the probability that an objective thing will transition from one state to another.

[0051] In this embodiment, upon obtaining the target access sequence, multiple target transition probabilities corresponding to the target access sequence can be obtained based on the pre-obtained correspondence between the website and page state transition probability matrix and the target website corresponding to the target access sequence. The page state transition probability matrix includes the transition probabilities between pages of the website during normal access; that is, the page state transition probability matrix includes the probability that one page of the website in the normal access path will transition to another page of the website. The page state transition probability matrix can also be pre-obtained by the electronic device 100 based on normal user access records, or by other devices based on normal user access records, or it can be pre-set by staff based on experience. The specific method of obtaining the page state transition probability matrix can be determined according to actual needs.

[0052] It is worth noting that steps S130 and S140 can be executed simultaneously or sequentially, depending on the actual needs.

[0053] Step S160: Based on the multiple target weights and the multiple target transition probabilities, determine whether the access request corresponding to the target access sequence is an abnormal access request.

[0054] Given multiple target weights and multiple target transition probabilities, the access request corresponding to the target access sequence can be analyzed to determine whether it is an abnormal access request. If the access request corresponding to the target access sequence is determined to be an abnormal access request, the target user can be identified as an abnormal user, and appropriate protective measures can then be taken.

[0055] In this way, abnormal access can be identified based on the page state transition probability matrix corresponding to normal user access and the target user's target access sequence, thereby reducing the possibility of misidentifying access requests from illegal crawlers or scanners as normal accesses when judging based on only a single access.

[0056] Please refer to Figure 3 , Figure 3 This is a second schematic flowchart illustrating the website access request detection method provided in this application embodiment. In this embodiment, before step S150, the method may further include step S110.

[0057] Step S110: For each website, calculate the page state transition probability matrix corresponding to that website based on the access records of each normal user to that website.

[0058] In this embodiment, access records of each normal user to each website can be obtained from user access log data to analyze and obtain the corresponding page state transition probability matrix for each website. That is, for each website, based on the recorded normal access information, the state transition probability between different URLs of that website is calculated. The identification of normal users can be based on relevant rules and the analysis of logs by security experts. For example, the user's access duration, the number of pages accessed, and whether the accessing user proxy is normal can be used to determine whether the user is a normal user. In this embodiment, an accessing IP address can be considered as one accessing user.

[0059] Optionally, as a possible implementation, for each website, based on the access records of each normal user to the website, the transition probabilities between the URLs included in the access records of each normal user to the website can be analyzed, and then the page state transition probability matrix of the website can be obtained by summarizing them.

[0060] Alternatively, as another possible implementation, it can be achieved through... Figure 4 The method shown yields the page state transition probability matrix for each website. Please refer to [the provided text]. Figure 4 , Figure 4 for Figure 3 A flowchart illustrating the sub-steps included in step S110. In this embodiment, step S110 may include sub-steps S111 to S112.

[0061] Sub-step S111: Calculate the initial page state transition probability matrix corresponding to the website based on the access records of each normal user to the website.

[0062] In sub-step S112, if there are cases where some pages have no transition probability in the initial page state transition probability matrix, the transition probability is filled, and each row element in the filled initial page state transition probability matrix is ​​normalized to obtain the page state transition probability matrix corresponding to the website.

[0063] In this embodiment, basic characteristic information of the website can first be extracted from user access log data and then saved to the website knowledge base. Optionally, the website knowledge base includes: a dataset of webpage URLs from each website, that is, storing the correspondence between websites and the webpage URLs included in the websites; and basic information of each webpage URL, such as whether the URL contains static resources, the URL submission method, whether the URL contains parameters, and a reasonable parameter range. In actual access, URL submission methods include POST, GET, DELETE, and HEAD.

[0064] Next, when obtaining the page state transition probability matrix of a website, the dataset of all web page URLs of each website can be extracted from the website knowledge base, that is, the URLs included in each website can be determined so as to subsequently determine the transition probabilities between the URLs included in each website.

[0065] Furthermore, for each website, the URL sequence corresponding to each normal user on that website can be obtained from the user access log data, i.e., the second normal user access sequence for that website. Optionally, after obtaining the second normal user sequence, if at least three consecutive URLs are identical in the second normal user sequence, these at least three consecutive URLs can be processed into two consecutive URLs before being used in subsequent probability calculations. This reduces the transition probability between identical web pages, preventing the transition probability of other web pages from being almost negligible due to the high transition probability between identical web pages. For example, if a second normal user sequence obtained from the user access log data is (a,a,a,b,c), it can be processed into (a,a,b,c).

[0066] Whether two URLs are identical can be determined based on specific rules. For example, two URLs can be considered identical only if they are completely identical. Alternatively, they can be considered identical if they are completely identical or differ only in the last one or two characters. For instance, if each page of an article is represented by a separate page, meaning the URLs of each page in the article differ only in the last digit, then the URLs of all pages in the article can be considered identical.

[0067] Since the next page a user visits when accessing a website is only related to the current page and is accessed from the current page, the transition probabilities between different pages on each website can be calculated based on the second normal user access sequence obtained above, forming the initial page state transition probability matrix for each website. For example, for the same website, if 10 users who accessed page A subsequently access page B and 20 users subsequently access page C, then the transition probability from page A to page B is 33.3%, and the transition probability from page A to page C is 66.6%.

[0068] The initial page state transition probability matrix calculated based on user visits to web pages may contain cases where some pages have no transition probability. For example, if a website is identified as including web pages A, B, C, and D based on a website knowledge base, and the second normal user access sequence of this website only reflects the transition between web pages A, B, and C, then the initial page state transition probability matrix obtained based on the second normal user access sequence of this website may contain cases where web pages A, B, C and web page D have no transition probability.

[0069] To avoid the aforementioned situation, when some pages in a website's initial page state transition probability matrix have no transition probability, transition probabilities can be padded. The padded transition probabilities can be default values, which can be relatively small, such as 0.01, to ensure that there are few weak links between web pages. For example, continuing the example above, the transition probability between web pages A, B, C and web page D can be padded to 0.01.

[0070] After padding, the sum of each row of the matrix will be greater than 1. Therefore, after padding is complete, the elements of each row in the initial page state transition probability matrix can be normalized to a sum of 1 to obtain the page state transition probability matrix corresponding to the website.

[0071] In this embodiment, a preset total probability can also be obtained in advance to determine whether the access represented by the target user's target access sequence is normal.

[0072] Optionally, only one preset total probability can be set in advance, meaning that this preset total probability is used for judgment regardless of the specific target website. Alternatively, multiple preset total probabilities can be set in advance, with each preset total probability corresponding to one website. Subsequently, the preset total probability corresponding to the target website can be selected from these multiple preset total probabilities, and then the access sequence represented by the target access sequence can be judged as normal based on the preset total probability corresponding to the target website.

[0073] Optionally, the preset total probability can be set in advance by staff based on experience, or it can be obtained by analyzing the access sequences of normal users, and can be determined according to actual needs.

[0074] As one possible implementation method, it can be achieved through Figure 5 Step S120 in the above steps obtains the preset total probability. Please refer to... Figure 5 , Figure 5 This is a third schematic flowchart illustrating the website access request detection method provided in this application embodiment. In this embodiment, before step S160, the method may include step S120.

[0075] Step S120: Based on the multiple first normal user access sequences corresponding to each website, the weights of each URL in each first normal user access sequence, and the multiple transition probabilities corresponding to each first normal user access sequence, the preset total probability corresponding to the website is obtained through analysis.

[0076] In this embodiment, for each website, the following can be obtained: a first normal user access sequence corresponding to a normal user's access to the website, the weights of each URL in each first normal user access sequence, and multiple transition probabilities corresponding to each first normal user access sequence. The normal probability value for each normal user can then be calculated. Subsequently, based on the distribution of the normal probability values ​​of normal users, the probability range corresponding to the normal user access sequence is determined, thereby determining the preset total probability for that website. For example, when analyzing based on the 3δ method, the range outside 3δ can be determined as the abnormal access range, and the probability value corresponding to 3δ can be used as the preset total probability. The preset total probability for each website includes the target preset total probability used to determine whether the access request corresponding to the target access sequence is an abnormal access request.

[0077] Optionally, the method for determining the first normal user access sequence can be the same as the method for determining the target access sequence, and the method for determining the weight corresponding to each URL in each first normal user access sequence can be the same as the method for determining the target weight corresponding to the target URL, thus ensuring that the preset total probability is applicable.

[0078] In this embodiment, log data of user visits to the website can be obtained first. The log data may include: access time, website host (i.e., website domain name), URL, access proxy, access IP, request method (i.e., the URL submission method), etc.

[0079] An access IP can be considered as an access user. Based on the data of the above users accessing the website, the access sequence of each user can be obtained as the target access sequence for each target user.

[0080] Optionally, for the target user, when the number of times the target user visits the target website is greater than or equal to a preset value, the target access sequence can be obtained based on the target user's access records to the target website. The preset value can be set according to actual needs, for example, the preset value is 10. In this way, when determining the weight of a URL based on the characteristics of user access, it avoids situations where infrequent user access makes feature extraction difficult, leading to poor identification. That is, when the number of times the target user visits the target website is low, the target user's access is not identified.

[0081] Optionally, when the number of times the target user accesses the target website is greater than or equal to a preset value, the URLs of the target websites accessed by the target user consecutively can be sorted in chronological order based on the target user's access records, and the sorting result can be used as the target access sequence.

[0082] When the number of times the target user visits the target website is greater than or equal to a preset value, these target URLs can be sorted based on the order in which they were accessed (i.e., target URLs). Then, the M most recent accessed URLs are taken from the current time t backwards to form the target access sequence. That is, the number of target URLs in the target access sequence is the preset value. For example, if the preset value is 10, then the 10 most recent accessed URLs taken from the current time t backwards are denoted as (u t-9 ,u t-8 ,u t-7 ,...u t This log extraction is performed using a sliding window approach, which allows for faster and more real-time calculation of the user's current state. In this case, when setting a preset total probability based on the recorded normal access patterns, the length of the first normal user access sequence can be the same as the length of the target access sequence, which also includes M URLs.

[0083] If the target access sequence is obtained, it can be accessed through... Figure 6 The method shown obtains the target weight corresponding to each of the multiple target URLs. Please refer to... Figure 6 , Figure 6for Figure 2 A flowchart illustrating the sub-steps included in step S140. In this embodiment, step S140 may include sub-steps S141 to S142.

[0084] Sub-step S141: For each target URL, obtain the feature vector corresponding to the target URL based on various information in the access information corresponding to the target URL.

[0085] Sub-step S142: Calculate the target weight corresponding to the target URL based on the feature vector corresponding to the target URL and the preset weights corresponding to the various information.

[0086] In this embodiment, for each target URL in the target access sequence, a feature vector corresponding to that target URL can be obtained through processing based on the access information corresponding to that target URL and the website knowledge base previously obtained from user log data. The access information corresponding to the target URL can be obtained from the access records. A feature vector corresponding to a target URL consists of 0s and 1s. Preset weights for various information items in the access information can be pre-set. Then, based on the preset weights of the various information items in the access information and the feature vectors obtained from those information items, a weighted summation is performed to obtain the target weight corresponding to that target URL. After performing the above processing on each target URL, the target weight corresponding to each target URL can be obtained.

[0087] For example, for each target URL, we can combine it with the website's knowledge base to determine whether the URL contains static resources, has a referrer, whether the submission method matches the records in the knowledge base, whether the time interval is reasonable, and whether the parameters are reasonable, thereby constructing a feature vector f = (f1, f2, ..., f...) corresponding to the target URL. n ).

[0088] The following examples illustrate some of the judgments involved in constructing feature vectors.

[0089] For example, regarding website submission methods, if the submission method corresponding to the target URL in the website knowledge base is POST, and the user actually uses POST when accessing the target URL, then the submission methods are considered consistent. Another example is whether the URL of the accessed page contains parameters. If the target URL in the website knowledge base contains parameters, and the user actually accesses the page with parameters, then the parameters are considered consistent. Yet another example is determining the parameter range. For a target URL of an accessed page, if the range of parameters corresponding to the target URL in the website knowledge base is 1-100, and the user actually accesses the page with 99 parameters, then the parameter ranges are considered consistent; if the parameter is 101, then the parameter ranges are different. It can be understood that when the above judgment results are yes, the corresponding feature value in the feature vector is 1; otherwise, it is 0.

[0090] Different weights can be assigned to features in advance based on their importance, i.e., w = (w1, w2, ..., w n ),and

[0091] Given the feature vector of a target URL and the preset weights corresponding to various information, the target weight of the target URL can be calculated by weighted summation. Thus, the weight of the target URL can be calculated by combining its features. The process of calculating the target weight based on the feature vector and the preset weights corresponding to various information can be represented as follows:

[0092]

[0093] For example, if the feature vector of a target URL is f = (1,0,1,1,0) and the weight of the feature vector is w = (0.2,0.2,0.2,0.2,0.2), then the target weight of the URL can be determined to be W = 0.6 by weighted summation.

[0094] For each target URL in the target access sequence, based on the feature vector corresponding to that target URL and the weight corresponding to that feature vector, the target weight of each target URL can be calculated, i.e., (Wu t-9 Wu t-8 ,...Wu t ).

[0095] Given the target access sequence, it can be obtained through Figure 7 The method shown obtains the transition probabilities of multiple targets. Please refer to... Figure 7 , Figure 7 for Figure 2A flowchart illustrating the sub-steps included in step S150. In this embodiment, step S150 may include sub-steps S151 to S152.

[0096] Sub-step S151: Based on the correspondence between the website and page state transition probability matrices, determine the target page state transition probability matrix corresponding to the target website.

[0097] Sub-step S152: Obtain the target transition probability corresponding to each adjacent target URL in the target access sequence from the target page state transition probability matrix, so as to obtain the multiple target transition probabilities.

[0098] In this embodiment, the page state transition probability matrix corresponding to the target website can first be determined based on the stored correspondence between website and page state transition probability matrices, and this matrix can be used as the target page state transition probability matrix. Then, based on the target URL jumps reflected in the target access sequence, the transition probability corresponding to each jump in the target access sequence is obtained from the target page state transition probability matrix, and this probability is used as the target transition probability. The specific target URL jumps are determined based on the arrangement of the target URLs in the target access sequence.

[0099] For example, if the target access sequence is: URL1, URL2, URL3, then two redirects are determined from the target access sequence: from URL1 to URL2 and from URL2 to URL3. Then, the target transition probability corresponding to the redirect from URL1 to URL2 and the target transition probability corresponding to the redirect from URL2 to URL3 can be obtained from the page state transition probability matrix. In this way, the target transition probability corresponding to each adjacent target URL in the target access sequence can be obtained, thus obtaining multiple target transition probabilities.

[0100] Given the target weights and transition probabilities corresponding to multiple target URLs, it is possible to... Figure 8 The method shown demonstrates how to identify whether an access request is abnormal. Please refer to [the instructions]. Figure 8 , Figure 8 for Figure 2 A flowchart illustrating the sub-steps included in step S160. In this embodiment, step S160 may include sub-steps S161 to S164.

[0101] Sub-step S161: Calculate the total probability based on the multiple target weights and the multiple target transition probabilities.

[0102] Sub-step S162: Determine whether the total probability is less than the target preset total probability.

[0103] When the total probability is less than the target preset total probability, sub-step S163 is executed.

[0104] Sub-step S63: Determine that the access request corresponding to the target access sequence is an abnormal access request.

[0105] When the total probability is not less than the target preset total probability, sub-step S164 is executed.

[0106] Sub-step S164: Determine that the access request corresponding to the target access sequence is a normal access request.

[0107] In this embodiment, the total probability can be calculated based on a preset calculation formula, according to the weights of multiple targets and the transition probabilities of the multiple targets. The total probability represents the probability that the target access sequence is a normal access. The preset calculation formula can be specifically set according to actual needs. For example, as a possible implementation, the preset calculation formula is:

[0108]

[0109] Where P represents the total probability, N represents the number of target URLs, and W... i+1 a represents the target weight corresponding to the (i+1)th target URL in the target access sequence. i This represents the target transition probability that the i-th target URL in the target access sequence is converted to the (i+1)-th target URL.

[0110] For example, suppose the target access sequence is: URL1, URL2, URL3, URL4, URL5, URL6, URL7, URL8, URL9, URL10; the target weights corresponding to each URL in the target sequence are W1, W2, W3, W4, W5, W6, W7, W8, W9, W 10 Where W1 represents the target weight corresponding to URL1, W2 represents the target weight corresponding to URL2, and so on; the obtained multiple target transition probabilities are: a1, a2, a3, a4, a5, a6, a7, a8, a9, where a1 represents the transition probability from URL1 to URL2 obtained from the target page state transition probability, a2 represents the transition probability from URL2 to URL3 obtained from the target page state transition probability, and so on. The calculated total probability is: a1*W2+a2*W3+a3*W4+a4*W5+a5*W6+a6*W7+a7*W8+a8*W9+a9*W 10 .

[0111] After obtaining the total probability of the target access sequence, this total probability can be compared with a target preset total probability. The target preset total probability can be manually set in advance based on experience. Optionally, multiple preset total probabilities can be manually set for each website, and then the preset total probability corresponding to the target website can be selected as the target preset total probability; alternatively, only one preset total probability can be manually set, and this preset total probability can be directly used as the target preset total probability. Alternatively, a preset total probability can be obtained by analyzing the normal user access patterns of each website, and this preset total probability can be used as the target preset total probability. Or, the preset total probabilities corresponding to each website can be obtained through step S120 described above, and then the preset total probability corresponding to the target website can be selected as the target preset total probability.

[0112] If the total probability of the target access sequence is less than the target preset total probability, then the access request corresponding to the target access sequence can be determined to be an abnormal access request, that is, the access represented by the target access sequence is abnormal. In this case, the target user can be marked as an abnormal user, and corresponding protection measures can be taken. If the total probability of the target access sequence is not less than the target preset total probability, then the access request corresponding to the target access sequence can be determined to be a normal access request.

[0113] For example, if the target has a preset total probability of θ, that is, the normal access range is [θ,1], then when the total probability P is less than θ, the target user can be determined to be an abnormal user; otherwise, it is determined to be a normal user.

[0114] Normal user website access is a sequential process with certain regularities. In this embodiment, the website's state transition matrix, i.e., the probability of navigation between different pages, is obtained based on the access sequences of normal users. Then, the probability range of normal users accessing these sequences is obtained, meaning that the sequences of normal users accessing the website also fall within this range. If a new user arrives, and they are normal, their access sequence probability value will be within the normal range; conversely, if they are an abnormal user, their access sequence will be outside the normal range. For example, if most users access the website in the sequence url_1, url_2, url_3, url_4, and a new user accesses the website in the sequence url_1, url_4, url_3, url_1, then their calculated access sequence probability value will be relatively lower than that of normal users, and they will be identified as an abnormal user.

[0115] Since website pages may be deleted or added, to ensure that the URLs corresponding to the saved page state transition probability matrix are consistent with the actual existing URLs, the method may further include, for example: Figure 9 The step S170 shown.

[0116] Step S170: When a page is added or deleted on any website, the corresponding page state transition probability matrix for that website is updated accordingly.

[0117] In this embodiment, when a page is added or deleted on any website, the page state transition probability matrix for that website can be re-analyzed based on the access data of normal users on that website; or the page state transition probability matrix for that website can be manually reset. In this way, the page state transition probability matrix for websites with added or deleted pages can be updated to ensure that the URLs corresponding to the page state transition probability matrix are consistent with the actual situation.

[0118] This application embodiment constructs a state transition probability matrix between web pages for each website based on the access path of a normal user's requested URL. Then, based on the user's website access sequence and the website's page state transition probability matrix, the total probability that the user's web page access sequence is a normal access is obtained. If the total probability value of the user's access web page sequence is lower than a set threshold, the user's access can be determined as abnormal access. In addition, this application embodiment tracks changes to website pages in real time, establishes a website knowledge base, and synchronously updates the corresponding web page state transition probability matrix when web pages are added or deleted, thereby better identifying abnormal user access.

[0119] As described above, in this embodiment, a state transition probability matrix is ​​constructed based on the distribution of web pages accessed by normal users, capturing the access paths of normal users. Furthermore, by combining this with a website knowledge base and monitoring real-time changes in website pages, the state transition probability matrix is ​​updated dynamically, allowing for dynamic tracking of normal user behavior. This embodiment, by incorporating user access sequence information, can better identify users with abnormal access. Moreover, by using only a certain number of the user's most recent accessed URLs for judgment, the computation time for identifying abnormal access can be significantly reduced.

[0120] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a website access request detection device 200 is given below. Optionally, the website access request detection device 200 can adopt the above-described... Figure 1 The device structure of the electronic device 100 shown. Further, please refer to... Figure 10 , Figure 10This is one of the block diagrams of a website access request detection device 200 provided in this application embodiment. It should be noted that the website access request detection device 200 provided in this embodiment has the same basic principle and technical effects as the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. In this embodiment, the website access request detection device 200 may include: a sequence acquisition module 230, a weight acquisition module 240, a probability acquisition module 250, and a judgment module 260.

[0121] The sequence acquisition module 230 is used to obtain the target access sequence of the target user. The target access sequence includes multiple target URLs of the corresponding target website, sorted by access time.

[0122] The weight acquisition module 240 is used to obtain the target weight corresponding to each of the multiple target URLs.

[0123] The probability acquisition module 250 is used to obtain multiple target transition probabilities corresponding to the target access sequence based on the stored correspondence between website and page state transition probability matrices. The page state transition probability matrix includes the transition probabilities between pages of the website during normal access.

[0124] The judgment module 260 is used to determine whether the access request corresponding to the target access sequence is an abnormal access request based on the multiple target weights and the multiple target transition probabilities.

[0125] Please refer to Figure 11 , Figure 11 This is a second block diagram of a website access request detection device 200 provided in an embodiment of this application. In this embodiment, the website access request detection device 200 may further include a matrix acquisition module 210.

[0126] The matrix acquisition module 210 is used to calculate the page state transition probability matrix corresponding to each website based on the access records of each normal user to the website.

[0127] Please refer to this again. Figure 11 In this embodiment, the website access request detection device 200 may further include a probability setting module 220.

[0128] The probability setting module 220 is used to analyze and obtain the preset total probability corresponding to the website based on the multiple first normal user access sequences corresponding to each website, the weights of each URL in each first normal user access sequence, and the multiple transition probabilities corresponding to each first normal user access sequence. The preset total probability corresponding to each website includes the target preset total probability.

[0129] Optionally, in an embodiment, the matrix obtaining module 210 is further configured to update the page state transition probability matrix corresponding to any website when a page is added or deleted.

[0130] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0131] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for detecting website access requests.

[0132] In summary, this application provides a method, apparatus, electronic device, and readable storage medium for detecting website access requests. First, a target access sequence of a target user is obtained. Then, based on the correspondence between a stored website and page state transition probability matrix, multiple target transition probabilities corresponding to the target access sequence are obtained, as well as target weights corresponding to multiple target URLs. The target access sequence includes multiple target URLs of the corresponding target website, sorted by access time. The page state transition probability matrix includes transition probabilities between pages of the website when accessing pages normally. Finally, based on the multiple target weights and multiple target transition probabilities, it is determined whether the access request corresponding to the target access sequence is an abnormal access request. In this way, abnormal access can be identified based on the page state transition probability matrix corresponding to normal user access and the target user's target access sequence, thereby reducing the possibility of misidentifying access requests from illegal crawlers or scanners as normal access when judging only a single access.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0134] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0135] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] The above description is merely an optional embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting website access requests, characterized in that, The method includes: Obtain the target access sequence of the target user, wherein the target access sequence includes multiple target URLs of the corresponding target website sorted by access time; Obtaining the target weights corresponding to each of the multiple target URLs includes: for each target URL, obtaining a feature vector corresponding to the target URL based on various information in the access information corresponding to the target URL, wherein the various information includes: whether the URL contains static resources, whether there is a referrer, whether the submission method is consistent with the record in the knowledge base, whether the time interval is reasonable, and whether the parameters are reasonable; calculating the target weight corresponding to the target URL based on the feature vector corresponding to the target URL and the preset weights corresponding to the various information. Based on the correspondence between the stored website and page state transition probability matrices, multiple target transition probabilities corresponding to the target access sequence are obtained. The page state transition probability matrix includes the transition probabilities between pages of the website when accessing pages of the website normally. The target transition probability is the transition probability between adjacent target URLs. Based on the multiple target weights and the multiple target transition probabilities, the total probability is calculated to determine whether the access request corresponding to the target access sequence is an abnormal access request.

2. The method according to claim 1, characterized in that, The step of calculating the total probability based on multiple target weights and multiple target transition probabilities, and determining whether the access request corresponding to the target access sequence is an abnormal access request, includes: Determine whether the total probability is less than the target preset total probability; When the total probability is less than the target preset total probability, the access request corresponding to the target access sequence is determined to be an abnormal access request; When the total probability is not less than the target preset total probability, the access request corresponding to the target access sequence is determined to be a normal access request.

3. The method according to claim 2, characterized in that, The total probability is calculated using the following preset formula: Where P represents the total probability, and N represents the number of target URLs. This represents the target weight corresponding to the (i+1)th target URL in the target access sequence. This represents the target transition probability that the i-th target URL in the target access sequence is converted to the (i+1)-th target URL.

4. The method according to claim 2, characterized in that, Before determining whether the access request corresponding to the target access sequence is an abnormal access request based on the multiple target weights and the multiple target transition probabilities, the method further includes: Based on the multiple first normal user access sequences corresponding to each website, the weights of each URL in each first normal user access sequence, and the multiple transition probabilities corresponding to each first normal user access sequence, the preset total probability corresponding to the website is obtained through analysis, wherein the preset total probability corresponding to each website includes the target preset total probability.

5. The method according to claim 1, characterized in that, The method for obtaining the target user's target access sequence includes: For the target user, when the number of times the target user visits the target website is greater than or equal to a preset value, the target access sequence is obtained based on the target user's access records to the target website, wherein the number of target URLs in the target access sequence is the preset value.

6. The method according to any one of claims 1-5, characterized in that, Before obtaining multiple target transition probabilities corresponding to the target access sequence based on the correspondence between the stored website and page state transition probability matrix, the method further includes: For each website, the page state transition probability matrix corresponding to that website is calculated based on the access records of each normal user.

7. The method according to claim 6, characterized in that, The step of calculating the page state transition probability matrix corresponding to the website based on the access records of each normal user includes: Based on the access records of each normal user to the website, the initial page state transition probability matrix corresponding to the website is calculated; If there are cases where there is no transition probability between some pages in the initial page state transition probability matrix, the transition probability is filled, and each element in the filled initial page state transition probability matrix is ​​normalized to obtain the page state transition probability matrix corresponding to the website.

8. The method according to any one of claims 1-5, characterized in that, The method further includes: When a page is added or deleted on any website, the corresponding page state transition probability matrix for that website is updated.

9. A device for detecting website access requests, characterized in that, The device includes: The sequence acquisition module is used to obtain the target access sequence of the target user, wherein the target access sequence includes multiple target URLs of the corresponding target website sorted by access time; The weight acquisition module is used to obtain the target weights corresponding to each of the multiple target URLs, including: for each target URL, obtaining the feature vector corresponding to the target URL based on various information in the access information corresponding to the target URL, wherein the various information includes: whether the URL contains static resources, whether there is a referrer, whether the submission method is consistent with the record in the knowledge base, whether the time interval is reasonable, and whether the parameters are reasonable; and calculating the target weight corresponding to the target URL based on the feature vector corresponding to the target URL and the preset weights corresponding to the various information. The probability acquisition module is used to obtain multiple target transition probabilities corresponding to the target access sequence based on the correspondence between the stored website and page state transition probability matrix. The page state transition probability matrix includes the transition probabilities between pages of the website when accessing pages of the website normally. The target transition probability is the transition probability between adjacent target URLs. The judgment module is used to calculate the total probability based on the multiple target weights and the multiple target transition probabilities, and to determine whether the access request corresponding to the target access sequence is an abnormal access request.

10. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the website access request detection method according to any one of claims 1-8.

11. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for detecting website access requests as described in any one of claims 1-8.

Images