Regular expression generation method, device, equipment and storage medium

By generating regular expressions based on frequency statistics to recognize and converge URLs, the accuracy and efficiency of recognition and convergence under dynamic changes of URLs are solved, and the use of human resources is reduced.

CN115130023BActive Publication Date: 2024-12-03ALIBABA (CHINA) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210800221.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-08
Publication Date
2024-12-03
Estimated Expiration
2042-07-08

AI Technical Summary

Technical Problem

The prior art is difficult to adapt to the dynamic changes of URLs, resulting in low accuracy, real-time and efficiency of identifying and converging divergent URLs, and requires a large amount of manual resources.

Method used

By obtaining at least two character segments predivided by each URL in the URL set, counting the content frequency of each character segment, and generating regular expressions based on the frequency to identify and converge divergent URLs.

Benefits of technology

It improves the accuracy and efficiency of URL recognition, realizes real-time convergence of URLs, reduces the cost of analysis, storage and display on the server, and reduces the waste of manual resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115130023B_ABST
    Figure CN115130023B_ABST
Patent Text Reader

Abstract

The present application discloses a regular expression generation method, device, equipment and storage medium. The regular expression generation method obtains at least two character segments pre-divided from each URL in a uniform resource locator URL set, counts the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set, and generates a regular expression of the URL according to the frequency of occurrence of the content of each character segment in the URL set, so as to accurately identify the divergent character segments in the URL according to the regular expression of the URL, thereby converging its URL, improving the efficiency of identifying divergent URLs, retaining valid content, shielding useless information, and reducing the subsequent analysis, storage, and display costs of the server. In addition, the above method can effectively adapt to the dynamic changes of URLs, update the regular expression of URLs in real time, and ensure the accuracy and real-time performance of recognizing and converging divergent URLs based on the regular expression of URLs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a regular expression generation method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of Internet technology, the location and access method of resources can be obtained from the Internet through Uniform Resource Locator (URL). In the fields of front-end page rendering, search analysis, storage, etc., the relevant service call requests will contain a large number of parameters, status codes, and tens of thousands of secondary and tertiary sub-interfaces, thus diverging into countless different URLs.

[0003] In the related art, the method for identifying and converging divergent URLs is to manually write regular expressions based on experience or to identify and converge divergent URLs through preset pattern templates. However, these methods are difficult to adapt to the dynamic changes of URLs, and while failing to guarantee accuracy, real-time and efficiency, they also waste human resources. Summary of the invention

[0004] The embodiments of the present application provide a regular expression generation method, apparatus, device and storage medium, which can effectively identify divergent URLs and converge them, thereby improving the accuracy, real-time and efficiency of identifying divergent URLs.

[0005] According to a first aspect of an embodiment of the present application, a regular expression generation method is provided, comprising:

[0006] Obtain a uniform resource locator URL set, each URL in the URL set includes at least two pre-divided character segments;

[0007] Count the frequency of occurrence of the content of each character segment in at least two character segments in the URL set;

[0008] Generate a regular expression for the URL based on the frequency with which the content of each character segment appears in the URL set.

[0009] According to a second aspect of an embodiment of the present application, a data processing method based on a regular expression is provided, comprising:

[0010] Receive the target URL sent by the user;

[0011] Matching the target URL with the regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of the content of the character segment in the historical URL set, and the content of the character segment is the content of at least two character segments pre-divided by each URL in the historical URL set;

[0012] In the case where the matching result indicates that the target URL matches the regular expression of the URL, the target URL is converged to the target regular expression corresponding to the target URL.

[0013] According to a third aspect of an embodiment of the present application, a regular expression generating device is provided, including:

[0014] An acquisition module, used for acquiring a set of uniform resource locators (URLs), each URL in the set of URLs comprising at least two pre-divided character segments;

[0015] A statistics module, used for counting the frequency of occurrence of the content of each character segment in at least two character segments in the URL set;

[0016] The generating module is used to generate a regular expression of the URL according to the frequency of occurrence of the content of each character segment in the URL set.

[0017] According to a fourth aspect of an embodiment of the present application, a data processing device based on a regular expression is provided, comprising:

[0018] A receiving module, used for receiving a target URL sent by a user terminal;

[0019] A matching module, used for matching the target URL with a regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of the content of the character segment in the historical URL set, and the content of the character segment is the content of at least two character segments pre-divided by each URL in the historical URL set;

[0020] The convergence module is used to converge the target URL to a target regular expression corresponding to the target URL when the matching result indicates that the target URL matches the regular expression of the URL.

[0021] According to a fifth aspect of an embodiment of the present application, there is provided a computer device, including: a memory and a processor;

[0022] A memory for storing a computer program;

[0023] A processor is used to execute a computer program stored in a memory, and when the computer program is run, the processor executes the steps of the regular expression generation method as shown in the first aspect, or executes the steps of the regular expression-based data processing method as shown in the second aspect.

[0024] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a computer device, the computer device executes the steps of the regular expression generation method as shown in the first aspect, or executes the steps of the regular expression-based data processing method as shown in the second aspect.

[0025] According to a seventh aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program, which, when executed by a computer device, enables the computer device to execute the steps of the regular expression generation method as shown in the first aspect, or to execute the steps of the regular expression-based data processing method as shown in the second aspect.

[0026] The regular expression generation method, device, equipment and storage medium in the embodiments of the present application obtain at least two character segments pre-divided for each URL in the uniform resource locator URL set, count the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set, and generate the regular expression of the URL according to the frequency of occurrence of the content of each character segment in the URL set, so as to accurately identify the divergent character segments in the URL according to the regular expression of the URL, thereby converging its URL, improving the efficiency of identifying divergent URLs, retaining valid content, shielding useless information, and reducing the subsequent analysis, storage, and display costs of the server. In addition, the above method can effectively adapt to the dynamic changes of URLs, update the regular expression of URLs in real time, and ensure the accuracy and real-time performance of the regular expression based on URLs to identify and converge the divergent URLs. No human intervention is required in the above process, effectively reducing human resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present application can be better understood from the following description of specific embodiments of the present application in conjunction with the accompanying drawings, wherein the same or similar figure numbers represent the same or similar features.

[0028] Figure 1 is a schematic diagram showing a regular expression generation architecture according to an embodiment;

[0029] Figure 2 is a schematic diagram showing a regular expression generation process according to an embodiment;

[0030] Figure 3 is a schematic diagram showing a data processing architecture according to an embodiment;

[0031] Figure 4 is a flow chart showing a regular expression generation method according to one embodiment;

[0032] Figure 5is a flow chart showing a data processing method based on regular expressions according to one embodiment;

[0033] Figure 6 is a schematic structural diagram showing a regular expression generating device according to an embodiment;

[0034] Figure 7 is a schematic structural diagram showing a data processing device based on regular expressions according to an embodiment;

[0035] Figure 8 FIG. 4 is a schematic diagram showing the hardware structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0036] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0037] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0038] In the fields of front-end page rendering, search analysis, detection and storage, there is a problem of difficulty in converging high-cardinality indicators. Specifically, URLs (or URL interfaces) are divergent. The same and related service call requests contain a large number of parameters, status codes, and tens of thousands of secondary and tertiary sub-interfaces, which diverge into countless different URLs. This causes a lot of cost for subsequent analysis, storage, detection and display. Therefore, it is necessary to converge these divergent URLs, retain valuable content, shield useless information, and reduce storage costs, so as to improve the user's storage experience and reduce the user's storage costs.

[0039] In the related art, regular expressions can be written manually based on experience. This method relies on manual experience, is time-consuming and laborious, is difficult to adapt to the dynamic changes of URLs, and cannot guarantee accuracy, real-time and efficiency. In addition, the input URL list can be divided into multiple URL subsets according to a predetermined division rule, and the divided multiple URL subsets are aggregated into multiple URL classes according to a predetermined clustering rule, and a regular expression with certain characteristics is generated from the aggregated multiple URL classes. Although this method can divide URLs according to different division rules, and regular expressions generated by predetermined clustering rules, the clustering rules in this method need to be specified in advance and cannot be adjusted dynamically. In addition, the clustering rules implement clustering according to a distance threshold, so that the regular expression generated based on this method has low accuracy in identifying divergent URLs, and can only adapt to the scene of URL recognition with a small number of URLs, and is not suitable for complex scenes containing massive URLs and extremely divergent. It should be noted that the regular expression (RE) in the embodiment of the present application is also called a regular expression or a conventional expression, which is used to use a single string to describe and match a series of URLs that conform to a certain syntactic rule.

[0040] Therefore, the current methods for identifying and converging divergent URLs are mostly to manually write regular expressions based on experience or to identify and converge divergent URLs through preset pattern templates. However, these methods are difficult to adapt to the dynamic changes of URLs, and while failing to guarantee accuracy, real-time and efficiency, they also waste human resources.

[0041] Based on this, the regular expression generation method provided in the embodiment of the present application obtains at least two character segments pre-divided from each URL in the uniform resource locator URL set, counts the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set, and generates a regular expression for the URL based on the frequency of occurrence of the content of each character segment in the URL set, so as to accurately identify the divergent character segments in the URL according to the regular expression of the URL, thereby converging its URL, improving the efficiency of identifying divergent URLs, retaining valid content, shielding useless information, and reducing the subsequent analysis, storage, and display costs of the server. In addition, the above method can effectively adapt to the dynamic changes of URLs, update the regular expression of URLs in real time, and ensure the accuracy and real-time performance of recognizing and converging divergent URLs based on the regular expression of URLs. No human intervention is required in the above process, effectively reducing human resources.

[0042] Based on this, the following Figure 1 , the regular expression generation architecture provided in the embodiment of the present application is described in detail.

[0043] In one or more possible embodiments, Figure 1 As shown, the regular expression generation architecture 10 proposed in the embodiment of the present application may include a clustering module 101, a determination module 102 and a generation module 103. The regular expression generation architecture is described in detail below.

[0044] The clustering module 101 is used to identify similar character segments in multiple URLs (in the same batch), and cluster the URLs of similar character segments to obtain at least one URL set, so as to accurately identify which fields are diverging. In one example, the clustering module 101 clusters multiple URLs through a streaming clustering algorithm to obtain a URL set, and the similarity of at least two URLs in the URL set is greater than or equal to a preset similarity value. Here, multiple URLs are clustered by a streaming clustering algorithm without defining how many set classifications in advance, which is suitable for scenarios where the clustering scale and category are unknown in advance. Further, the streaming clustering algorithm can be based on the similarity of the measurement algorithm, and the URL is divided according to " / " such as [' / h5 / product / search / 1069240061'], and each character segment after the division is used as an independent string, and the similarity is measured by the Jacard similarity algorithm. The Jacard similarity algorithm is used to measure the difference between two URLs, that is, the more strings the two URLs have in common, the more similar they are. Therefore, after the URLs are clustered by the clustering module 101, similar URLs have been clustered into a URL set.

[0045] The determination module 102 is used to determine the divergence degree of each character segment in each URL in the URL set output by the clustering module 101. Here, the determination module 102 may include a first determination model and a second determination model.

[0046] Among them, the first determination model includes at least one of the following models: a pattern library and a vocabulary library model, and a frequency detection model. Specifically, the pattern library and the vocabulary library model, based on the common pattern of the URL, such as the existence of an MD5 string, uuid, date, etc., identify the possible divergent character segments in at least two character segments of the URL. In this way, by identifying the possible divergent character segments in at least two character segments of the URL through the pattern library and the vocabulary library model, the possibility of divergence in some positions can be better identified, reducing the amount of subsequent calculations. In addition, the pattern library and the vocabulary library model also provide for identifying the customized URL of the customized service or the character segments in the URL, which facilitates the pattern library and the vocabulary library model to identify the divergence of certain character segments, that is, to retain the valid content customized by different services. In one example, the pattern library and the vocabulary library model are used to determine whether the content of the character segment includes the preset content. When it is determined that the content of the character segment includes the preset content, the character segment is characterized as a customized URL of the customized service or a character segment in the URL, so that it does not need to be replaced with a preset identifier. The frequency detection model is used to identify the locations where the character segments in the URL are divergent by the frequency of occurrence of the content of each character segment in the URL set, so as to generate a module 103 to generate a regular expression for the URL according to the frequency of occurrence of the content of each character segment in the URL set.

[0047] The second determination model is used to further accurately identify whether the divergent character segment is meaningful through the natural language processing (NLP) algorithm, that is, whether the letters in the character segment are valid words or Chinese word pinyin, so as to converge the URL where the meaningless character segment is located in the later stage to obtain the regular expression of the URL. In one example, the second determination model identifies the letters in the first character segment through the natural language processing algorithm, obtains multiple target words composed of the letters in the first character segment, calculates the validity value of the first character segment based on each target word in the multiple target words and the first character segment, and determines whether the character segment is a divergent character segment based on the validity value.

[0048] The generating module 103 is used to generate a regular expression of the URL based on the output result of the determining module 102. In one example, according to the frequency of occurrence of the content of each character segment in the URL set, a first character segment of at least two character segments is obtained, and the frequency of occurrence of the content of the first character segment in the URL set is less than or equal to a preset frequency; the first character segment is replaced with a preset identifier; based on the preset identifier and the second character segment of the at least two character segments, a regular expression of the URL is generated; wherein the frequency of occurrence of the content of the second character segment in the URL set is greater than the preset frequency.

[0049] In addition, in another possible embodiment, the regular expression generation architecture 10 may further include a verification module 105 for verifying the regular expression of the URL generated by the generation module 103 to improve the accuracy of the regular expression of the URL.

[0050] Based on Figure 1 The regular expression generation architecture shown is used to explain in detail the regular expression generation method provided in the embodiment of the present application.

[0051] like Figure 2 As shown, first, the clustering module 101 selects URLs that meet the preset format from the multiple URLs of the user obtained, and clusters the multiple URLs through the churn clustering algorithm to obtain at least one URL set, wherein the similarity of at least two URLs in the URL set is greater than or equal to the preset similarity value, wherein each URL in the URL set includes at least two pre-divided character segments.

[0052] Next, the determination module 102 counts the frequency of occurrence of the content of each of the at least two character segments in the URL set, and based on the frequency of occurrence of the content of each character segment in the URL set, obtains the first character segment of the at least two character segments, and the frequency of occurrence of the content of the first character segment in the URL set is less than or equal to the preset frequency.

[0053] Then, the first character segment is replaced with a preset identifier through the generation module 103; based on the preset identifier and the second character segment of at least two character segments, a regular expression of the URL is generated; wherein the frequency of occurrence of the content of the second character segment in the URL set is greater than the preset frequency.

[0054] In this way, a large number of URLs can be clustered first, reducing the scale of URL recognition, improving the relevance of URL content recognition, and cleverly utilizing the combination of multiple models in the judgment module to improve the accuracy of recognizing divergent character segments in URLs, and using the second judgment model to recognize the meaning of words, further measuring the effectiveness of information, solving the problem of effectively divergent URLs and processing single batches, which are not divergent in large quantities but have meaningless positions. Service vulnerabilities improve the efficiency of identifying divergent URLs, retaining effective content, shielding useless information, and reducing the subsequent analysis, storage, and display costs of the server. In addition, the above method can be used to obtain URLs at any time, effectively adapt to the dynamic changes of URLs, and update the regular expression of URLs in real time to ensure the accuracy and real-time performance of URL-based regular expression recognition and convergence of divergent URLs. No human intervention is required in the above process, effectively reducing human resources.

[0055] Based on the above regular expression generation architecture, the embodiment of the present application also provides a data processing architecture based on the above regular expression generation architecture. Figure 3 , the regular expression-based data processing architecture provided in the embodiments of the present application is described in detail.

[0056] In one or more possible embodiments, Figure 3 As shown, the data processing architecture 30 proposed in the embodiment of the present application may include a receiving module 310, a matching module 320 and a processing module 330. The data processing architecture 30 is described in detail below.

[0057] The receiving module 310 is used to receive the target URL sent by the user terminal.

[0058] The matching module 320 is used to match the target URL with the regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of the content of the character segment in the historical URL set, and the content of the character segment is the content of at least two character segments pre-divided for each URL in the historical URL set.

[0059] The processing module 330 is used to converge the target URL to the target regular expression corresponding to the target URL when the matching result indicates that the target URL matches the regular expression of the URL. Alternatively, when the matching result indicates that the target URL does not match the regular expression of the URL, store the key-value pair corresponding to the target URL.

[0060] Therefore, the divergent character segments in the URL are accurately identified according to the regular expression of the URL, so that the URL is converged, the efficiency of identifying divergent URLs is improved, effective content is retained, useless information is shielded, and the subsequent analysis, storage, and display costs of the server are reduced. In addition, the above method can effectively adapt to the dynamic changes of the URL, update the regular expression of the URL in real time, and ensure the accuracy and real-time performance of the regular expression based on the URL to identify and converge the divergent URL. In the above process, no human intervention is required, which effectively reduces human resources.

[0061] It should be noted that the regular expression generation method provided in the embodiment of the present application can be applied to any scenario where divergent URLs need to be identified, such as scenarios where a large number of divergent URLs have already appeared, scenarios where divergent URLs appear infrequently, and scenarios where divergent URLs have not yet begun to diverge. For example, it can be applied to scenarios where application performance management (APM) services such as Application Real-Time Monitoring Service (ARMS) need to identify divergent URLs. In this way, the ability to respond in seconds can be quickly provided to users based on dimensions such as the front end, application, and service customization.

[0062] According to the above architecture and application scenarios, the following are combined Figure 4 The regular expression generation method provided in the embodiment of the present application is described in detail.

[0063] Figure 4 is a flowchart illustrating a regular expression generation method according to an embodiment.

[0064] like Figure 4 As shown, the regular expression generation method can be applied to the regular expression generation architecture, which may specifically include:

[0065] Step 410, obtaining a uniform resource locator URL set, each URL in the URL set includes at least two pre-divided character segments; step 420, counting the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set; step 430, generating a regular expression for the URL based on the frequency of occurrence of the content of each character segment in the URL set.

[0066] The above steps are described in detail below, as shown below.

[0067] First, step 410 is involved. In one or more possible examples, before step 410, the regular expression generation method may further include:

[0068] Filter the URLs that meet the preset format from the multiple URLs of the user obtained;

[0069] A plurality of URLs are clustered by a churn clustering algorithm to obtain a URL set, wherein the similarity between at least two URLs in the URL set is greater than or equal to a preset similarity value.

[0070] Exemplarily, a batch of URLs input in massive quantity are preprocessed, that is, direct convergence is performed on preset formats such as URLs ending with static resources, and the static resource endings include: RULES = ['.jpg', '.jpeg', '.png', '.gif', '.css', '.js', '.ico', '.woff2', '.htm', '.html', '.svg', '.pdf', '.txt', '.text', '.ppt', '.word', '.php'], so that URLs with static resource endings can be converged in massive quantities of URLs, that is, [' / index.php'] -→ {'*.php'}, [' / tmp.txt'] -→ {'*.txt'}.

[0071] Then, the above-mentioned URLs that directly converge according to the preset format are clustered. Here, the purpose of clustering is to identify similar character segments in a batch of URLs with a preset format, which can more effectively identify which character segments diverge in the batch. The clustering algorithm used in the embodiment of the present application is a streaming clustering algorithm. The advantage is that there is no need to define how many groups of classifications in advance, which is suitable for situations where the clustering scale and category are unknown in advance. The similarity measurement algorithm uses segmentation according to " / ". After segmentation, each string is treated as an independent character segment. The similarity is measured by the Jacard similarity algorithm. The Jacard similarity algorithm is used to measure the difference between two URLs. That is, the more character segments two URLs have in common, the more similar they are. If there are two character segments, namely A and B, then the Jacard similarity between the two is calculated by the following formula (1):

[0072]

[0073] Based on this, after clustering, at least two URLs whose similarity is greater than or equal to the preset similarity value are classified into a URL set, as follows:

[0074] [' / h5 / product / search / 1069240061',' / h5 / product / search / 1070648935',' / h5 / product / search / 1070930386',' / h5 / product / sea rch / 1082049256',' / h5 / product / search / 635733936',' / h5 / product / search / *',' / h5 / product / search / 73676547',' / h5 / product / s earch / 7395',' / h5 / product / search / 1066391966',…' / h5 / product / search / 1042167665',' / h5 / product / search / 993695770',' / h5 / p product / search / 1069447566',' / h5 / product / search / 80242918',' / h5 / product / search / 8548',' / h5 / product / search / 1069447545']

[0075] Therefore, the embodiment of the present application reduces the recognition scale and improves the relevance of the recognized content by clustering massive amounts of URLs. In addition, by combining the streaming clustering algorithm with the Jacard similarity algorithm, the calculated similarity measure is more accurate, and while accurately completing the pre-classification, the calculation scale of the subsequent statistics is reduced.

[0076] Next, step 420 is involved. In one or more possible examples, some character segments that may have divergent content can be pre-identified, that is, before step 420, the pattern library and vocabulary library model generated by expert experience can be used to identify the character segments that may have divergent content in at least two character segments of the URL. Then, the frequency detection model is used to count the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set, so as to accurately identify the divergent position based on the frequency of occurrence of the content of each character segment in the URL set. Based on this, the example in the above step 410 is still used for illustration, that is, the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set is counted as follows:

[0077] {1:{'h5':271},2:{'product':271},3:{'search':270,'categorySearch':1},4:{'1069240061':2,'1070648935':1,'1070930386':1,'1082049256':1,' 635733936':1,'*':1,'73676547':1,'7395':1,'1066391966':1,'106680262 7':1,'1067481333':1,'1067612906':1,'1070568070':1,'1070648924':1,'1 070648968':1,'701038665':1,'1070648870':1,…'348242718':1,'1042167665':1,'993695770':1,'1069447566':1,'80242918':1,'8548':1,'1069447545':1}}, it can be seen that after the URL is split according to " / " to obtain at least two character segments, the changes in the character segments at different positions of multiple URLs in the URL set can be viewed. It can be found that the character segments at the first three positions: h5, product, search are not divergent, while the last character segment is a number that is constantly changing, so it can be identified that it is divergent.

[0078] Therefore, by combining the pattern library, vocabulary library model and frequency detection model to identify divergent URLs, it is possible to accurately identify which URL character segments are diverging in a batch of URLs.

[0079] Then, step 430 is involved. In one or more possible examples, step 430 may specifically include:

[0080] Step 4301, obtaining a first character segment of at least two character segments according to the frequency of occurrence of the content of each character segment in the URL set, wherein the frequency of occurrence of the content of the first character segment in the URL set is less than or equal to a preset frequency;

[0081] Step 4302, replacing the first character segment with a preset identifier;

[0082] Step 4303, based on the preset identifier and the second character segment of the at least two character segments, generate a regular expression of the URL; wherein the frequency of occurrence of the content of the second character segment in the URL set is greater than the preset frequency.

[0083] Exemplarily, when it is determined that the character segment in the URL is meaningless and divergent, the URL in the URL set can be rewritten, that is, a process of first obtaining an incoming URL request and then rewriting it into another URL that the website can handle. For example, if the URL coming in through the browser is "ARMS.UserProfile.php?ID=1", then it can be rewritten as "ARMS.UserProfile / 1.php". Such a URL can be better read by the website. For another example, in the field of multidimensional data storage, the URL "ARMS.UserProfile.php?ID=1" is directly stored as a key, which will result in too many storage keys and a huge waste of resources. At this time, it is often processed as "ARMS.UserProfile.aspx?ID=*", so that similar URLs share the same key, greatly reducing storage costs.

[0084] Based on this principle, after determining which character segments need to converge, the embodiment of the present application needs to generate corresponding regular rules. Here, the embodiment of the present application is divided into three categories of regular rules. Among them, if the character segments that need to converge are all numbers, the preset identifier associated with the character category of the content of the first character segment is NUM; if the character segments that need to converge are all letters, the preset identifier associated with the character category of the content of the first character segment is NUM; if the character segments that need to converge are all numbers, the preset identifier associated with the character category of the content of the first character segment is ALPHA; similarly, if the character segments that need to converge are other, such as both letters and numbers, the preset identifier of other characters associated with the character category of the content of the first character segment is CODE, so that the character segment that needs to converge is replaced with the above-mentioned preset identifier, and then, based on the preset identifier and the second character segment of the at least two character segments, a regular expression for the URL is generated.

[0085] In addition, after step 4301, it is possible to determine whether the first character segment is a custom character segment in a custom URL through the pattern library and the vocabulary library model to determine whether the character segment in the URL is meaningful. In this way, the regular expression generation method may further include:

[0086] Determine whether the content of the first character segment includes preset content through the pattern library and vocabulary library model;

[0087] When it is determined that the content of the first character segment includes preset content, a regular expression of the URL is generated based on the first character segment.

[0088] Exemplarily, through the pattern library and vocabulary library model, it is determined whether the content of the character segment includes preset content. When it is determined that the content of the character segment includes preset content, that is, the character segment is characterized as a customized URL of a customized service or a character segment in a URL, there is no need to replace it with a preset identifier, but it is retained to generate a regular expression for the URL.

[0089] In another or more possible examples, before step 4302, the regular expression generation method may further include:

[0090] Based on the preset association information between the character category and the identifier, a preset identifier associated with the character category of the content of the first character segment is obtained; wherein,

[0091] The character category of the content includes a numeric character category, an alphabetic character category or other character categories, and the other character category is a combination category of at least two of the following characters: data, letters, and symbols.

[0092] In yet another one or more possible examples, before step 4302, the regular expression generation method may further include:

[0093] Step 4401, identifying letters in the first character segment by a natural language processing algorithm to obtain a plurality of target words composed of the letters in the first character segment;

[0094] Step 4402, based on each target word in the plurality of target words and the first character segment, calculating the validity degree value of the first character segment;

[0095] Step 4403: When the validity level value meets a preset threshold, the first character segment is replaced with a first preset identifier.

[0096] Furthermore, the above step 4402 may specifically include:

[0097] Step 44021, based on the position coordinates of each target word in the first character segment, calculate the total length of each target word in the first character segment;

[0098] Step 44022, taking the ratio of the total length of each target word in the first character segment to the total character length of the first character segment as the effectiveness value of the first character segment.

[0099] Specifically, the above-mentioned step 44021 may include:

[0100] Based on the position coordinates of each target word in the first character segment, duplicate words in multiple target words are filtered out by a union algorithm;

[0101] Calculate the effective length of the valid words in the first character segment based on the position coordinates of the valid words excluding the repeated words in the plurality of target words in the first character segment;

[0102] The effective length is determined as the total length of each target word in the first character segment.

[0103] For example, NLP algorithms are used to accurately identify whether the content of a location is meaningful (considering only the case where the string is letters). In the process of identifying the validity of a word, other NLP algorithms such as Jieba for identifying Chinese pinyin can also be used. Here, unlike many solutions, the embodiment of the present application uses an NLP algorithm to identify the meaningfulness of a word. Sometimes, although some character segments are divergent, they are interfaces composed of meaningful words. These are often important viewing information, and users often do not want to give convergence to such as

[0104] [' / ssrService / allotwant / clearAllotProductTemp',' / ssrService / allotwant / queryInputProductMaxSize',' / ssrService / allotwant / deleteAllotProductSumTemp',' / ssrService / allotwant / updateEditAllotwantList',…

[0105] ' / ssrService / allotwant / queryCurMonthStoreSettlePictureMap',' / ssrService / allotwant / queryAllotProductSumSizeMap']

[0106] In addition, there are some character segments that do not diverge in frequency, but contain some content that may diverge, such as:

[0107] [' / saasService / batchOutOrder / batchOutOrderView / 707970342815973376 / 418914857213953']

[0108] This phenomenon is a real problem in some low-frequency URLs. If convergence is not performed at this time, then after a period of accumulation, these divergent contents will become extremely large. Specifically, the NLP algorithm can use the enchant natural language processing package to identify words, such as: "queryAllotProductSumSizeMap", which can identify the valid words as:

[0109] {'query':range(0,5),'All':range(5,8),'Allot':range(5,10),'lot':range(7,10),'Pro':range(10,13),'Prod':range(10,14),'Product':range(10,17),'rod':range(11,14),'duct':range(13,17),'Sum':range(17,20),'Size':range(20,24),'Map':range(24,27)}, Since the validity of the recognized words has repeated positions, it is necessary to combine the n interval union algorithm to calculate the total length of each target word in the first character segment, total_meaningful_word_length. Then, the validity of the entire character segment can be determined by setting the ratio threshold, that is, the validity value of the first character segment is calculated by formula (2):

[0110] total_meaningful_word_length / word_length>=threshold (2)

[0111] Among them, word_length is the total character length of the first character segment. In this way, when the ratio of the total length of each target word in the first character segment to the total character length of the first character segment meets the preset threshold, it is determined to be invalid, and the step of replacing the first character segment with the first preset identifier is executed; otherwise, it is determined to be valid, retained, and stored.

[0112] Therefore, we cleverly utilize the combination of the first judgment model and the second judgment model to accurately identify the divergent character segments in the URL, utilize NLP to identify the validity of words, better measure the validity of information from the user's perspective, and effectively resolve the effectively divergent URLs and handle the service vulnerabilities of a single batch that is not divergent in large quantities but has meaningless locations.

[0113] In addition, in one or more possible examples, after step 430, the regular expression generation method may further include:

[0114] Based on the historical URL, the regular expression is verified to obtain the verified regular expression;

[0115] The verified regular expression is determined as the regular expression of the URL.

[0116] For example, after the regular expression is generated, it is necessary to re-traverse all URLs to detect whether the current regular expression can cover the content that needs to be converged and whether it will cover more content that should not be converged, so as to perform a second verification on the regular expression.

[0117] In one or more possible examples, after step 430, the regular expression generation method may further include:

[0118] Get the first URL currently received;

[0119] Match the first URL with the regular expression of the URL to obtain a matching result;

[0120] If the matching result indicates that the first URL does not match the regular expression of the URL, storing the key-value pair corresponding to the first URL;

[0121] When the matching result indicates that the first URL matches the regular expression of the URL, the first URL is converged to a first regular expression corresponding to the first URL, and a key-value pair corresponding to the first regular expression is stored.

[0122] In summary, the regular expression generation method provided in the embodiment of the present application can dynamically adapt the overall solution to URL changes, has strong versatility, can rely less on manual experience, and can even work without relying on manual presets, and has good regular convergence effect. While retaining effective information, it greatly reduces storage costs. In addition, after similarity convergence through clustering, through the combination of pattern library and vocabulary library model and frequency detection model, it can accurately identify which URL character segments in a batch of URLs are diverging. Combined with the NLP algorithm, it can accurately identify which content does not have effective meaning in the possible divergent position or even in the non-divergent position. The overall model recognizes whether it is divergent and meaningful. The effect is very good. While retaining effective information, it is very likely to identify the divergent position and converge it, which greatly reduces the storage cost.

[0123] Based on the same inventive concept, the embodiment of the present application provides a data processing method based on a regular expression based on the regular expression generation method involved above, specifically in combination with Figure 6 Provide detailed explanation.

[0124] According to the above architecture and application scenarios, the following are combined Figure 6 The data processing method based on regular expressions provided in the embodiments of the present application is described in detail.

[0125] Figure 6 is a flowchart illustrating a data processing method based on regular expressions according to one embodiment.

[0126] like Figure 6 As shown, the data processing method based on regular expressions can be applied to Figure 3 The data processing architecture shown may specifically include:

[0127] Step 610, receiving a target URL sent by a user terminal; Step 620, matching the target URL with a regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of a character segment content in a historical URL set, and the character segment content is the content of at least two character segments pre-divided for each URL in the historical URL set; Step 630, when the matching result indicates that the target URL matches the regular expression of the URL, converging the target URL to a target regular expression corresponding to the target URL.

[0128] The above steps are described in detail below, as shown below.

[0129] After step 620, the data processing method based on regular expressions may further include:

[0130] When the matching result indicates that the target URL does not match the regular expression of the URL, a key-value pair corresponding to the target URL is stored.

[0131] It should be clear that the present application is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the convenience and brevity of description, a detailed description of the known methods is omitted here, and the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, which will not be repeated here.

[0132] Based on the same inventive concept, the present application provides a regular expression generation device corresponding to the regular expression generation method involved above, Figure 6 Provide detailed explanation.

[0133] Figure 6 The figure is a schematic diagram showing the structure of a regular expression generating device according to an embodiment.

[0134] like Figure 6 As shown, the regular expression generating device 60 is applied to Figure 1 The regular expression generation architecture shown, the regular expression generation device 60 may specifically include:

[0135] The acquisition module 601 is used to acquire a set of uniform resource locators (URLs), each URL in the set of URLs includes at least two pre-divided character segments;

[0136] A statistics module 602, used to count the frequency of occurrence of the content of each character segment in at least two character segments in the URL set;

[0137] The generating module 603 is used to generate a regular expression of the URL according to the frequency of occurrence of the content of each character segment in the URL set.

[0138] Based on this, the regular expression generating device 60 provided in the embodiment of the present application is described in detail below.

[0139] In one or more possible embodiments, the regular expression generating device 60 provided in the embodiment of the present application may further include a first replacement module; wherein,

[0140] The acquisition module 601 in the embodiment of the present application may also be used to acquire a first character segment of at least two character segments according to the frequency of occurrence of the content of each character segment in the URL set, where the frequency of occurrence of the content of the first character segment in the URL set is less than or equal to a preset frequency;

[0141] A first replacement module, used to replace the first character segment with a preset identifier;

[0142] The generation module 603 may also be used to generate a regular expression of the URL based on a preset identifier and a second character segment of at least two character segments; wherein the frequency of occurrence of the content of the second character segment in the URL set is greater than a preset frequency.

[0143] In another or more possible embodiments, the acquisition module 601 in the embodiment of the present application may also be used to acquire a preset identifier associated with the character category of the content of the first character segment based on preset association information between the character category and the identifier; wherein,

[0144] The character category of the content includes a numeric character category, an alphabetic character category or other character categories, and the other character category is a combination category of at least two of the following characters: data, letters, and symbols.

[0145] In one or more possible embodiments, the regular expression generating device 60 provided in the embodiment of the present application may further include a recognition module, a calculation module and a second replacement module; wherein,

[0146] A recognition module, used for recognizing letters in the first character segment by using a natural language processing algorithm to obtain a plurality of target words composed of the letters in the first character segment;

[0147] A calculation module, used for calculating the validity value of the first character segment based on each target word in the plurality of target words and the first character segment;

[0148] The second replacement module is used to replace the first character segment with a first preset identifier when the validity level value meets a preset threshold.

[0149] In one or more possible embodiments, the calculation module may be specifically configured to calculate the total length of each target word in the first character segment based on the position coordinates of each target word in the first character segment;

[0150] The ratio of the total length of each target word in the first character segment to the total character length of the first character segment is used as the effectiveness value of the first character segment.

[0151] In one or more possible embodiments, the calculation module may be specifically configured to filter out duplicate words from the plurality of target words by a union algorithm based on the position coordinates of each target word in the first character segment;

[0152] Calculate the effective length of the valid words in the first character segment based on the position coordinates of the valid words excluding the repeated words in the plurality of target words in the first character segment;

[0153] The effective length is determined as the total length of each target word in the first character segment.

[0154] In one or more possible embodiments, the regular expression generating device 60 provided in the embodiment of the present application may further include a first determining module, which is used to determine whether the content of the first character segment includes preset content through the pattern library and the vocabulary library model;

[0155] The generation module 603 may also be configured to generate a regular expression of the URL based on the first character segment when it is determined that the content of the first character segment includes preset content.

[0156] In one or more possible embodiments, the regular expression generating device 60 provided in the embodiment of the present application may further include a verification module and a second determination module; wherein,

[0157] The verification module is used to verify the regular expression based on the historical URL to obtain the verified regular expression;

[0158] The second determination module is used to determine the verified regular expression as the regular expression of the URL.

[0159] In one or more possible embodiments, the regular expression generating device 60 provided in the embodiment of the present application may further include a matching module, a storage module and a processing module; wherein,

[0160] The acquisition module 601 in the embodiment of the present application can also be used to acquire the first URL currently received;

[0161] A matching module, used for matching the first URL with a regular expression of the URL to obtain a matching result;

[0162] A storage module, configured to store a key-value pair corresponding to the first URL when the matching result indicates that the first URL does not match the regular expression of the URL;

[0163] The processing module is used to converge the first URL to a first regular expression corresponding to the first URL and store a key-value pair corresponding to the first regular expression when the matching result indicates that the first URL matches the regular expression of the URL.

[0164] In one or more possible embodiments, the regular expression generating device 60 provided in the application embodiment may further include a screening module and a clustering module; wherein,

[0165] A screening module is used to screen URLs that meet a preset format from among the multiple user URLs obtained;

[0166] The clustering module is used to cluster multiple URLs through a churn clustering algorithm to obtain a URL set, wherein the similarity between at least two URLs in the URL set is greater than or equal to a preset similarity value.

[0167] Based on the same inventive concept, the present application provides a regular expression generation device corresponding to the regular expression generation method involved above, Figure 7 Provide detailed explanation.

[0168] Figure 7 It is a schematic diagram showing the structure of a data processing device based on regular expressions according to an embodiment.

[0169] like Figure 7 As shown, the data processing device 70 based on regular expressions is applied to Figure 3 The data processing architecture shown, the data processing device 70 based on regular expressions may specifically include:

[0170] Receiving module 701, used to receive the target URL sent by the user terminal;

[0171] A matching module 702 is used to match the target URL with a regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of the content of the character segment in the historical URL set, and the content of the character segment is the content of at least two character segments pre-divided by each URL in the historical URL set;

[0172] The convergence module 703 is used to converge the target URL to the target regular expression corresponding to the target URL when the matching result indicates that the target URL matches the regular expression of the URL.

[0173] Based on this, the regular expression-based data processing device 70 provided in an embodiment of the present application is described in detail below.

[0174] In one or more possible embodiments, the regular expression-based data processing device 70 provided in the application embodiment may also include a storage module for storing a key-value pair corresponding to the target URL when the matching result indicates that the target URL does not match the regular expression of the URL.

[0175] Thus, by obtaining at least two character segments pre-divided from each URL in the uniform resource locator URL set, counting the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set, and generating a regular expression of the URL according to the frequency of occurrence of the content of each character segment in the URL set, the divergent character segments in the URL can be accurately identified according to the regular expression of the URL, thereby converging its URL, improving the efficiency of identifying divergent URLs, retaining valid content, shielding useless information, and reducing the subsequent analysis, storage, and display costs of the server. In addition, the above method can effectively adapt to the dynamic changes of URLs, update the regular expression of URLs in real time, and ensure the accuracy and real-time performance of recognizing and converging divergent URLs based on the regular expression of URLs. No human intervention is required in the above process, effectively reducing human resources.

[0176] Figure 8 FIG. 4 is a schematic diagram showing the hardware structure of a computer device according to an embodiment.

[0177] like Figure 8 As shown, the computer device 800 includes an input device 801 , an input interface 802 , a processor 803 , a memory 804 , an output interface 805 , and an output device 806 .

[0178] The input interface 802, the processor 803, the memory 804, and the output interface 805 are interconnected via the bus 810, and the input device 801 and the output device 806 are connected to the bus 810 via the input interface 802 and the output interface 805, respectively, and are further connected to other components of the computer device 800. Specifically, the input device 801 receives input information from the outside, and transmits the input information to the processor 803 via the input interface 802; the processor 803 processes the input information based on the computer executable instructions stored in the memory 804 to generate output information, temporarily or permanently stores the output information in the memory 804, and then transmits the output information to the output device 806 via the output interface 805; the output device 806 outputs the output information to the outside of the computer device 800 for use by the user.

[0179] In one embodiment, Figure 8 The computer device 800 shown can be implemented as a regular expression generation device, which may include: a memory configured to store a program; a processor configured to run the program stored in the memory to execute the regular expression generation method described in the above embodiment. Or, Figure 8 The computer device 800 shown can be implemented as a regular expression-based data processing device, which may include: a memory configured to store programs; a processor configured to run the programs stored in the memory to execute the regular expression-based data processing method described in the above embodiment.

[0180] In one embodiment, the memory can also be used to store a URL set, each URL in the URL set, at least two character segments pre-divided in each URL, a regular expression of the URL, and a storage medium for storing the above Figures 1 to 5 The calculation results of each step in the regular expression generation process described, as well as the calculation results of each step in the data processing process based on regular expressions.

[0181] According to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer-readable storage medium. For example, an embodiment of the present application includes a computer-readable storage medium, which includes storing a program or instruction on the computer-readable storage medium, and when the program or instruction is executed by a computer device, the computer device performs the steps of the above method.

[0182] According to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable storage medium.

[0183] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions, and when it is run on a computer, the computer executes the method described in each of the above embodiments. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state drive), etc.

[0184] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A regular expression generation method, include: Obtain a uniform resource locator URL set, each URL in the URL set includes at least two pre-divided character segments; Counting the frequency of occurrence of the content of each character segment in the at least two character segments in the URL set; According to the frequency of occurrence of the content of each character segment in the URL set, obtaining a first character segment of the at least two character segments, wherein the frequency of occurrence of the content of the first character segment in the URL set is less than or equal to a preset frequency; In the case where the validity level value of the first character segment meets a preset threshold, the first character segment is replaced with a preset identifier; the validity level value is determined based on a plurality of target words composed of letters in the first character segment and the first character segment; A regular expression of the URL is generated based on the preset identifier and a second character segment of the at least two character segments; wherein the frequency of occurrence of the content of the second character segment in the URL set is greater than the preset frequency.

2. The method according to claim 1, in, Before replacing the first character segment with a preset identifier, the method further includes: Based on the preset association information between character categories and identifiers, a preset identifier associated with the character category of the content of the first character segment is obtained; wherein, The character category of the content includes a numeric character category, an alphabetic character category or other character categories, and the other character category is a combination category of at least two of the following characters: data, letters, and symbols.

3. The method according to claim 1 or 2, in, Before replacing the first character segment with a first preset identifier, the method further includes: Recognize letters in the first character segment by a natural language processing algorithm to obtain a plurality of target words composed of letters in the first character segment; Based on each target word in the plurality of target words and the first character segment, a validity degree value of the first character segment is calculated.

4. The method according to claim 3, in, The calculating the validity value of the first character segment based on each target word in the plurality of target words and the first character segment includes: Calculate the total length of each target word in the first character segment based on the position coordinates of each target word in the first character segment; The ratio of the total length of each target word in the first character segment to the total character length of the first character segment is used as the effectiveness value of the first character segment.

5. The method according to claim 4, in, The calculating the total length of each target word in the first character segment based on the position coordinates of each target word in the first character segment includes: Based on the position coordinates of each target word in the first character segment, filtering out duplicate words from the multiple target words by a union algorithm; Calculating the effective length of the effective word in the first character segment based on the position coordinates of the effective word in the first character segment except the repeated word among the multiple target words; The effective length is determined as the total length of each target word in the first character segment.

6. The method according to claim 1, in, After obtaining the first character segment of the at least two character segments, the method further includes: Determining whether the content of the first character segment includes preset content through the pattern library and the vocabulary library model; When it is determined that the content of the first character segment includes preset content, a regular expression of the URL is generated based on the first character segment.

7. The method according to claim 1, in, The method further comprises: Based on the historical URL, the regular expression is verified to obtain a verified regular expression; The verified regular expression is determined as the regular expression of the URL.

8. The method according to claim 1, in, The method further comprises: Get the first URL currently received; Matching the first URL with the regular expression of the URL to obtain a matching result; If the matching result indicates that the first URL does not match the regular expression of the URL, storing a key-value pair corresponding to the first URL; In a case where the matching result indicates that the first URL matches the regular expression of the URL, the first URL is converged to a first regular expression corresponding to the first URL, and a key-value pair corresponding to the first regular expression is stored.

9. The method according to claim 1, in, Before obtaining the uniform resource locator URL set, the method further includes: Filter the URLs that meet the preset format from the multiple URLs of the user obtained; The multiple URLs are clustered by a churn clustering algorithm to obtain the URL set, and the similarity between at least two URLs in the URL set is greater than or equal to a preset similarity value.

10. A data processing method based on regular expressions, in, The regular expression is generated based on the regular expression generation method according to any one of claims 1 to 9, the method comprising: Receive the target URL sent by the user; Matching the target URL with a regular expression of the URL to obtain a matching result, wherein the regular expression of the URL is determined by the frequency of occurrence of the content of the character segment in the historical URL set, and the content of the character segment is the content of at least two character segments pre-divided for each URL in the historical URL set; In a case where the matching result indicates that the target URL matches the regular expression of the URL, the target URL is converged to the target regular expression corresponding to the target URL.

11. The method according to claim 10, in, The method further comprises: In a case where the matching result indicates that the target URL does not match the regular expression of the URL, a key-value pair corresponding to the target URL is stored.

12. A computer device, include: Memory and processor, The memory is used to store a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor executes the steps of the regular expression generation method described in any one of claims 1 to 9, or executes the steps of the regular expression-based data processing method described in any one of claims 10 to 11.

13. A computer-readable storage medium storing a program or instruction, wherein when the program or instruction is executed by a computer device, the computer device executes the steps of the regular expression generation method as described in any one of claims 1 to 9, or executes the steps of the regular expression-based data processing method as described in any one of claims 10 to 11.