A method and apparatus for URL normalization

Abstract

This disclosure provides a method and apparatus for URL normalization, relating to the field of computer technology, and particularly to the field of big data processing technology. The specific implementation scheme is as follows: Obtain the target URL; determine whether the target URL matches the path configuration information in the preset normalization rules, the normalization rules including: path configuration information and parameter processing rules; if yes, perform preset normalization processing on the path field in the target URL, and process the parameter field of the target URL according to the parameter processing rules; if no, perform default normalization processing on the target URL. It is evident that processing is performed in the order of user configuration priority and default normalization as a fallback, avoiding excessive URL expansion, reasonably aggregating data, and reducing the overhead of log storage and computation. It can solve the problem that parameter processing results in URL normalization schemes based on regular expressions or distance in vector space do not meet actual needs. It eliminates the need to sort and classify a large number of URLs in the business, significantly reducing workload.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the field of big data processing technology. Background Technology

[0002] URL stands for Uniform Resource Locator, which is an identifier used on the Internet to describe the address of web pages and other resources. Summary of the Invention

[0003] This disclosure provides a method and apparatus for URL normalization.

[0004] According to one aspect of this disclosure, a method for URL normalization is provided, comprising:

[0005] Get the target URL;

[0006] Determine whether the target URL matches the path configuration information in the preset normalization rules, wherein the normalization rules include: path configuration information and parameter processing rules;

[0007] If so, the path field in the target URL is subjected to preset normalization processing, and the parameter field of the target URL is processed according to the parameter processing rules;

[0008] If not, the target URL will be normalized by default.

[0009] According to one aspect of this disclosure, an apparatus for URL normalization is provided, comprising:

[0010] The retrieval module is used to retrieve the target URL;

[0011] The judgment module is used to determine whether the target URL matches the path configuration information in the preset normalization rules, wherein the normalization rules include: path configuration information and parameter processing rules;

[0012] The first processing module is configured to perform preset normalization processing on the path field in the target URL and process the parameter field of the target URL according to the parameter processing rules if the judgment result of the judgment module is yes.

[0013] The second processing module is used to perform default normalization processing on the target URL if the judgment result of the judgment module is negative.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a URL normalization method.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform a method of URL normalization.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method for URL normalization.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 A flowchart illustrating a URL normalization method provided in an embodiment of this disclosure;

[0023] Figure 2 A schematic diagram of an interface for a normalization rule configuration platform provided in an embodiment of this disclosure;

[0024] Figure 3 A schematic diagram of a URL normalization method provided in an embodiment of this disclosure;

[0025] Figure 4 This is a block diagram of an apparatus for implementing the URL normalization method according to embodiments of the present disclosure;

[0026] Figure 5 This is a block diagram of an electronic device used to implement the URL normalization method of embodiments of the present disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] URL stands for Uniform Resource Locator, which is an identifier that describes the address of a webpage or other resource on the Internet.

[0029] Each webpage has a unique name identifier called the page URL, the API (Application Programming Interface) call address is called the API URL, and there are also various resource URLs.

[0030] The general syntax format of a URL is:

[0031] The protocol is defined as follows: protocol: / / hostname:port / path / ?parameters#Anchor. This includes fields such as protocol, hostname, port number, path, and parameters.

[0032] When performing webpage log analysis, it is often necessary to perform statistical calculations on URLs, such as counting daily visits, visit distribution, and stability indicators at the page level.

[0033] During the statistical calculation of URLs, URL normalization is necessary. This is because: 1) Changes in URL parameters only result in a partial page refresh or data modification; essentially, the URL before and after the parameter change corresponds to the same page. 2) Failure to normalize URLs will lead to a massive log volume, making it impossible to aggregate data effectively and wasting storage and computing resources on the log platform.

[0034] Therefore, page URLs must be normalized when performing statistical analysis at the page level. Similarly, API URLs and various resource URLs also face the same issue and require normalization.

[0035] Currently, there are two main methods for URL normalization:

[0036] 1) User-configured approach.

[0037] With the development of web technologies and the emergence of front-end frameworks, the widespread use of SPAs (single-page applications) has led to the possibility of parameters appearing in various locations within URLs, such as path fields. Based on user configuration, path fields and parameter fields need to be handled separately.

[0038] First, front-end developers organize the large number of existing URLs in the business logic and create a mapping table based on the website's soft routing logic. When analyzing logs, they use this mapping table to normalize the path fields of the URLs in the logs. Alternatively, they can use regular expressions to normalize multiple page paths. For example, the regular expression www\.aaa\.com / mp[1-4] can be used to group www.aaa.com / mp1, www.aaa.com / mp2, www.aaa.com / mp3, and www.aaa.com / mp4 together.

[0039] Subsequently, for each group of URLs that have undergone path normalization, the parameter fields are processed again, setting which parameters should be excluded or retained, and then normalizing the parameter fields. For example, for the following URLs:

[0040] In the URL `http: / / www.example.com:80 / path / to / myfile.html?key1=values&key2=values#SomewhereInTheDocument`, if the parameter to be excluded is set to `key1`, then after normalizing the parameter fields, the following result is obtained:

[0041] The URL `http: / / www.example.com:80 / path / to / myfile.html?key1=&key2=values#SomewhereInTheDocument` removes the specific value of parameter `key1`.

[0042] The drawback of this approach is that it requires front-end developers to sift through and categorize a large number of existing page URLs in the business logic, resulting in a significant workload. Furthermore, as the business rapidly evolves, URL variables may include path fields and parameter fields, necessitating separate and phased normalization processing for these fields.

[0043] 2) Processing is unified based on extraction rules.

[0044] In this approach, regular expressions are designed using general rules, and the URL is directly transformed based on the regular expressions to obtain the normalized result.

[0045] However, if the expression is set too simply, it often leads to overprocessing and fails to retain the important parameter information that users want to focus on. If the expression is designed to be comprehensive and complex, it may still fail to cover all business scenarios and will also increase performance overhead.

[0046] For example, API URLs often contain version markers such as V1, V2, etc., so for the following two URLs:

[0047] http: / / www.example.com:80 / api / v1 / users / me

[0048] http: / / www.example.com:80 / api / v2 / users / me

[0049] The goal is to separate the statistics while retaining version tags. However, due to the very similar URL characteristics, they are easily uniformized by default, meaning version tags can easily be deleted.

[0050] Some solutions suggest encoding the original URLs into numerical vectors using methods such as deep learning, so that URLs with the same path but different parameters are very close in the encoded vector space. Then, URLs with similar numerical vectors are merged to achieve normalization.

[0051] However, normalization based on distance in vector space may not necessarily meet practical needs.

[0052] For example, for the following three page URLs:

[0053] http: / / www.example.com:80 / search / electronics

[0054] http: / / www.example.com:80 / search / computers

[0055] http: / / www.example.com:80 / search / luggage

[0056] The last segment is business variables. We want to normalize these business variables, but in URL normalization solutions based on regular expressions or distance in vector space, business variables are easily retained, which cannot meet the actual needs.

[0057] To address the aforementioned technical problems, this disclosure provides a method and apparatus for URL normalization.

[0058] In one embodiment of this disclosure, a method for URL normalization is provided, the method comprising:

[0059] Get the target URL;

[0060] Determine whether the target URL matches the path configuration information in the preset normalization rules, wherein the normalization rules include: path configuration information and parameter processing rules;

[0061] If so, the path field in the target URL is subjected to preset normalization processing, and the parameter field of the target URL is processed according to the parameter processing rules;

[0062] If not, the target URL will be normalized by default.

[0063] As can be seen, in this embodiment, normalization rules are pre-defined according to business needs, including path configuration information and parameter processing rules. If the target URL matches the path configuration information, the path field and parameter field are processed synchronously, eliminating the need for two separate stages. Processing the parameter field according to the customized parameter processing rules solves the problem that parameter processing results in URL normalization schemes based on regular expressions or distance in vector space do not meet actual needs. Furthermore, the method of matching configuration information and processing parameter fields according to rules is simpler and more convenient than the normalization method based on regular expressions.

[0064] If the target URL does not match the path configuration information, it is normalized by default. This process prioritizes user configuration and uses default normalization as a fallback, preventing excessive URL expansion, rationally aggregating data, and reducing the overhead of log storage and computation.

[0065] In addition, users (URL analysts, etc.) only need to configure normalization rules on the platform, without having to sort out and classify a large number of URLs in the business, which significantly reduces the workload.

[0066] The URL normalization method and apparatus provided in the embodiments of this disclosure will be described in detail below.

[0067] See Figure 1 , Figure 1 The URL normalization method provided in the embodiments of this disclosure, such as Figure 1 As shown, the method may include the following steps:

[0068] S101: Get the target URL.

[0069] The target URL is a URL that needs to be normalized. For example, a large number of page URLs collected from the front end can all be used as target URLs.

[0070] S102: Determine whether the target URL matches the path configuration information in the preset normalization rules. The normalization rules include: path configuration information and parameter processing rules. If yes, proceed to S103; if no, proceed to S104.

[0071] In this embodiment of the disclosure, normalization rules can be set in advance according to requirements.

[0072] The normalization rules include: path configuration information and parameter processing rules.

[0073] After obtaining the target URL, the path field of the target URL can be matched based on the route matching pattern familiar to front-end developers. Specifically, path-to-reqexp can be used as the route matching engine.

[0074] Among them, path-to-reqexp is a route matching engine well known to those skilled in the art, capable of matching path fields.

[0075] If the path field in the target URL is the same as the preset path configuration information, except for numbers and / or Chinese characters, then the target URL matches the path configuration information.

[0076] S103: Perform preset normalization processing on the path field in the target URL, and process the parameter field of the target URL according to the parameter processing rules.

[0077] In this embodiment of the disclosure, if the target URL matches the path configuration information, normalization processing is performed according to the preset normalization rules.

[0078] Specifically, you can directly perform normalization processing on numbers and / or Chinese characters. That is, for the path field in the target URL, directly delete the numbers and / or Chinese characters in the path field; or map the numbers and / or Chinese characters in the path field in the target URL to preset symbols.

[0079] For example, change all numbers and / or Chinese characters in the path field of the target URL to the uniform character "*".

[0080] For parameter fields in the target URL, process them according to the parameter processing rules.

[0081] In one embodiment of this disclosure, the parameter processing rule may be: retaining a preset first type of custom parameter, and / or deleting a preset second type of custom parameter.

[0082] As an example, see Figure 2 , Figure 2 A schematic diagram of an interface for a normalization rule configuration platform provided in this embodiment of the disclosure, such as... Figure 2 As shown, the configured path information is: / user / :id / overview, and the parameter processing rule is: retain parameters key1 and key2.

[0083] For the following three URLs:

[0084] / user / 123 / overview? key1=value1&key2=value2&key3=value3;

[0085] / user / 456 / overview? key1=value1&key2=value2&key3=value3&key4=value4;

[0086] / user / 789 / overview? key2=value2&key1=value1;

[0087] All URLs matched the path configuration information, so the path field was pre-normalized to / user / * / overview. For the parameter fields, key1 and key2 were retained, while other parameters were removed. Therefore, all three URLs were normalized to:

[0088] / user / * / overview? key1=value1&key2=value2.

[0089] For the scenario mentioned above, where API URLs often contain version markers such as V1 and V2, and it is desirable to retain these version markers, the URL normalization method provided in this disclosure can be used. The parameter processing rule is configured to retain parameters V1 and V2. It is evident that this customized approach can meet the actual requirements.

[0090] Furthermore, it is simpler and more convenient than the normalization method based on regular expressions. For example, the regular expression configuration for matching / user / :id is: / ^\ / user\ / ((?:[^\ / ]+?))(?:\ / (?=$))? $ / i, which has higher complexity.

[0091] S104: Perform default normalization on the target URL.

[0092] In this embodiment of the disclosure, if the target URL does not match the path configuration information, it means that the target URL has not hit the custom normalization rule, and the default normalization process can be applied to it.

[0093] In one embodiment of this disclosure, default normalization processing of the target URL is performed, including:

[0094] Perform pre-defined normalization on the path field in the target URL and remove the parameter field from the target URL.

[0095] Specifically, remove the parameters field from the target URL and obfuscate numbers and Chinese characters.

[0096] As an example, for the following URL:

[0097] http: / / www.example.com:80 / path / 123456 / myfile.html?key1=values&key2=values#SomewhereInTheDocument

[0098] After performing default normalization, the path field "123456" is blurred to the specific symbol "*", and all parameters in the parameter field are deleted. The result is:

[0099] http: / / www.example.com:80 / path / * / myfile.html#SomewhereInTheDocument.

[0100] As can be seen, in this embodiment, normalization rules are pre-defined according to business needs, including path configuration information and parameter processing rules. If the target URL matches the path configuration information, the path field and parameter field are processed synchronously, eliminating the need for two separate stages. Processing the parameter field according to the customized parameter processing rules solves the problem that parameter processing results in URL normalization schemes based on regular expressions or distance in vector space do not meet actual needs. Furthermore, the method of matching configuration information and processing parameter fields according to rules is simpler and more convenient than the normalization method based on regular expressions.

[0101] If the target URL does not match the path configuration information, it is normalized by default. This process prioritizes user configuration and uses default normalization as a fallback, preventing excessive URL expansion, rationally aggregating data, and reducing the overhead of log storage and computation.

[0102] In addition, users (URL analysts, etc.) only need to configure normalization rules on the platform, without having to sort out and classify a large number of URLs in the business, which significantly reduces the workload.

[0103] In one embodiment of this disclosure, in addition to platform interface configuration, function configuration can also be performed on the JSSDK (JavaScript Software Development Kit) frontend.

[0104] Specifically, configure normalization functions in the software development kit on the front end of the page. Normalization functions are used to normalize URLs that do not follow the syntax format.

[0105] Before reporting the initial page URL, the front-end calls a normalization function to normalize the initial page URL before sending it to the back-end.

[0106] Therefore, in this embodiment of the disclosure, the target URL can be obtained by processing the initial page URL through a normalization function at the front end of the page.

[0107] As an example, for the following URL:

[0108] http: / / www.example.com:80 / main.html# / SomewhereInTheDocument~key1=values&key2=values.

[0109] The parameter field starts with "~", not the standard "?". After the standardized function is called on the front end of the page for processing, a standardized URL is obtained, which is then reported to the back end as the target URL.

[0110] The standardized URL is:

[0111] http: / / www.example.com:80 / main.html# / SomewhereInTheDocument?key1=values&key2=values.

[0112] As can be seen, in this embodiment of the disclosure, for scenarios where the front-end page contains a large number of URLs that do not follow the syntax format, a standardization function can be pre-configured in the software development kit running on the front-end. During the process of running the page using JSSDK on the front-end, the initial page URL is collected, the standardization function is called to standardize the initial page URL, and then it is reported to the back-end, thus realizing the rapid standardization of URLs that do not follow the syntax format.

[0113] See Figure 3 , Figure 3 This is a schematic diagram of a URL normalization method provided in an embodiment of the present disclosure.

[0114] like Figure 3 As shown, in the first case: if the URL matches the front-end JSSDK configuration and the platform's normalization rules, then standardization is performed on the front-end, and normalization is performed on the platform. Here, "platform" refers to the back-end platform used for log processing or URL analysis.

[0115] In the second case: if the URL matches the front-end JSSDK configuration but does not match the platform's normalization rules, then the processing is only performed on the front-end.

[0116] In the third case: if no JSSDK configuration is performed on the front end, and the URL matches the normalization rules configured by the platform, normalization processing is performed on the back end.

[0117] In the fourth case: if the front-end JSSDK is not configured and the URL does not match the normalization rules configured by the platform, then the URL will be normalized by default.

[0118] As can be seen, processing should be carried out in the order of user configuration priority and default normalization as a fallback, to avoid excessive URL expansion, reasonably aggregate data, and reduce the overhead of log storage and computation.

[0119] See Figure 4 , Figure 4 This is a block diagram of an apparatus for implementing the URL normalization method according to embodiments of the present disclosure, such as... Figure 4 As shown, the device may include:

[0120] Module 401 is used to obtain the target URL;

[0121] The judgment module 402 is used to determine whether the target URL matches the path configuration information in the preset normalization rules, wherein the normalization rules include: path configuration information and parameter processing rules;

[0122] The first processing module 403 is used to perform preset normalization processing on the path field in the target URL and process the parameter field of the target URL according to the parameter processing rules if the judgment result of the judgment module is yes.

[0123] The second processing module 404 is used to perform default normalization processing on the target URL if the judgment result of the judgment module is negative.

[0124] In one embodiment of this disclosure, the parameter processing rule is as follows:

[0125] Retain the preset first type of custom parameters, and / or delete the preset second type of custom parameters.

[0126] In one embodiment of this disclosure, the first processing module 403 is specifically used for:

[0127] Remove numbers and / or Chinese characters from the path field of the target URL;

[0128] Alternatively, the numbers and / or Chinese characters in the path field of the target URL can be converted to preset symbols.

[0129] In one embodiment of this disclosure, the software development kit (SDK) of the page front-end is configured with a normalization function, which is used to normalize URLs that do not follow the syntax format, and the target URL is obtained by normalizing the initial page URL after it has been processed by the normalization function in the page front-end.

[0130] In one embodiment of this disclosure, the second processing module 404 is specifically used for:

[0131] The path field in the target URL is subjected to the preset normalization process, and the parameter field of the target URL is deleted.

[0132] As can be seen, in this embodiment, normalization rules are pre-defined according to business needs, including path configuration information and parameter processing rules. If the target URL matches the path configuration information, the path field and parameter field are processed synchronously, eliminating the need for two separate stages. Processing the parameter field according to the customized parameter processing rules solves the problem that parameter processing results in URL normalization schemes based on regular expressions or distance in vector space do not meet actual needs. Furthermore, the method of matching configuration information and processing parameter fields according to rules is simpler and more convenient than the normalization method based on regular expressions.

[0133] If the target URL does not match the path configuration information, it is normalized by default. This process prioritizes user configuration and uses default normalization as a fallback, preventing excessive URL expansion, rationally aggregating data, and reducing the overhead of log storage and computation.

[0134] In addition, users (URL analysts, etc.) only need to configure normalization rules on the platform, without having to sort out and classify a large number of URLs in the business, which significantly reduces the workload.

[0135] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0136] This disclosure provides an electronic device, including:

[0137] At least one processor; and

[0138] A memory communicatively connected to the at least one processor; wherein,

[0139] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a URL normalization method.

[0140] This disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform a method of URL normalization.

[0141] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements a method for URL normalization.

[0142] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0143] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0144] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the URL normalization method. For example, in some embodiments, the URL normalization method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the URL normalization method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the URL normalization method by any other suitable means (e.g., by means of firmware).

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0151] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0152] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for URL normalization, comprising: obtaining a target URL; determining whether the target URL matches path configuration information in a preset normalization rule, including determining whether a path field in the target URL other than numbers and / or Chinese characters is identical to the path configuration information in the preset normalization rule; the normalization rule including the path configuration information and a parameter processing rule; if yes, performing preset normalization processing on the path field in the target URL, and processing a parameter field of the target URL according to the parameter processing rule; wherein the parameter processing rule is configured by a normalization rule configuration platform to retain a preset first type of custom parameter and / or delete a preset second type of custom parameter; if no, performing default normalization processing on the target URL, including performing the preset normalization processing on the path field in the target URL, and deleting the parameter field of the target URL.

2. The method of claim 1, wherein, The step of performing the preset normalization processing on the path field in the target URL includes: deleting numbers and / or Chinese characters in the path field in the target URL; or converting the numbers and / or Chinese characters in the path field in the target URL into preset symbols.

3. The method of claim 1, wherein, A standardization function is configured in a software development kit of a page front end, the standardization function being used to perform standardization processing on a URL that does not follow a syntax format, and the target URL is obtained after the initial page URL is processed by the standardization function in the page front end. 4.An apparatus for URL normalization, comprising: an obtaining module configured to obtain a target URL; a determining module configured to determine whether the target URL matches path configuration information in a preset normalization rule, the normalization rule including the path configuration information and a parameter processing rule; a first processing module configured to, if the determining module determines that the target URL matches the path configuration information, perform preset normalization processing on a path field in the target URL, and process a parameter field of the target URL according to the parameter processing rule; wherein the parameter processing rule is configured by a normalization rule configuration platform to retain a preset first type of custom parameter and / or delete a preset second type of custom parameter; a second processing module configured to, if the determining module determines that the target URL does not match the path configuration information, perform default normalization processing on the target URL. The determining module is specifically configured to determine whether a path field in the target URL other than numbers and / or Chinese characters is identical to the path configuration information in the preset normalization rule. The second processing module is specifically configured to, if the determining module determines that the target URL does not match the path configuration information, perform the preset normalization processing on the path field in the target URL, and delete the parameter field of the target URL.

5. The apparatus of claim 4, wherein, The first processing module is specifically configured to: delete numbers and / or Chinese characters in the path field in the target URL; or convert the numbers and / or Chinese characters in the path field in the target URL into preset symbols.

6. The apparatus of claim 4, wherein, The software development kit of the page front end is configured with a standardized function for standardizing the URL not following the syntax format, and the target URL is obtained by processing the initial page URL through the standardized function at the page front end. 7.An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

8. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-3. 9.A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-3.

Images