Page content acquisition method and device, computer device and storage medium

Abstract

The application belongs to the technical field of page processing, and provides a page content acquisition method and device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving an access request triggered for a target URL address; judging whether the page corresponding to the target URL address can be cached according to the access request of the target URL address; when it is determined that the page pointed to by the target URL address can be cached, judging whether the target URL address is recorded in a preset whitelist; when it is determined that the target URL address is recorded in the whitelist, acquiring the page content of the page corresponding to the target URL address from a local cache; otherwise, acquiring the page content of the page corresponding to the target URL address from a system cache of a server, so that the system cache and the local cache can be simultaneously compatible, simple and efficient, and the flexibility of page content acquisition is improved.

Description

Technical Field

[0001] This application relates to the field of page processing technology, and more specifically, to a method, apparatus, computer device, and storage medium for obtaining page content. Background Technology

[0002] Caching refers to retrieving frequently used data from its original storage location in advance and storing the retrieved data in a buffer. When a user needs to retrieve the data, it can be retrieved directly from the buffer without being affected by factors such as the network or deployment of the original storage location, thereby improving the data retrieval speed.

[0003] Existing methods for retrieving page content all specify the use of system cache or local cache during browser initialization. This means that once the browser is initialized, the caching strategy is determined, and page content can only be retrieved according to a fixed caching strategy, resulting in poor flexibility. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, computer device, and storage medium for obtaining page content, so as to improve the flexibility of obtaining page content.

[0005] To achieve the above-mentioned objectives, this application provides a method for obtaining page content, which includes:

[0006] Receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address;

[0007] Based on the access request for the target URL address, determine whether the page corresponding to the target URL address can be cached;

[0008] When it is determined that the page pointed to by the target URL address can be cached, it is determined whether the target URL address is recorded in a preset whitelist;

[0009] When it is determined that the target URL address is recorded in the whitelist, the page content of the page corresponding to the target URL address is retrieved from the local cache;

[0010] When it is determined that the target URL address is not recorded in the whitelist, the page content corresponding to the target URL address is retrieved from the system cache on the server.

[0011] Preferably, determining whether the target URL address is recorded in a preset whitelist includes:

[0012] Resolve the domain name of the target URL address;

[0013] The resolved domain name is compared with the standard domain names pre-stored in the whitelist. If there is a standard domain name in the whitelist that is the same as the resolved domain name, it is determined that the target URL address is recorded in the preset whitelist.

[0014] Preferably, determining whether the page corresponding to the target URL address can be cached includes:

[0015] Determine the regular expression for the target URL address;

[0016] The regular expression of the target URL address is matched with a preset standard regular expression;

[0017] When the regular expression of the target URL address matches the standard regular expression, it is determined that the page corresponding to the target URL address can be cached.

[0018] If the regular expression of the target URL fails to match the standard regular expression, then the page corresponding to the target URL cannot be cached.

[0019] Preferably, retrieving the page content corresponding to the target URL address from the server's system cache includes:

[0020] A network request is initiated to the server using a preset framework to establish a data transmission pipeline with the server;

[0021] Receive network resource data returned from the data transmission pipeline, filter the network resource data, and obtain the page content of the page corresponding to the target URL address.

[0022] Furthermore, before receiving the access request triggered by the target URL address, the method further includes:

[0023] Collect multiple standard pages generated within the historical period;

[0024] Obtain the access frequency and cumulative access frequency of each standard page within the historical period;

[0025] The access frequency of each standard page is calculated based on the access frequency and cumulative access frequency of each standard page.

[0026] The access frequency of each standard page is compared with the preset access frequency, and the standard pages with access frequencies higher than the preset access frequency are stored in the local cache.

[0027] Standard pages with access frequency lower than the preset access frequency are stored in the system cache on the server.

[0028] Furthermore, before receiving the access request triggered by the target URL address, the method further includes:

[0029] Collect multiple standard pages generated within the historical period;

[0030] Extract the page content of each of the standard pages to obtain the standard page content of each of the standard pages;

[0031] The content of each standard page is evaluated to obtain an evaluation value for each standard page; wherein the evaluation value is used to assess the importance of the content of the standard page.

[0032] The evaluation value of each standard page is compared with a preset evaluation value, and the standard pages with evaluation values ​​lower than the preset evaluation value are stored in the local cache;

[0033] Standard pages with evaluation values ​​higher than the preset evaluation values ​​are stored in the system cache on the server.

[0034] Preferably, the step of evaluating the content of each standard page to obtain an evaluation value for each standard page includes:

[0035] The standard page content of each standard page is vectorized to obtain the target vector of each standard page.

[0036] The target vector of each standard page is input into a pre-trained evaluation model to obtain the evaluation value of each standard page.

[0037] This application also provides a page content acquisition device, which includes:

[0038] A receiving module is used to receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address;

[0039] The first judgment module is used to determine whether the page corresponding to the target URL address can be cached based on the access request of the target URL address;

[0040] The second judgment module is used to determine whether the target URL address is recorded in a preset whitelist when it is determined that the page pointed to by the target URL address can be cached.

[0041] The acquisition module is used to retrieve the page content of the page corresponding to the target URL address from the local cache when it is determined that the target URL address is recorded in the whitelist;

[0042] The acquisition module is further configured to, when it is determined that the target URL address is not recorded in the whitelist, retrieve the page content of the page corresponding to the target URL address from the system cache on the server side.

[0043] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0044] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0045] This application provides a method, apparatus, computer device, and storage medium for obtaining page content. It receives an access request triggered by a target URL address, determines whether the page corresponding to the target URL address can be cached based on the access request, and if the page pointed to by the target URL address can be cached, determines whether the target URL address is recorded in a preset whitelist. If the target URL address is recorded in the whitelist, it retrieves the page content corresponding to the target URL address from the local cache; otherwise, it retrieves the page content corresponding to the target URL address from the server's system cache. By using the whitelist to determine whether to retrieve the page content corresponding to the target URL address from the local cache or the system cache, it can simultaneously support both system and local caching, is simple and efficient, and improves the flexibility of page content retrieval. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a method for obtaining page content according to an embodiment of this application;

[0047] Figure 2 This is a schematic block diagram of a page content acquisition device according to an embodiment of this application;

[0048] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application.

[0049] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] This application proposes a method for obtaining page content. The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0052] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0053] This application proposes a page content retrieval method, using a computer device as the execution entity. This method addresses the problem that current page content retrieval methods typically specify the use of system cache or local cache during browser initialization. Once the browser is initialized, the caching strategy is fixed, limiting page content retrieval to a single, fixed strategy and resulting in poor flexibility. (Reference) Figure 1 In one embodiment, the method for obtaining the page content includes:

[0054] S11. Receive an access request triggered by a target URL address; wherein, the access request is used to request access to the page corresponding to the target URL address;

[0055] S12. Based on the access request of the target URL address, determine whether the page corresponding to the target URL address can be cached;

[0056] S13. When it is determined that the page pointed to by the target URL address can be cached, determine whether the target URL address is recorded in a preset whitelist;

[0057] S14. When it is determined that the target URL address is recorded in the whitelist, retrieve the page content of the page corresponding to the target URL address from the local cache;

[0058] S15. When it is determined that the target URL address is not recorded in the whitelist, the page content of the page corresponding to the target URL address is obtained from the system cache on the server.

[0059] In this embodiment, when a user clicks a link on the interface, the link corresponds to a target URL address, initiating an access request for the page corresponding to the target URL address. The computer device receives the access request triggered by the target URL address and determines whether the page corresponding to the target URL address can be cached based on the access request. Here, the Uniform Resource Locator (URL) locates web pages, multimedia files, etc. The server address in the URL (Internet Protocol (IP) address) is generally replaced by a domain name because IP addresses are not easy to remember.

[0060] Specifically, in this embodiment, a lookup table can be pre-built to record each cacheable URL address. The cache location can be the local cache of the computer device or the system cache of the server. Based on the access request for the target URL address, the lookup table is queried to see if the target URL address is recorded in the lookup table. If it is, the page corresponding to the target URL address is determined to be cacheable; otherwise, the page corresponding to the target URL address is determined not to be cacheable. When it is determined that the page corresponding to the target URL address cannot be cached, the user's access request triggered by the target URL address is rejected, and access to the page content is prohibited.

[0061] Once it's determined that the page pointed to by the target URL can be cached, the system further checks whether the target URL is recorded in a pre-defined whitelist. The whitelist stores URLs or domains that can be cached locally. This includes domains of pre-known secure websites, as well as domains of third-party websites with poor performance. For example, a website with excessive images might experience slow image loading, leading to poor website performance, or a website with limited resources might suffer from high traffic, resulting in performance degradation. Adding domains of third-party websites with poor performance to the whitelist aims to improve the speed at which users retrieve page content from those websites and increase the probability of users successfully retrieving page content from them.

[0062] When the target URL is found to be in the whitelist, it indicates that the page pointed to by the target URL is a frequently accessed page, a page with high security performance, or a page that loads slowly. These pages are generally cached in the local cache for quick retrieval. Therefore, the page content corresponding to the target URL is retrieved from the local cache. When the target URL is not found to be in the whitelist, the page content corresponding to the target URL is retrieved from the server's system cache. Pages cached in the system cache are generally pages with low access frequency, pages that consume a lot of memory, or pages that load quickly.

[0063] This application provides a method for retrieving page content. It receives an access request triggered by a target URL address, determines whether the page corresponding to the target URL address can be cached based on the access request, and checks whether the target URL address is recorded in a preset whitelist if the page can be cached. If the target URL address is recorded in the whitelist, it retrieves the page content of the page corresponding to the target URL address from the local cache; otherwise, it retrieves the page content of the page corresponding to the target URL address from the server's system cache. By using the whitelist to determine whether to retrieve the page content of the page corresponding to the target URL address from the local cache or the system cache, this method can simultaneously support both system and local caching, is simple and efficient, and improves the flexibility of page content retrieval.

[0064] In one embodiment, determining whether the target URL address is recorded in a preset whitelist may specifically include:

[0065] Resolve the domain name of the target URL address;

[0066] The resolved domain name is compared with the standard domain names pre-stored in the whitelist. If there is a standard domain name in the whitelist that is the same as the resolved domain name, it is determined that the target URL address is recorded in the preset whitelist.

[0067] In this embodiment, the domain name of the target URL address is parsed and compared with the standard domain name stored in the whitelist in advance. When there is a standard domain name in the whitelist that is the same as the parsed domain name, it is determined that the target URL address is recorded in the whitelist. The whitelist contains domain names of target URL addresses that can be cached locally.

[0068] In one embodiment, the existing process requires the complete domain name to be matched. If a URL contains dozens or hundreds of domain names, dozens or hundreds of whitelists need to be configured. When loading the URL https: / / abc.com, it requests resources from multiple different domain names, such as https: / / abcd.com and https: / / / abcde.com.

[0069] Therefore, based on the common characteristics of these domains, their request header information all have a common parameter referr: https: / / abc.com. So, we can consider changing domain name matching to referrer matching, which saves the technical problem of large computational load caused by configuring multiple domain names, and at the same time, using referrer configuration reduces the workload of domain name configuration.

[0070] In one embodiment, determining whether the page corresponding to the target URL address can be cached may specifically include:

[0071] Determine the regular expression for the target URL address;

[0072] The regular expression of the target URL address is matched with a preset standard regular expression;

[0073] When the regular expression of the target URL address matches the standard regular expression, it is determined that the page corresponding to the target URL address can be cached.

[0074] If the regular expression of the target URL fails to match the standard regular expression, then the page corresponding to the target URL cannot be cached.

[0075] In this embodiment, the type of the target URL is identified by matching its regular expression, thereby improving page security. The matching process primarily involves determining whether the syntax of the target URL's regular expression matches the syntax of a pre-defined standard regular expression. If they do, the target URL's regular expression is considered a match; otherwise, it is considered a mismatch.

[0076] When the regular expression for a target URL matches a standard regular expression, it indicates that the page content at that target URL has high security. To improve the speed of identifying the type of the target URL, the standard regular expression used for matching can be pre-processed. Specifically, the regular expression used to match the target URL containing the page content can be obtained by: identifying common features of the pages, collecting page samples based on these features, training the collected page samples to obtain a model, aggregating the obtained model, and outputting the regular expression corresponding to the model. The obtained regular expression is then optimized to obtain a standard regular expression by reducing the number of regular expressions and shortening their length. The optimized regular expression is pre-packaged into a database and integrated into the server.

[0077] Of course, since the time spent matching the target URL address with a regular expression is usually longer than the time spent determining whether the domain of the target URL address is in the whitelist, the method of matching the target URL address with a regular expression can be used to determine whether the content of the page pointed to by the target URL address can be cached when it is determined that the domain of the target URL address is not in the whitelist. No specific restrictions are made here.

[0078] In one embodiment, retrieving the page content corresponding to the target URL address from the server's system cache may specifically include:

[0079] A network request is initiated to the server using a preset framework to establish a data transmission pipeline with the server;

[0080] Receive network resource data returned from the data transmission pipeline, filter the network resource data, and obtain the page content of the page corresponding to the target URL address.

[0081] This application utilizes the OkHttp framework to initiate network requests to the server, establishing a data transmission pipeline. This pipeline includes an input pipeline and an output pipeline. It leverages this pipeline to enable direct communication between the main thread and child threads, allowing for simultaneous downloading and display. Specifically, during the process of reading network resource data returned by the data transmission pipeline, the currently read network resource data is filtered to obtain the page content pointed to by the target URL address, and the page content is loaded and displayed. Simultaneously, the currently read network resource data is synchronously cached. For this synchronous caching process, this application synchronously caches the webpage resource address corresponding to the currently read network resource data in the database.

[0082] In one embodiment, before receiving the access request triggered by the target URL address, the process may further include:

[0083] Collect multiple standard pages generated within the historical period;

[0084] Obtain the access frequency and cumulative access frequency of each standard page within the historical period;

[0085] The access frequency of each standard page is calculated based on the access frequency and cumulative access frequency of each standard page.

[0086] The access frequency of each standard page is compared with the preset access frequency, and the standard pages with access frequencies higher than the preset access frequency are stored in the local cache.

[0087] Standard pages with access frequency lower than the preset access frequency are stored in the system cache on the server.

[0088] This embodiment collects multiple standard pages generated within a historical period, obtains the access frequency and cumulative access frequency of each standard page within the historical period, and calculates the access frequency of each standard page by dividing the access frequency of each standard page by the cumulative access frequency. For example, when the access frequency of standard page A within the historical period is 1000 times, and the cumulative access frequency of all standard pages is 10000 times, then the access frequency of standard page A within the historical period is 10%.

[0089] In addition, this embodiment can also set a preset access frequency, which can be customized. For example, the preset access frequency can be set to 20%. Then, the access frequency of each standard page is compared with the preset access frequency. Standard pages with access frequencies higher than the preset access frequency are stored in the local cache, and standard pages with access frequencies lower than the preset access frequency are stored in the system cache on the server side, so as to quickly obtain the page content of the page corresponding to the target URL address from the local cache.

[0090] In one embodiment, before receiving the access request triggered by the target URL address, the process may further include:

[0091] Collect multiple standard pages generated within the historical period;

[0092] Extract the page content of each of the standard pages to obtain the standard page content of each of the standard pages;

[0093] The content of each standard page is evaluated to obtain an evaluation value for each standard page; wherein the evaluation value is used to assess the importance of the content of the standard page.

[0094] The evaluation value of each standard page is compared with a preset evaluation value, and the standard pages with evaluation values ​​lower than the preset evaluation value are stored in the local cache;

[0095] Standard pages with evaluation values ​​higher than the preset evaluation values ​​are stored in the system cache on the server.

[0096] This embodiment collects multiple standard pages generated within a historical period, extracts the page content of each standard page, and evaluates the standard page content of each standard page to obtain an evaluation value for each standard page. This evaluation includes factors such as page content access volume and page design. The evaluation value of each standard page is then compared with a preset evaluation value. Standard pages with evaluation values ​​lower than the preset value are stored in the local cache, while those with evaluation values ​​higher than the preset value are stored in the server-side system cache. This ensures that highly important pages are stored in the server-side system cache to prevent corruption. Furthermore, when caching page content, both symmetric and asymmetric encryption algorithms are used for encryption to ensure data security.

[0097] In symmetric encryption algorithms, the sender processes the plaintext (original data) and encryption key together using a special encryption algorithm, transforming it into complex ciphertext before sending it. The receiver, upon receiving the ciphertext, needs to use the same encryption key and the inverse algorithm to decrypt it and recover the readable plaintext. Because symmetric encryption uses only one key, both the sender and receiver use this key for encryption and decryption, the decryptor must know the encryption key beforehand.

[0098] Asymmetric encryption algorithms require two keys: a public key and a private key. The public and private keys are a pair; if data is encrypted using the public key, it can only be decrypted using the corresponding private key. Because encryption and decryption use two different keys, this type of algorithm is called an asymmetric encryption algorithm.

[0099] In one embodiment, evaluating the content of each standard page to obtain an evaluation value for each standard page may specifically include:

[0100] The standard page content of each standard page is vectorized to obtain the target vector of each standard page.

[0101] The target vector of each standard page is input into a pre-trained evaluation model to obtain the evaluation value of each standard page.

[0102] This embodiment can perform vector transformation on the content of each standard page to obtain the target vector of each standard page; for example, vector transformation can be performed on the text content in the standard page to obtain the text vector, and then the target vector of each standard page can be input into a pre-trained evaluation model to obtain the evaluation value of each standard page; wherein, the evaluation model is a pre-trained neural network model used for automatic evaluation of page content, thereby improving evaluation efficiency.

[0103] Reference Figure 2 This application also provides a page content acquisition device, including:

[0104] The receiving module 11 is used to receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address;

[0105] The first judgment module 12 is used to determine whether the page corresponding to the target URL address can be cached based on the access request of the target URL address;

[0106] The second judgment module 13 is used to determine whether the target URL address is recorded in a preset whitelist when it is determined that the page pointed to by the target URL address can be cached.

[0107] The acquisition module 14 is used to retrieve the page content of the page corresponding to the target URL address from the local cache when it is determined that the target URL address is recorded in the whitelist;

[0108] The acquisition module 14 is further configured to retrieve the page content of the page corresponding to the target URL address from the system cache of the server when it is determined that the target URL address is not recorded in the whitelist.

[0109] In this embodiment, when a user clicks a link on the interface, the link corresponds to a target URL address, initiating an access request for the page corresponding to the target URL address. The computer device receives the access request triggered by the target URL address and determines whether the page corresponding to the target URL address can be cached based on the access request. Here, the Uniform Resource Locator (URL) locates web pages, multimedia files, etc. The server address in the URL (Internet Protocol (IP) address) is generally replaced by a domain name because IP addresses are not easy to remember.

[0110] Specifically, in this embodiment, a lookup table can be pre-built to record each cacheable URL address. The cache location can be the local cache of the computer device or the system cache of the server. Based on the access request for the target URL address, the lookup table is queried to see if the target URL address is recorded in the lookup table. If it is, the page corresponding to the target URL address is determined to be cacheable; otherwise, the page corresponding to the target URL address is determined not to be cacheable. When it is determined that the page corresponding to the target URL address cannot be cached, the user's access request triggered by the target URL address is rejected, and access to the page content is prohibited.

[0111] Once it's determined that the page pointed to by the target URL can be cached, the system further checks whether the target URL is recorded in a pre-defined whitelist. The whitelist stores URLs or domains that can be cached locally. This includes domains of pre-known secure websites, as well as domains of third-party websites with poor performance. For example, a website with excessive images might experience slow image loading, leading to poor website performance, or a website with limited resources might suffer from high traffic, resulting in performance degradation. Adding domains of third-party websites with poor performance to the whitelist aims to improve the speed at which users retrieve page content from those websites and increase the probability of users successfully retrieving page content from them.

[0112] When the target URL is found to be in the whitelist, it indicates that the page pointed to by the target URL is a frequently accessed page, a page with high security performance, or a page that loads slowly. These pages are generally cached in the local cache for quick retrieval. Therefore, the page content corresponding to the target URL is retrieved from the local cache. When the target URL is not found to be in the whitelist, the page content corresponding to the target URL is retrieved from the server's system cache. Pages cached in the system cache are generally pages with low access frequency, pages that consume a lot of memory, or pages that load quickly.

[0113] As described above, it is understood that each component of the page content acquisition device proposed in this application can realize the function of any of the page content acquisition methods described above, and the specific structure will not be described in detail.

[0114] Reference Figure 3 This application also provides a computer device whose internal structure can be as follows: Figure 3As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a storage medium and internal memory. The storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the storage medium. The database stores data related to the page content retrieval method. The network interface communicates with external terminals via a network connection. When the processor executes the computer program, it implements a page content retrieval method.

[0115] The processor described above executes the page content retrieval method described above, including:

[0116] Receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address;

[0117] Based on the access request for the target URL address, determine whether the page corresponding to the target URL address can be cached;

[0118] When it is determined that the page pointed to by the target URL address can be cached, it is determined whether the target URL address is recorded in a preset whitelist;

[0119] When it is determined that the target URL address is recorded in the whitelist, the page content of the page corresponding to the target URL address is retrieved from the local cache;

[0120] When it is determined that the target URL address is not recorded in the whitelist, the page content corresponding to the target URL address is retrieved from the system cache on the server.

[0121] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a page content acquisition method, including the following steps:

[0122] Receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address;

[0123] Based on the access request for the target URL address, determine whether the page corresponding to the target URL address can be cached;

[0124] When it is determined that the page pointed to by the target URL address can be cached, it is determined whether the target URL address is recorded in a preset whitelist;

[0125] When it is determined that the target URL address is recorded in the whitelist, the page content of the page corresponding to the target URL address is retrieved from the local cache;

[0126] When it is determined that the target URL address is not recorded in the whitelist, the page content corresponding to the target URL address is retrieved from the system cache on the server.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0128] In summary, the greatest benefit of this application is that:

[0129] This application provides a method, apparatus, computer device, and storage medium for obtaining page content. It receives an access request triggered by a target URL address, determines whether the page corresponding to the target URL address can be cached based on the access request, and if the page pointed to by the target URL address can be cached, determines whether the target URL address is recorded in a preset whitelist. If the target URL address is recorded in the whitelist, it retrieves the page content corresponding to the target URL address from the local cache; otherwise, it retrieves the page content corresponding to the target URL address from the server's system cache. By using the whitelist to determine whether to retrieve the page content corresponding to the target URL address from the local cache or the system cache, it can simultaneously support both system and local caching, is simple and efficient, and improves the flexibility of page content retrieval.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0131] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of obtaining page content, characterized by, include: Receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address; Based on the access request for the target URL address, determine whether the page corresponding to the target URL address can be cached; When it is determined that the page pointed to by the target URL address can be cached, it is determined whether the target URL address is recorded in a preset whitelist; When it is determined that the target URL address is recorded in the whitelist, the page content corresponding to the target URL address is retrieved from the local cache; When it is determined that the target URL address is not recorded in the whitelist, the page content of the page corresponding to the target URL address is obtained from the system cache on the server. The method further includes, before receiving the access request triggered by the target URL address: Collect multiple standard pages generated within the historical period; Extract the page content of each of the standard pages to obtain the standard page content of each of the standard pages; The content of each standard page is evaluated to obtain an evaluation value for each standard page; wherein the evaluation value is used to assess the importance of the content of the standard page. The evaluation value of each standard page is compared with a preset evaluation value, and the standard pages with evaluation values ​​lower than the preset evaluation value are stored in the local cache; Standard pages with evaluation values ​​higher than the preset evaluation values ​​are stored in the system cache on the server side, and the page content is encrypted using symmetric encryption algorithm and asymmetric encryption algorithm when caching; The step of evaluating the content of each standard page to obtain an evaluation value for each standard page includes: The standard page content of each standard page is vectorized to obtain the target vector of each standard page. The target vector of each standard page is input into the pre-trained evaluation model to obtain the evaluation value of each standard page; The method further includes, before receiving the access request triggered by the target URL address: Collect multiple standard pages generated within the historical period; Obtain the access frequency and cumulative access frequency of each standard page within the historical period; The access frequency of each standard page is calculated based on the access frequency and cumulative access frequency of each standard page. The access frequency of each standard page is compared with the preset access frequency, and the standard pages with access frequencies higher than the preset access frequency are stored in the local cache. Standard pages with access frequency lower than the preset access frequency are stored in the system cache on the server side; This involves setting a preset access frequency, which can be customized. The access frequency of each standard page is compared with the preset access frequency. Standard pages with access frequencies higher than the preset access frequency are stored in the local cache, while standard pages with access frequencies lower than the preset access frequency are stored in the system cache on the server.

2. The method of claim 1, wherein, The step of determining whether the target URL address is recorded in a preset whitelist includes: Resolve the domain name of the target URL address; The resolved domain name is compared with the standard domain names pre-stored in the whitelist. If there is a standard domain name in the whitelist that is the same as the resolved domain name, it is determined that the target URL address is recorded in the preset whitelist.

3. The method of claim 1, wherein, The step of determining whether the page corresponding to the target URL address can be cached includes: Determine the regular expression for the target URL address; The regular expression of the target URL address is matched with a preset standard regular expression; When the regular expression of the target URL address matches the standard regular expression, it is determined that the page corresponding to the target URL address can be cached. If the regular expression of the target URL fails to match the standard regular expression, then the page corresponding to the target URL cannot be cached.

4. The method of claim 1, wherein, The step of retrieving the page content corresponding to the target URL address from the server's system cache includes: A network request is initiated to the server using a preset framework to establish a data transmission pipeline with the server; Receive network resource data returned from the data transmission pipeline, filter the network resource data, and obtain the page content of the page corresponding to the target URL address.

5. A page content acquisition apparatus characterized by comprising: include: A receiving module is used to receive an access request triggered by a target URL address; wherein the access request is used to request access to the page corresponding to the target URL address; The first judgment module is used to determine whether the page corresponding to the target URL address can be cached based on the access request of the target URL address; The second judgment module is used to determine whether the target URL address is recorded in a preset whitelist when it is determined that the page pointed to by the target URL address can be cached. The acquisition module is used to retrieve the page content of the page corresponding to the target URL address from the local cache when it is determined that the target URL address is recorded in the whitelist; The acquisition module is further configured to, when it is determined that the target URL address is not recorded in the whitelist, retrieve the page content of the page corresponding to the target URL address from the system cache on the server side; The method further includes, before receiving the access request triggered by the target URL address: Collect multiple standard pages generated within the historical period; Extract the page content of each of the standard pages to obtain the standard page content of each of the standard pages; The content of each standard page is evaluated to obtain an evaluation value for each standard page; wherein the evaluation value is used to assess the importance of the content of the standard page. The evaluation value of each standard page is compared with a preset evaluation value, and the standard pages with evaluation values ​​lower than the preset evaluation value are stored in the local cache; Standard pages with evaluation values ​​higher than the preset evaluation values ​​are stored in the system cache on the server side, and the page content is encrypted using symmetric encryption algorithm and asymmetric encryption algorithm when caching; The step of evaluating the content of each standard page to obtain an evaluation value for each standard page includes: The standard page content of each standard page is vectorized to obtain the target vector of each standard page. The target vector of each standard page is input into the pre-trained evaluation model to obtain the evaluation value of each standard page; The method further includes, before receiving the access request triggered by the target URL address: Collect multiple standard pages generated within the historical period; Obtain the access frequency and cumulative access frequency of each standard page within the historical period; The access frequency of each standard page is calculated based on the access frequency and cumulative access frequency of each standard page. The access frequency of each standard page is compared with the preset access frequency, and the standard pages with access frequencies higher than the preset access frequency are stored in the local cache. Standard pages with access frequency lower than the preset access frequency are stored in the system cache on the server side; This involves setting a preset access frequency, which can be customized. The access frequency of each standard page is compared with the preset access frequency. Standard pages with access frequencies higher than the preset access frequency are stored in the local cache, while standard pages with access frequencies lower than the preset access frequency are stored in the system cache on the server.

6. A computer device, comprising: include: processor; Memory; The memory stores a computer program, and the processor executes the computer program to implement the page content acquisition method according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the page content acquisition method according to any one of claims 1-4.

Images