Page cache processing method, system and device, and storage medium

Abstract

This invention discloses a page caching method, system, device, and storage medium, relating to the field of computer technology. By pre-setting a link template definition table to characterize the correspondence between defined parameter features and link templates, after obtaining modification data released by the distributor, a distributed lookup is performed from the link template definition table based on the parameter features in the modification data to determine the relevant link templates affected by the modification data. Then, based on the parameter values ​​and link templates in the modification data, specific page link requests are obtained. New page content is then retrieved from the business service cluster based on the page link requests to proactively update the page cache, improving the real-time performance of front-end display and reducing manual costs.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a page caching method, system, apparatus, and storage medium. Background Technology

[0002] As the company's vertical media business expands and more and more merchants join, the volume of data is growing exponentially. Taking vehicle dealerships as an example, the number of vehicle models managed by dealers on the company's vertical media platform has reached 100,000, with nearly 100,000 dealers distributed across more than 300 regions nationwide. Through the combination of these data dimensions and the daily market information released, the number of dealer price quotation pages and market information pages is enormous, reaching hundreds of millions, and the requirements for timely page updates are becoming increasingly stringent.

[0003] Currently, the front-end pages or interfaces use passive caching. After the back-end page content is updated, the front-end can only display the updated content after the current page cache expires. In the company's vertical media business, distributors often need to update price quotes or market trends in real time. This often requires manual processing to find the page links involved in the distributor's content modifications and then update the front-end page cache. Identifying relevant links from massive amounts of data is very difficult, sometimes even requiring an exhaustive search of all possible links before clearing the cache, resulting in very high manual costs. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a page caching processing method, system, device, and storage medium, which can automatically update the page cache, improve the real-time performance of front-end display, and reduce manual costs.

[0005] On one hand, embodiments of the present invention provide a page caching processing method, including the following steps:

[0006] Obtain modified data, wherein the modified data includes parameter features and parameter values;

[0007] Multiple link templates are matched from the link template definition table based on the parameter features, wherein the link template definition table is used to characterize the mapping relationship between the link templates and multiple features;

[0008] The page link request is determined based on the parameter values ​​and each of the link templates;

[0009] Based on the page link request, new page content is retrieved from the business service cluster to update the page cache.

[0010] According to some embodiments of the present invention, the page caching processing method further includes the following steps:

[0011] The parameter values ​​in the modified data are deduplicated to obtain the corrected modified data.

[0012] According to some embodiments of the present invention, determining the page link request based on the parameter value and each of the link templates includes the following steps:

[0013] The parameters of each link template are determined from the link template definition table;

[0014] By concatenating the parameters and parameter values ​​into the link template, each page link request is obtained.

[0015] According to some embodiments of the present invention, the link templates in the link template definition table are further configured with priorities, and the page caching processing method further includes the following steps:

[0016] The priority of the corresponding page link request is determined based on the priority of the link template.

[0017] Page link requests are scheduled sequentially according to their priority to access the business service cluster.

[0018] According to some embodiments of the present invention, the step of obtaining new page content from the business service cluster based on the page link request to update the page cache includes the following steps:

[0019] Retrieve new page content from the business service cluster based on the page link request;

[0020] The new page content is compared with the cached page content. If the new page content is inconsistent with the cached page content, the page cache in the cache server is updated.

[0021] According to some embodiments of the present invention, the link template definition table is obtained by statistics from the business service cluster, and the link template definition table includes parameters, parameter characteristics, and mapping relationships between link templates.

[0022] According to some embodiments of the present invention, the page caching processing method further includes the following steps:

[0023] Extract parameters and corresponding link templates from the obtained link template definition table to obtain a parameter whitelist;

[0024] The parameter whitelist is sent to the cache server so that the cache server can filter invalid page access requests based on the parameter whitelist.

[0025] On the other hand, embodiments of the present invention also provide a page caching processing system, including:

[0026] The first module is used to acquire modified data, wherein the modified data includes parameter features and parameters;

[0027] The second module is used to match multiple link templates from the link template definition table according to the parameter features, wherein the link template definition table is used to characterize the mapping relationship between the link template and multiple features;

[0028] The third module is used to determine the page link request based on the parameter value and each of the link templates;

[0029] The fourth module is used to retrieve new page content from the business service cluster based on the page link request in order to update the page cache.

[0030] On the other hand, embodiments of the present invention also provide a page caching processing apparatus, comprising:

[0031] At least one processor;

[0032] At least one memory for storing at least one program;

[0033] When the at least one program is executed by the at least one processor, the at least one processor implements the page caching method as described above.

[0034] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the page caching processing method described above.

[0035] The above-mentioned technical solution of the present invention has at least one of the following advantages or beneficial effects: by setting a link template definition table in advance to characterize the correspondence between the defined parameter features and the link template, after obtaining the modified data released by the dealer, a distributed lookup is performed from the link template definition table based on the parameter features in the modified data to determine the relevant link templates affected by the modified data, and then the specific page link request is obtained according to the parameter values ​​and link templates in the modified data. New page content is obtained from the business service cluster according to the page link request to actively update the page cache, improve the real-time performance of the front-end display, and reduce manual costs. Attached Figure Description

[0036] Figure 1 This is a flowchart of the page caching processing method provided in the embodiments of the present invention;

[0037] Figure 2 This is a schematic diagram of the page caching processing device provided in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the cache management platform provided in an embodiment of the present invention;

[0039] Figure 4 This is a representation of the link template definition provided in the embodiments of the present invention. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar originals or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0041] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0042] In the description of this invention, the use of terms such as "first," "second," etc., is merely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order of the technical features indicated.

[0043] This invention provides a page caching method, referring to... Figure 1 The page caching processing method of this invention includes, but is not limited to, steps S110, S120, S130 and S140.

[0044] Step S110: Obtain modified data, wherein the modified data includes parameter characteristics and parameter values;

[0045] Step S120: Match multiple link templates from the link template definition table according to the parameter features, wherein the link template definition table is used to characterize the mapping relationship between the link template and multiple features;

[0046] Step S130: Determine the page link request based on the parameter values ​​and each link template;

[0047] Step S140: Obtain new page content from the business service cluster based on the page link request to update the page cache.

[0048] Specifically, refer to Figure 3 The page caching processing method of this invention can be applied to, for example, Figure 3The TSM (Traffic Server Manager Platform) shown includes a cache management platform, a cache control unit, a business service cluster, and an ATS (Apache Traffic Server, a high-performance cache server). The backend business service cluster includes, but is not limited to, merchant services, pricing services, and market data services. The cache control unit includes notification service subunits, matching service subunits, scheduling service subunits, push service subunits, and backend management subunits. The operational terminals and the cache management platform can publish and subscribe to messages via the MQTT protocol to achieve real-time transmission of massive amounts of messages.

[0049] When an operational terminal publishes modified data for pages displaying price quotes, vehicle model information, etc., the notification service subunit of the cache management platform receives this modified data. The matching service subunit searches for all relevant link templates in the link template definition table based on the parameter characteristics in the modified data, and obtains multiple page link requests based on the parameter values ​​in the modified data and the multiple link templates. The scheduling service subunit consumes the page link requests according to certain rules and sends them to the push service subunit. The push service subunit accesses the backend business service cluster based on the page link requests, retrieves the page content of the requested page, and pushes the page content to the cache server for caching. This achieves automatic querying of all relevant pages involved in the modified parameters, avoiding omissions, and real-time updating of the page content in the cache server.

[0050] When users query page content from the cache server via PC, WAP, or API, they can obtain real-time updated content without waiting for the previous page content to expire before retrieving data modified by the backend operation terminal. Furthermore, this embodiment of the invention avoids cache avalanche, which occurs when a large number of page caches expire simultaneously, triggering concurrent access to the backend server (i.e., the business service cluster). If the traffic is too concentrated, the backend server load will spike instantly, impacting normal service. The cache management platform of this embodiment adds a cache control unit to the cache server and the business service cluster. When the operation terminal changes page data, the cache control unit proactively updates the cache without setting an expiration date. Therefore, it prevents a surge of requests to the backend server due to cache expiration, ensuring stable service operation.

[0051] According to some specific embodiments of the present invention, the page caching processing method of the present invention further includes, but is not limited to, step S120.

[0052] Step S120: Deduplicate the parameter values ​​in the modified data to obtain the corrected modified data.

[0053] In this embodiment, merchants may repeatedly modify the same parameter value within a short period of time. To avoid wasting computer resources on invalid modification operations of the operating terminal, the parameter values ​​of the acquired modification data are deduplicated. Specifically, the modification data includes parameters and parameter values. The notification service subunit saves non-duplicate parameters within a certain period. When the notification service subunit receives the current modification data, it searches for all saved parameter values ​​in the modification data and determines whether the parameter value in the current modification data is the same as the saved parameter value. If they are the same, the page cache update operation for the same parameter value is not continued. If they are different, the different parameter value in the modification data is sent to the matching service subunit for processing.

[0054] According to some specific embodiments of the present invention, the business service cluster, based on its own page development plan, statistically analyzes which parameters affect the page, thereby obtaining a link template definition table. For example... Figure 4 As shown, the link template definition table includes the mapping relationship between parameters, parameter characteristics, and link templates. Furthermore, each link template in the table is also configured with a priority. Priority is used to characterize the processing order of the corresponding link template; the processing order can be either lower priority or higher priority. Parameters are used to characterize the valid parameters used by the corresponding link template. Parameter characteristics are used to characterize what type of parameters affect the link template. For example, "DEALER" indicates that it is affected by the dealer, meaning the user may have modified the dealer information; "DEALER" and "SERIALGROUP" indicate that it is affected by the dealer and the vehicle series, meaning the user may have modified the vehicle series information, pricing, etc.

[0055] According to some specific embodiments of the present invention, step S130, which involves determining the page link request based on the parameter value and each link template, includes, but is not limited to, steps S310 and S320.

[0056] Step S310: Determine the parameters of each link template from the link template definition table;

[0057] Step S320: Concatenate the parameters and parameter values ​​into the link template to obtain each page link request.

[0058] In this embodiment, after the matching service subunit obtains the modified data, it matches the link templates " / qd / dealer / intf / getDealerlnfo" and " / qd / price / web / shop / sgPrice.html" based on the parameter feature "DEALER" in the modified data. Then, it combines the corresponding parameter "dealerld" and the corresponding parameter value in the modified data and concatenates them into the two link templates to obtain the two specific page link requests.

[0059] According to some specific embodiments of the present invention, the link templates in the link template definition table are further configured with priorities, and the page caching processing method of the present invention includes, but is not limited to, steps S410 and S420.

[0060] Step S410: Determine the priority of the corresponding page link request based on the priority of the link template;

[0061] Step S420: Page link requests are scheduled in order of priority to access the business service cluster.

[0062] In this embodiment, as Figure 4 As shown, each link template is configured with a priority. Correspondingly, page link requests obtained based on this link template and specific parameter values ​​also have the same priority as the link template. In the matching service subunit, after determining multiple link templates based on the parameter characteristics in the modified data, page link requests are processed according to the priority order of the link templates and sent to the scheduling service subunit in sequence. The scheduling service subunit adds page link requests to the corresponding priority queue according to their priority. Only after all page link requests in the higher priority queue have been consumed will the next priority queue be consumed, thus making it easier to update more important pages and improving the real-time performance of important page updates.

[0063] According to some specific embodiments of the present invention, in step S140, the step of obtaining new page content from the business service cluster according to the page link request to update the page cache includes, but is not limited to, steps S510 and S520.

[0064] Step S510: Obtain new page content from the business service cluster based on the page link request;

[0065] Step S520: Compare the new page content with the cached page content. If the new page content is inconsistent with the cached page content, update the page cache in the cache server.

[0066] In this embodiment, the scheduling service subunit consumes page link requests and passes them to the push service subunit. If an exception occurs during scheduling, a failover can be performed via retry. The push service subunit accesses the business service cluster based on the page link request, calls the backend service to obtain new page content, generates a content digest, and compares the new content digest with the cached content digest. If they are inconsistent, it indicates that the new page content has changed compared to the cached page content. In this case, the API interface of the cache server is requested to push the new content to overwrite the old cached content.

[0067] According to some specific embodiments of the present invention, the page caching processing method of the present invention further includes, but is not limited to, steps S610 and S620.

[0068] Step S610: Extract parameters and corresponding link templates from the obtained link template definition table to obtain the parameter whitelist;

[0069] Step S620: Send the parameter whitelist to the cache server so that the cache server can filter invalid page access requests based on the parameter whitelist.

[0070] In this embodiment, the background management subunit in the cache control unit obtains the link template definition table from the business service cluster, extracts parameters and corresponding link templates based on the link template definition table, obtains a parameter whitelist, and installs the parameter whitelist and the parameter order in the page access request into the cache server in the form of a cache-key plugin. This achieves non-intrusive automatic configuration of the cache server's filtering function and the automatic sorting function of parameters in the front-end page access links, avoiding cache penetration of the cache server.

[0071] Specifically, cache penetration is a common phenomenon. A typical caching server generates cached content based on parameters in the page request received from the frontend. If the caching server cannot find the parameters in the page request, it will further access the backend server. In this scenario, if a malicious request or web crawler modifies the format of the page request, including changing parameters or their order, the caching server will fail to find the parameters and will instead access the backend server. This is equivalent to the caching server becoming ineffective, increasing the load on the backend server and potentially causing it to crash. This embodiment of the invention uses a cache-key plugin to configure a whitelist of valid parameters and their order for each page link on the caching server. After receiving a page request, the caching server filters and sorts the parameters according to the whitelist, transforming the request into a format that conforms to the specified format for cache content retrieval. Therefore, regardless of how many parameters are added to the page request, only valid parameters are retained before reaching the cache query, thus avoiding cache penetration and its impact on service performance.

[0072] According to some specific embodiments of the present invention, the background management subunit supports the management of links, such as deletion, disabling, and refreshing the cache. For example, if the database adjusts the data, the cached content can be refreshed completely or individually.

[0073] On the other hand, embodiments of the present invention also provide a page caching processing system, including:

[0074] The first module is used to obtain modified data, which includes parameter characteristics and parameters;

[0075] The second module is used to match multiple link templates from the link template definition table based on parameter features. The link template definition table is used to characterize the mapping relationship between link templates and multiple features.

[0076] The third module is used to determine the page link request based on the parameter values ​​and each link template.

[0077] The fourth module is used to retrieve new page content from the business service cluster based on page link requests in order to update the page cache.

[0078] It is understood that the content of the above-described page caching processing method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above-described page caching processing method embodiments, and the beneficial effects achieved are also the same as those achieved in the above-described page caching processing method embodiments.

[0079] Reference Figure 2 , Figure 2 This is a schematic diagram of a page caching processing apparatus according to an embodiment of the present invention. The page caching processing apparatus of this embodiment includes one or more control processors and a memory. Figure 2 The example consists of a control processor and a memory.

[0080] The control processor and memory can be connected via a bus or other means. Figure 2 Taking the example of a connection between China and Israel via a bus.

[0081] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the control processor, and these remote memories can be connected to the page cache processing device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] Those skilled in the art will understand that Figure 2 The device structure shown does not constitute a limitation on the page caching processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] The non-transitory software program and instructions required to implement the page cache processing method applied to the page cache processing device in the above embodiments are stored in the memory. When executed by the controlled processor, the page cache processing method applied to the page cache processing device in the above embodiments is executed.

[0084] Furthermore, one embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that are executed by one or more control processors, causing the one or more control processors to perform the page caching processing method in the above method embodiment.

[0085] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0086] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A page cache handling method, characterized by, Includes the following steps: Obtain modified data, wherein the modified data includes parameter features and parameter values; Multiple link templates are matched from the link template definition table based on the parameter features, wherein the link template definition table includes a mapping relationship between parameters, parameter features, and link templates; the parameters are used to characterize the effective parameters used by the corresponding link template, and the parameter features are used to characterize the type of parameters that affect the link template; The page link request is determined based on the parameter values ​​and each of the link templates; Based on the page link request, new page content is retrieved from the business service cluster to update the page cache; The page caching method further includes: Extract parameters and corresponding link templates from the obtained link template definition table to obtain a parameter whitelist; The parameter whitelist and the parameter order in the page access request are sent to the cache server in the form of a plugin, so that the cache server can filter invalid page access requests according to the parameter whitelist and sort the parameters of the page access request according to the parameter order.

2. The page cache handling method of claim 1, wherein, The page caching method further includes the following steps: The parameter values ​​in the modified data are deduplicated to obtain the corrected modified data.

3. The page cache handling method of claim 1, wherein, Determining the page link request based on the parameter value and each link template includes the following steps: The parameters of each link template are determined from the link template definition table; By concatenating the parameters and parameter values ​​into the link template, each page link request is obtained.

4. The page cache handling method of claim 1, wherein, The link templates in the link template definition table are also configured with priorities, and the page caching method further includes the following steps: The priority of the corresponding page link request is determined based on the priority of the link template. Page link requests are scheduled sequentially according to their priority to access the business service cluster.

5. The page caching method according to claim 1, characterized in that, The step of retrieving new page content from the business service cluster based on the page link request to update the page cache includes the following steps: Retrieve new page content from the business service cluster based on the page link request; The new page content is compared with the cached page content. If the new page content is inconsistent with the cached page content, the page cache in the cache server is updated.

6. A page caching system, characterized in that, include: The first module is used to acquire modified data, wherein the modified data includes parameter features and parameters; The second module is used to match multiple link templates from the link template definition table according to the parameter features. The link template definition table includes a mapping relationship between parameters, parameter features, and link templates. The parameters are used to characterize the effective parameters used by the corresponding link template, and the parameter features are used to characterize the type of parameters that affect the link template. The third module is used to determine the page link request based on the parameter value and each of the link templates; The fourth module is used to retrieve new page content from the business service cluster based on the page link request in order to update the page cache; The page caching system is further configured to extract parameters and corresponding link templates from the obtained link template definition table to obtain a parameter whitelist; and send the parameter whitelist and the parameter order in the page access request to the cache server in the form of a plugin, so that the cache server filters invalid page access requests according to the parameter whitelist and sorts the parameters of the page access request according to the parameter order.

7. A page caching processing device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the page caching processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, When the processor executes the program, it is used to implement the page caching method as described in any one of claims 1 to 5.

Images