Method, apparatus, electronic device, and storage medium for data processing

Abstract

This invention discloses a data processing method, apparatus, electronic device, and storage medium, relating to the field of computer technology. One specific embodiment of the method includes: receiving a data processing instruction; obtaining the corresponding sliding window for each stored data; calculating the access parameters for each stored data; responding to a reporting instruction for the access parameters; obtaining the corresponding data identifier; obtaining the remaining expiration time of the data identifier to determine the current remaining time of the data identifier; and updating the local window lock of the data identifier based on the current remaining time; responding to a successful update of the local window lock; reporting the access parameters associated with the data identifier to a caching system; and responding to a triggering of a popularity acquisition task; obtaining the access popularity of each stored data from a preset caching system. This embodiment can solve the problem that analyzing client access data through a server requires establishing an independent server, resulting in high operational complexity and resource consumption.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In distributed applications and similar scenarios, clients typically use external caching systems for data storage to shorten API response times. However, when the interfaces related to the external cache are under high concurrency, the corresponding cached data is also accessed frequently, easily leading to bottlenecks. Related technologies often use third-party servers to analyze client access data, identify frequently accessed data, and allow clients to store this data locally, reducing access to the external cache. However, this approach requires establishing a separate server implementation and communication methods with the client, resulting in high complexity and resource consumption. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data processing method, apparatus, electronic device, and storage medium that can solve the problem that analyzing client access data through a server requires establishing an independent server, which is highly complex and consumes a lot of resources.

[0004] To achieve the above objectives, according to one aspect of the present invention, a data processing method is provided.

[0005] An embodiment of the present invention provides a data processing method comprising: receiving a data processing instruction, obtaining a corresponding sliding window for each stored data, and calculating access parameters for each stored data; responding to a reporting instruction for access parameters, obtaining a corresponding data identifier, obtaining the remaining expiration time of the data identifier from a preset cache system to determine the current remaining time of the data identifier, and updating the local window lock of the data identifier based on the current remaining time; responding to a successful update of the local window lock, reporting the access parameters associated with the data identifier to the cache system so that the cache system calculates the access popularity corresponding to the data identifier; and responding to a popularity acquisition task being triggered, obtaining the access popularity of each stored data from the cache system and establishing a mapping relationship between the data identifier of each stored data and the corresponding access popularity.

[0006] In one embodiment, obtaining the remaining expiration time of the data identifier from the caching system to determine the current remaining time of the data identifier includes: obtaining the remote window lock corresponding to the data identifier from the caching system, parsing the remote window lock to obtain the remaining expiration time of the data identifier; determining the remaining expiration time as the current remaining time in response to the remaining expiration time being greater than a preset threshold; and obtaining a preset remaining time in response to the remaining expiration time not being greater than the preset threshold to determine the current remaining time.

[0007] In another embodiment, the corresponding sliding window for each stored data is obtained to calculate the access parameters of each stored data, including: for each stored data, obtaining the time interval of the corresponding sliding window, obtaining the number of times the stored data is accessed within the time interval to determine the access parameters of the stored data corresponding to the time interval, and updating the time interval of the sliding window based on a preset sliding period.

[0008] In yet another embodiment, updating the local window lock of the data identifier based on the current remaining time includes: obtaining the state of the local window lock corresponding to the data identifier; updating the time parameter in the local window lock based on the current remaining time in response to the state being a first state; and stopping the update operation in response to the state being a second state.

[0009] In another embodiment, the method further includes: receiving a data access instruction, obtaining a corresponding target data identifier, and querying the access popularity associated with the target data identifier; in response to the access popularity meeting a preset condition, obtaining the data corresponding to the target data identifier from a local cache; and in response to the access popularity not meeting the preset condition, obtaining the data corresponding to the target data identifier from the cache system.

[0010] In another embodiment, establishing a mapping relationship between the data identifier of each stored data and the corresponding access popularity includes: for each stored data, comparing the access popularity with a preset popularity value; in response to the access popularity being greater than the preset popularity value, storing the stored data in a local cache; in response to the access popularity not being greater than the preset popularity value, storing the stored data in the cache system.

[0011] To achieve the above objectives, according to another aspect of the present invention, a data processing method is provided.

[0012] An embodiment of the present invention provides a data processing method for a caching system, comprising: receiving a time request sent by a client, obtaining a corresponding data identifier, querying the remaining expiration time corresponding to the data identifier and sending it to the client; receiving access parameters for each stored data, calculating the access popularity of each stored data based on the access parameters and storing it; receiving a popularity acquisition task from the client, obtaining the corresponding stored data, querying the associated access popularity and sending it to the client.

[0013] In another embodiment, calculating the access popularity of each piece of stored data based on the access parameters includes: obtaining the remote window lock corresponding to the stored data and updating the remaining expiration time of the remote window lock; in response to the successful update of the remote window lock, calculating the corresponding popularity value based on the access parameters to update the access popularity of the stored data; in response to the unsuccessful update of the remote window lock, calculating the corresponding popularity value based on the access parameters and adding the popularity value to the access popularity of the stored data.

[0014] To achieve the above objectives, according to another aspect of the present invention, a data processing apparatus is provided.

[0015] An embodiment of the present invention provides a data processing apparatus, disposed on a client, comprising: a receiving unit, configured to receive a data processing instruction, acquire a corresponding sliding window for each stored data, and calculate access parameters for each stored data; an updating unit, configured to, in response to a reporting instruction for access parameters, acquire a corresponding data identifier, acquire the remaining expiration time of the data identifier from a preset cache system to determine the current remaining time of the data identifier, and update the local window lock of the data identifier based on the current remaining time; a reporting unit, configured to, in response to a successful window lock update, report the access parameters associated with the data identifier to the cache system, so that the cache system calculates the access popularity corresponding to the data identifier; and an acquisition unit, configured to, in response to a popularity acquisition task being triggered, acquire the access popularity of each stored data from the cache system, and establish a mapping relationship between the data identifier of each stored data and the corresponding access popularity.

[0016] In one embodiment, the update unit is specifically configured to: obtain the remote window lock corresponding to the data identifier from the cache system, parse the remote window lock, and obtain the remaining expiration time of the data identifier; in response to the remaining expiration time being greater than a preset threshold, determine the remaining expiration time as the current remaining time; in response to the remaining expiration time not being greater than the preset threshold, obtain a preset remaining time and determine it as the current remaining time.

[0017] In another embodiment, the receiving unit is specifically configured to: for each piece of stored data, obtain the time interval of the corresponding sliding window, obtain the number of times the stored data is accessed within the time interval to determine the access parameters of the stored data corresponding to the time interval, and update the time interval of the sliding window based on a preset sliding period.

[0018] In another embodiment, the update unit is specifically configured to: obtain the state of the local window lock corresponding to the data identifier; update the time parameter in the local window lock based on the current remaining time in response to the state being a first state; and stop the update operation in response to the state being a second state.

[0019] In another embodiment, the update unit is further configured to receive a data access instruction, obtain a corresponding target data identifier, and query the access popularity associated with the target data identifier; the acquisition unit is further configured to, in response to the access popularity meeting a preset condition, obtain the data corresponding to the target data identifier from the local cache; and in response to the access popularity not meeting the preset condition, obtain the data corresponding to the target data identifier from the cache system.

[0020] In another embodiment, the acquisition unit is specifically configured to: for each piece of stored data, compare the access popularity with a preset popularity value; in response to the access popularity being greater than the preset popularity value, store the stored data in a local cache; and in response to the access popularity not being greater than the preset popularity value, store the stored data in the cache system.

[0021] To achieve the above objectives, according to another aspect of the present invention, a data processing apparatus is provided.

[0022] An embodiment of the present invention provides a data processing apparatus, configured in a caching system, comprising: a receiving unit, configured to receive a time request sent by a client, obtain a corresponding data identifier, query the remaining expiration time corresponding to the data identifier, and send it to the client; a calculation unit, configured to receive access parameters of each stored data, calculate the access popularity of each stored data according to the access parameters, and store it; and a query unit, configured to receive a popularity acquisition task from the client, obtain the corresponding stored data, query the associated access popularity, and send it to the client.

[0023] In one embodiment, the computing unit is specifically configured to: obtain the remote window lock corresponding to the stored data and update the remaining expiration time of the remote window lock; in response to the successful update of the remote window lock, calculate the corresponding popularity value according to the access parameters to update the access popularity of the stored data; in response to the unsuccessful update of the remote window lock, calculate the corresponding popularity value according to the access parameters and add the popularity value to the access popularity of the stored data.

[0024] To achieve the above objectives, according to another aspect of the present invention, an electronic device is provided.

[0025] An electronic device according to an embodiment of the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided in the embodiment of the present invention.

[0026] To achieve the above objectives, according to another aspect of the present invention, a computer-readable medium is provided.

[0027] An embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method provided in the embodiment of the present invention.

[0028] To achieve the above objectives, according to another aspect of the present invention, a computer program product is provided.

[0029] A computer program product according to an embodiment of the present invention includes a computer program that, when executed by a processor, implements the data processing method provided in the embodiment of the present invention.

[0030] One embodiment of the above invention has the following advantages or beneficial effects: In this embodiment, the client can calculate the access parameters corresponding to different times for each stored data through a sliding window and report them to a preset caching system. This allows the caching system to calculate the access popularity of each stored data based on the access parameters reported by each client and then feed it back to the client. The client can determine whether to store the data locally based on the access popularity, thereby storing highly accessed hot data locally, reducing data access to the caching system, and avoiding access bottlenecks in high-concurrency scenarios. At the same time, in this embodiment, the client reports access parameters, the caching system performs data popularity analysis, and then feeds it back to the client. Furthermore, the reporting frequency is controlled through window locking, thus eliminating the need to set up a separate server to analyze data popularity and reducing access to the caching system, reducing resource waste, and lowering operational complexity.

[0031] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0032] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of a system architecture for a data processing system according to an embodiment of the present invention; Figure 2This is a schematic diagram of a main flow of a data processing method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a sliding window according to an embodiment of the present invention; Figure 4 This is a schematic diagram of another main flow of a data processing method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of another main flow of a data processing method according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the main unit of a data processing apparatus according to an embodiment of the present invention; Figure 7 This is yet another schematic diagram of the main unit of a data processing apparatus according to an embodiment of the present invention; Figure 8 This is yet another exemplary system architecture diagram to which embodiments of the present invention can be applied; Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present invention. Detailed Implementation

[0033] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other. The acquisition, transmission, storage, use, and processing of data in this application comply with relevant national laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0035] This invention provides a data processing system that can be used to process stored data, specifically for processing stored data in a distributed environment.

[0036] In this embodiment of the invention, taking a distributed scenario as an example, it may include clients for data processing and a caching system for data caching. For example... Figure 1As shown, the client can be used for business processing and data generation. The data generated by the client can be cached locally or stored in the caching system. Therefore, in this embodiment of the invention, by analyzing the popularity of data access, the data that the client segment caches locally can be determined.

[0037] This invention provides a data processing method, which can be performed by... Figure 1 In the system architecture shown, the client executes, as follows: Figure 2 As shown, the method includes the following steps.

[0038] S201: Receive data processing instructions, obtain the corresponding sliding window for each stored data, and calculate the access parameters for each stored data.

[0039] The data processing instruction can represent an instruction to calculate access parameters for stored data. It can be a time-triggered instruction or an instruction triggered by other instructions. The data processing instruction may include an identifier of the data to be processed to indicate that this data will be processed, or it may not include a data identifier to indicate that all stored data will be processed.

[0040] Access parameters represent parameters calculated based on the access status of stored data, such as the number of accesses, access frequency, etc. In this embodiment of the invention, access parameters represent parameter values ​​calculated within a preset time period, such as the number of accesses within the preset time period, etc. The preset time period can be set according to needs, for example, it can be set to 1 second.

[0041] Specifically, in this embodiment of the invention, a sliding window can be used to determine the specific time period corresponding to the preset duration. Therefore, in this step, for each piece of stored data that needs to be processed, the time interval of its corresponding sliding window can be obtained first, and then the access data of the stored data within the time interval, such as the number of accesses, can be obtained to determine the access parameters of the stored data corresponding to the time interval. Subsequently, the time interval of the sliding window is updated based on a preset sliding period to achieve the update of the time interval corresponding to the sliding window. The sliding period can be set according to requirements, such as 200ms.

[0042] For example, with a preset duration of 1 second and a sliding period of 200 ms, the sliding window's schematic diagram can be as follows: Figure 3 As shown. Figure 3 As shown, each square represents a time window of 200 milliseconds, and the number in the square represents the number of visits within that time window. Therefore, the number of visits in the first 1-second time window is 22 (2+4+5+6+5). The sliding window slides forward in 200-ms intervals, and the number of visits in the second 1-second time window is 120 (4+5+6+5+100), and so on.

[0043] S202: In response to the reporting instruction of the access parameters, obtain the corresponding data identifier, obtain the remaining expiration time of the data identifier from the preset cache system, determine the current remaining time of the data identifier, and update the local window lock of the data identifier based on the current remaining time.

[0044] The reporting instruction can be triggered based on a preset period or after the access parameters are calculated in step S201. The reporting instruction may include a data identifier to indicate which stored data's access parameters are being reported. To avoid frequent reporting of access parameters, in this embodiment, the cache system can set an expiration time to determine the reporting time. In this step, when reporting, the expiration time of the data identifier can first be obtained from the cache system, representing the remaining time for this round of reporting. This time can then be used to determine the client's current remaining time, allowing reporting to be performed within the current remaining time.

[0045] In this embodiment of the invention, to ensure that only one report is made within each round of reporting events, a window lock can be set. Specifically, the window lock can be a distributed lock. The remaining time is set by the window lock, and the state of the window lock is used to determine whether reporting can be performed. Therefore, in this step, after determining the current remaining time, the local window lock of the data identifier can be updated based on the current remaining time to determine whether the reporting operation can be performed.

[0046] Specifically, determining the current remaining time of the data identifier in this step can be performed as follows: obtain the remote window lock corresponding to the data identifier from the cache system, parse the remote window lock to obtain the remaining expiration time of the data identifier; in response to the remaining expiration time being greater than a preset threshold, determine the remaining expiration time as the current remaining time; in response to the remaining expiration time not being greater than the preset threshold, obtain the preset remaining time and determine it as the current remaining time.

[0047] The caching system also sets a corresponding window lock, i.e., a remote window lock, for each data identifier. This window lock stores the remaining expiration time, which automatically decreases over time. Therefore, after acquiring the remote window lock, the remaining expiration time can be parsed. If the remaining expiration time is greater than a preset threshold, it means the current time is still within the reporting time range for this round, so the remaining expiration time can be determined as the current remaining time. If the remaining expiration time is not greater than the preset threshold, it means the current time is not within the reporting time range for this round, so a preset remaining time can be obtained and determined as the current remaining time, i.e., the current remaining time is set as the reporting time for the next round. The preset threshold and preset remaining time can be set based on requirements; for example, the preset remaining time can be set to 200ms, and the preset threshold can be set to 0.

[0048] In one implementation, the step of updating the local window lock of the data identifier can be specifically performed as follows: obtaining the state of the local window lock corresponding to the data identifier; in response to the state being a first state, updating the time parameter in the local window lock based on the current remaining time; in response to the state being a second state, stopping the update operation.

[0049] The local window lock state can include a first state and a second state. The second state indicates a locked state, in which it cannot be relocked, so updating it will also fail. The first state indicates an unlocked state, in which it can be relocked, so updating it will also succeed. Therefore, in this step, subsequent operations can be performed based on the local window lock state.

[0050] It should be noted that a successful update of the local window lock indicates a successful relock, and parameters indicating successful locking are usually returned. Therefore, these parameters can be used to determine whether the local window lock has been updated successfully.

[0051] S203: In response to a successful local window lock update, the access parameters associated with the data identifier are reported to the cache system.

[0052] After a successful local window lock update, the corresponding access parameter reporting can be executed, that is, the access parameters associated with the data identifier can be reported to the caching system so that the caching system can calculate the access frequency corresponding to the data identifier. If it is determined that the local window lock update is unsuccessful, the reporting operation can be stopped, and the system can wait for the next access parameter reporting instruction.

[0053] S204: In response to the triggering of the heat acquisition task, retrieve the access heat of each stored data from the cache system and establish a mapping relationship between the data identifier of each stored data and the corresponding access heat.

[0054] The popularity acquisition task can be a timed task used to periodically acquire the access popularity of each stored data. Clients can then use this information to identify high-access data and perform local storage and access. A mapping relationship is established between the data identifier of each stored data and its corresponding access popularity to facilitate querying and other operations during data access.

[0055] In another implementation, after receiving a data access instruction, the client can first obtain the corresponding target data identifier and query the access popularity associated with the target data identifier. If the access popularity meets a preset condition, the data corresponding to the target data identifier can be retrieved from the local cache. If the access popularity does not meet the preset condition, the data corresponding to the target data identifier can be retrieved from the cache system. The preset condition can be pre-set, for example, the access popularity can be greater than a preset value.

[0056] In this embodiment of the invention, the client can calculate access parameters for each stored data at different times using a sliding window and report them to the caching system. This allows the caching system to calculate the access frequency of each stored data based on the access parameters reported by each client and then feed this information back to the client. The client can then determine whether to store the data locally based on the access frequency, thereby storing frequently accessed data locally, reducing data access to the caching system, and avoiding access bottlenecks in high-concurrency scenarios. Furthermore, in this embodiment, by having the client report access parameters, the caching system performs data frequency analysis and feeds it back to the client. A window lock is used to control the reporting frequency, eliminating the need for a separate server to analyze data frequency, reducing access to the caching system, minimizing resource waste, and lowering operational complexity.

[0057] This invention provides a data processing method, which can be performed by... Figure 1 In the system architecture shown, the caching system executes, such as... Figure 4 As shown, the method includes the following steps.

[0058] S401: Receive the time request sent by the client, obtain the corresponding data identifier, query the remaining expiration time corresponding to the data identifier, and send it to the client.

[0059] The time request sent by the client is used to request the remaining expiration time of the cached data. In this step, the remaining expiration time can be queried according to the data identifier and returned to the client.

[0060] Specifically, in this step, the information of the corresponding remote window lock can be queried based on the data identifier and returned to the client so that the client can parse out the remaining expiration time. Alternatively, the information of the corresponding remote window lock can be queried directly, the remaining expiration time can be parsed out, and then returned to the client.

[0061] S402: Receive access parameters for each stored data, calculate the access frequency of each stored data based on the access parameters, and store the data.

[0062] The access parameters for each stored data are reported by the client. In this step, after obtaining the access parameters of the stored data, the access frequency of the stored data can be calculated.

[0063] Taking the number of visits within 1 second as an example, the access popularity can be the sum of the access parameters reported by each client within the same reporting time period.

[0064] In this step, when calculating access popularity, the remote window lock corresponding to the stored data can be obtained first to update the remaining expiration time of the remote window lock. If the remote window lock is updated successfully, it means that the previous time period for calculating access popularity has ended, and the access parameters reported this time belong to the next time period. In this case, the current access popularity can be reset, and the corresponding popularity value can be calculated based on the access parameters to update the access popularity of the stored data. If the remote window lock update fails, it means that the previous time period for calculating access popularity has not ended, and the access parameters reported this time belong to that time period. In this case, the corresponding popularity value can be calculated based on the access parameters, and the popularity value can be added to the access popularity of the stored data.

[0065] S403: Receive the client's popularity acquisition task, obtain the corresponding stored data, query the associated access popularity and send it to the client.

[0066] The client can periodically send a task to the caching system to retrieve the popularity data. After the caching system retrieves the corresponding stored data, it can query the associated access popularity and send it to the client.

[0067] In this embodiment of the invention, the client can calculate access parameters for each stored data at different times using a sliding window and report them to the caching system. This allows the caching system to calculate the access frequency of each stored data based on the access parameters reported by each client and then feed this information back to the client. The client can then determine whether to store the data locally based on the access frequency, thereby storing frequently accessed data locally, reducing data access to the caching system, and avoiding access bottlenecks in high-concurrency scenarios. Furthermore, in this embodiment, by having the client report access parameters, the caching system performs data frequency analysis and feeds it back to the client. A window lock is used to control the reporting frequency, eliminating the need for a separate server to analyze data frequency, reducing access to the caching system, minimizing resource waste, and lowering operational complexity.

[0068] The following is combined Figure 1 The system architecture shown illustrates the data processing methods in this embodiment of the invention, such as... Figure 5 As shown, the method includes the following steps.

[0069] S501: The client receives the data processing instruction, obtains the corresponding sliding window for each stored data, and calculates the access parameters for each stored data.

[0070] S502: In response to the reporting instruction for access parameters, the client obtains the corresponding data identifier and sends a time request to the caching system.

[0071] S503: The caching system obtains the corresponding data identifier, queries the remaining expiration time corresponding to the data identifier, and sends it to the client.

[0072] S504: If the remaining time after expiration is greater than a preset threshold, the client determines the remaining time after expiration as the current remaining time; if the remaining time after expiration is not greater than the preset threshold, the client obtains the preset remaining time and determines it as the current remaining time.

[0073] S505: The client updates the local window lock based on the current remaining time.

[0074] S506: The client responds to the successful update of the local window lock by reporting the access parameters associated with the data identifier to the cache system.

[0075] S507: The caching system obtains the corresponding remote window lock and updates the remaining expiration time of the remote window lock according to the preset remaining time.

[0076] S508: When the remote window lock update is successful, the caching system calculates the corresponding heat value based on the access parameters and updates it to the access heat of the stored data; when the remote window lock update is unsuccessful, the system calculates the corresponding heat value based on the access parameters and adds the heat value to the access heat of the stored data.

[0077] S509: When the hotness acquisition task is triggered, the client retrieves the access hotness of each stored data from the preset cache system and establishes a mapping relationship between the data identifier of each stored data and the corresponding access hotness.

[0078] It should be noted that the data processing principle in the embodiments of the present invention is the same as... Figure 2 and Figure 4 The corresponding data processing principle is the same, so it will not be repeated here.

[0079] To address the problems existing in the prior art, embodiments of the present invention provide a data processing device 600, configured on a client side, such as... Figure 6 As shown, the device 600 includes: a receiving unit 601, configured to receive data processing instructions, obtain the corresponding sliding window of each stored data, and calculate the access parameters of each stored data; an updating unit 602, configured to, in response to a reporting instruction for access parameters, obtain the corresponding data identifier, obtain the remaining expiration time of the data identifier from the cache system to determine the current remaining time of the data identifier, and update the local window lock of the data identifier based on the current remaining time; a reporting unit 603, configured to, in response to a successful window lock update, report the access parameters associated with the data identifier to the cache system, so that the cache system can calculate the access popularity corresponding to the data identifier; and an obtaining unit 604, configured to, in response to a popularity obtaining task being triggered, obtain the access popularity of each stored data from the cache system, and establish a mapping relationship between the data identifier of each stored data and the corresponding access popularity.

[0080] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the implementation method. Figure 2 The methods used in the illustrated embodiments are the same and will not be repeated here.

[0081] In one embodiment, the update unit 602 is specifically configured to: obtain the remote window lock corresponding to the data identifier from the cache system, parse the remote window lock, and obtain the remaining expiration time of the data identifier; in response to the remaining expiration time being greater than a preset threshold, determine the remaining expiration time as the current remaining time; in response to the remaining expiration time not being greater than the preset threshold, obtain a preset remaining time and determine it as the current remaining time.

[0082] In another embodiment, the receiving unit 601 is specifically configured to: for each piece of stored data, obtain the time interval of the corresponding sliding window, obtain the number of times the stored data is accessed within the time interval to determine the access parameters of the stored data corresponding to the time interval, and update the time interval of the sliding window based on a preset sliding period.

[0083] In another embodiment, the update unit 602 is specifically configured to: obtain the state of the local window lock corresponding to the data identifier; update the time parameter in the local window lock based on the current remaining time in response to the state being a first state; and stop the update operation in response to the state being a second state.

[0084] In another embodiment, the receiving unit 601 is further configured to receive a data access instruction, obtain a corresponding target data identifier, and query the access popularity associated with the target data identifier; the obtaining unit 604 is further configured to, in response to the access popularity meeting a preset condition, obtain the data corresponding to the target data identifier from the local cache; and in response to the access popularity not meeting the preset condition, obtain the data corresponding to the target data identifier from the cache system.

[0085] In another embodiment, the acquisition unit 604 is specifically configured to: compare the access popularity with a preset popularity value for each piece of stored data; in response to the access popularity being greater than the preset popularity value, store the stored data in a local cache; and in response to the access popularity not being greater than the preset popularity value, store the stored data in the cache system.

[0086] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the implementation method. Figure 2 , 5 The methods used in the illustrated embodiments are the same and will not be repeated here.

[0087] In this embodiment of the invention, the client can calculate access parameters for each stored data at different times using a sliding window and report them to the caching system. This allows the caching system to calculate the access frequency of each stored data based on the access parameters reported by each client and then feed this information back to the client. The client can then determine whether to store the data locally based on the access frequency, thereby storing frequently accessed data locally, reducing data access to the caching system, and avoiding access bottlenecks in high-concurrency scenarios. Furthermore, in this embodiment, by having the client report access parameters, the caching system performs data frequency analysis and feeds it back to the client. A window lock is used to control the reporting frequency, eliminating the need for a separate server to analyze data frequency, reducing access to the caching system, minimizing resource waste, and lowering operational complexity.

[0088] To address the problems existing in the prior art, embodiments of the present invention provide a data processing apparatus 700, configured in a cache system, such as... Figure 7 As shown, the device 700 includes: a receiving unit 701, configured to receive a time request sent by the client, obtain the corresponding data identifier, query the remaining expiration time corresponding to the data identifier, and send it to the client; a calculation unit 702, configured to receive access parameters of each stored data, calculate the access popularity of each stored data according to the access parameters, and store it; and a query unit 703, configured to receive the popularity acquisition task from the client, obtain the corresponding stored data, query the associated access popularity, and send it to the client.

[0089] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the implementation method. Figure 4 The methods used in the illustrated embodiments are the same and will not be repeated here.

[0090] In one embodiment, the calculation unit 702 is specifically configured to: obtain the remote window lock corresponding to the stored data and update the remaining expiration time of the remote window lock; in response to the successful update of the remote window lock, calculate the corresponding popularity value according to the access parameters to update the access popularity of the stored data; in response to the unsuccessful update of the remote window lock, calculate the corresponding popularity value according to the access parameters and add the popularity value to the access popularity of the stored data.

[0091] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the implementation method. Figure 4 , 5 The methods used in the illustrated embodiments are the same and will not be repeated here.

[0092] In this embodiment of the invention, the client can calculate access parameters for each stored data at different times using a sliding window and report them to the caching system. This allows the caching system to calculate the access frequency of each stored data based on the access parameters reported by each client and then feed this information back to the client. The client can then determine whether to store the data locally based on the access frequency, thereby storing frequently accessed data locally, reducing data access to the caching system, and avoiding access bottlenecks in high-concurrency scenarios. Furthermore, in this embodiment, by having the client report access parameters, the caching system performs data frequency analysis and feeds it back to the client. A window lock is used to control the reporting frequency, eliminating the need for a separate server to analyze data frequency, reducing access to the caching system, minimizing resource waste, and lowering operational complexity.

[0093] According to embodiments of the present invention, an electronic device and a readable storage medium are also provided.

[0094] An electronic device according to an embodiment of the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the data processing method provided in the embodiment of the present invention.

[0095] Figure 8 An exemplary system architecture 800 is shown, in which a data processing method or data processing apparatus to which embodiments of the present invention can be applied is illustrated.

[0096] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, and 803, a network 804, and a server 805. Network 804 serves as the medium for providing communication links between terminal devices 801, 802, and 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0097] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various client applications can be installed on terminal devices 801, 802, and 803.

[0098] Terminal devices 801, 802, and 803 can be, but are not limited to, smartphones, tablets, laptops, and desktop computers, etc.

[0099] Server 805 can be a server that provides various services. The server can analyze and process data such as received product information query requests, and feed back the processing results (such as product information - just an example) to the terminal device.

[0100] It should be noted that the data processing method provided in the embodiments of the present invention is generally executed by the server 805, and correspondingly, the data processing device is generally located in the server 805.

[0101] It should be understood that Figure 8 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0102] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 900 suitable for implementing embodiments of the present invention. Figure 9 The computer system shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0103] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0104] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0105] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of this invention.

[0106] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a unit, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] The units described in the embodiments of the present invention can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a receiving unit, an updating unit, a reporting unit, and an acquiring unit. The names of these units do not necessarily limit the specific unit; for example, a receiving unit can also be described as a "unit with receiving functions."

[0109] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to perform the data processing method provided by the present invention.

[0110] In another aspect, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in the embodiments of the present invention.

[0111] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, For client use, including: Receive data processing instructions, obtain the corresponding sliding window for each stored data, and calculate the access parameters for each stored data; In response to the reporting instruction of access parameters, the corresponding data identifier is obtained, the expiration time of the data identifier is obtained from the preset cache system to determine the current remaining time of the data identifier, and the local window lock of the data identifier is updated based on the current remaining time. In response to the successful update of the local window lock, the access parameters associated with the data identifier are reported to the caching system so that the caching system can calculate the access frequency corresponding to the data identifier; In response to the triggering of the heat acquisition task, the access heat of each of the stored data is obtained from the cache system, and a mapping relationship is established between the data identifier of each of the stored data and the corresponding access heat.

2. The method according to claim 1, characterized in that, Obtaining the remaining expiration time of the data identifier from the caching system to determine the current remaining time of the data identifier includes: Retrieve the remote window lock corresponding to the data identifier from the cache system, parse the remote window lock, and obtain the remaining expiration time of the data identifier; In response to the remaining time after expiration being greater than a preset threshold, the remaining time after expiration is determined as the current remaining time; in response to the remaining time after expiration not being greater than the preset threshold, a preset remaining time is obtained and determined as the current remaining time.

3. The method according to claim 1, characterized in that, Obtain the corresponding sliding window for each stored data to calculate the access parameters for each stored data, including: For each piece of stored data, obtain the time interval of the corresponding sliding window, obtain the number of times the stored data is accessed within the time interval, and determine the access parameter of the stored data corresponding to the time interval. Then, update the time interval of the sliding window based on the preset sliding period.

4. The method according to claim 1, characterized in that, Updating the local window lock of the data identifier based on the current remaining time includes: Obtain the state of the local window lock corresponding to the data identifier; In response to the state being the first state, the time parameter in the local window lock is updated based on the current remaining time; in response to the state being the second state, the update operation is stopped.

5. The method according to claim 1, characterized in that, Establish a mapping relationship between the data identifiers of each stored data item and their corresponding access popularity, including: For each piece of stored data, the access popularity is compared with a preset popularity value; in response to the access popularity being greater than the preset popularity value, the stored data is stored in the local cache; in response to the access popularity not being greater than the preset popularity value, the stored data is stored in the cache system.

6. A data processing method, characterized in that, For use in caching systems, including: Receive a time request sent by the client, obtain the corresponding data identifier, query the remaining expiration time corresponding to the data identifier, and send it to the client; Receive access parameters for each stored data, calculate the access frequency of each stored data based on the access parameters, and store the data. The system receives the popularity acquisition task from the client, obtains the corresponding stored data, queries the associated access popularity, and sends it to the client.

7. The method according to claim 6, characterized in that, Calculating the access frequency of each of the stored data based on the access parameters includes: Obtain the remote window lock corresponding to the stored data, and update the remaining expiration time of the remote window lock; In response to a successful remote window lock update, a corresponding heat value is calculated based on the access parameters to update the access heat of the stored data; in response to a failed remote window lock update, a corresponding heat value is calculated based on the access parameters and added to the access heat of the stored data.

8. A data processing apparatus, characterized in that, Settings on the client side include: The receiving unit is used to receive data processing instructions, obtain the corresponding sliding window of each stored data, and calculate the access parameters of each stored data. The update unit is used to respond to the reporting instruction of the access parameters, obtain the corresponding data identifier, obtain the expiration time of the data identifier from the cache system, determine the current remaining time of the data identifier, and update the local window lock of the data identifier based on the current remaining time. The reporting unit is used to report the access parameters associated with the data identifier to the caching system in response to the successful update of the window lock, so that the caching system can calculate the access popularity corresponding to the data identifier; The acquisition unit is used to acquire the access popularity of each of the stored data from the cache system in response to the triggering of the popularity acquisition task, and to establish a mapping relationship between the data identifier of each of the stored data and the corresponding access popularity.

9. A data processing apparatus, characterized in that, Configured in the caching system, including: The receiving unit is configured to receive a time request sent by the client, obtain the corresponding data identifier, query the remaining expiration time corresponding to the data identifier, and send it to the client. A computing unit is used to receive access parameters for each stored data, calculate the access frequency of each stored data according to the access parameters, and store the data. The query unit is used to receive the popularity acquisition task from the client, obtain the corresponding stored data, query the associated access popularity, and send it to the client.

10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images