A data processing method based on a cache system, an electronic device, and a storage medium

Abstract

This invention provides a data processing method, electronic device, and storage medium based on a caching system. The method includes: when a data repository receives service data and its corresponding validity period, determining the storage priority of each unexpired data queue based on the predicted data increment of the data repository within a target time period and the remaining data volume of several unexpired data queues within the target time period; storing the service data and validity period sent by the user server into the unexpired data queue with the highest storage priority; the user server queries the validity period in each unexpired data queue, and if the validity period is less than or equal to the current time, retrieves the corresponding service data from the unexpired data queue and stores it in the expired data queue of the data repository, so that the stored data volume of service data in any unexpired data queue does not exceed a preset storage data volume threshold, thereby reducing data processing latency.

Description

Technical Field

[0001] This invention relates to the field of service data storage technology, and in particular to a data processing method, electronic device and storage medium based on a caching system. Background Technology

[0002] Service data from different servers typically has a validity period, meaning it's only valid for operations on that server within that timeframe. For example, in airline ticket purchases, after a user clicks the "buy" button, they must complete the payment within the corresponding validity period. If this period expires, the purchase is considered abandoned. Therefore, to store service data from different servers, a data repository is usually used to store the generated service data and monitor its validity period. Currently, the method for storing service data is to directly store the server-generated service data in the data queue of the data repository. However, in practice, it has been found that the larger the amount of service data stored in the data queue, the greater the latency in data processing. Furthermore, when the amount of service data stored in the data queue exceeds a certain threshold, the latency in data processing suddenly increases. This situation increases the data processing burden on the server and reduces the accuracy of validity period monitoring. Summary of the Invention

[0003] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows: According to one aspect of this application, a data processing method based on a caching system is provided, comprising: Step S100: When the data repository receives the service data sent by the user server and the preset valid time corresponding to the service data, the storage priority corresponding to each unexpired data queue is determined according to the predicted data increment corresponding to the data repository within the target time period and the remaining data volume corresponding to several unexpired data queues of the data repository within the target time period; the start time of the target time period is the current time, and the duration of the target time period is the preset duration. Step S200: Store the service data sent by the user server and the corresponding validity period of the service data in the queue of unexpired data with the highest storage priority; Step S300: Every preset time interval, the user server queries the valid time corresponding to each service data in each unexpired data queue. If the valid time corresponding to any service data is less than or equal to the current time, the service data is taken out from the unexpired data queue and stored in the expired data queue of the data storage repository. Step S400: The user server obtains each service data in the expired data queue in real time, and returns the service termination information corresponding to the service data to the business server corresponding to the service data according to the service data in the expired data queue.

[0004] In one exemplary embodiment of this application, step S100 includes: Step S110: After the data storage repository receives service data sent by the user server and the preset validity period corresponding to the service data, obtain the amount of stored data corresponding to each unexpired data queue at the current time to obtain a first list of stored data amounts B=(B1,B2,...,B...). i ,...,B m ); where i = 1, 2, ..., m; m is the number of data queues that have not yet expired; B i Let i be the amount of data stored in the i-th unexpired data queue at the current time. Step S120: Traverse the list of stored data volumes B. If B 02 ≤B i ≤B 01 Then the i-th unexpired data queue is determined as the first data queue; where B 01 B is the preset first data storage threshold; 02 B is the preset second data storage threshold; 02 <B 01 ; Step S130: Obtain the predicted data increment corresponding to each data processing time point within the target time period from the data repository, to obtain the predicted data increment list A=(A1,A2,...,A1). j ,...,A k ); where j=1,2,...,k; k is the number of data processing moments included in the target time period; A j This represents the predicted data increment of the data repository at the j-th data processing time within the target time period; Step S140: Obtain the predicted data expiration amount corresponding to each data processing time point of each first data queue within the target time period, to obtain several lists of predicted data expiration amounts C1, C2, ..., C p ,...,C q Where p = 1, 2, ..., q; q is the number of items in the first data queue; C p This is a list of predicted data due dates for the p-th first data queue within the target time period. C p =(C p1 C p2 ,...,C pj ,...,Cpk );C pj This represents the predicted data expiration amount corresponding to the j-th data processing time within the target time period for the p-th first data queue. Step S150: Based on a list of predicted amounts C1, C2, ..., C... p ,...,C q Based on the predicted data increment list A, determine the remaining data volume of several first data queues at each data processing time within the target time period; Step S160: If the amount of remaining data corresponding to each data processing moment in all first data queues within the target time period is greater than zero, then the storage priority corresponding to each first data queue is determined as the first priority, and the storage priority corresponding to the remaining unexpired data queues in the data storage repository is determined as the second priority; wherein, the first priority is higher than the second priority.

[0005] In one exemplary embodiment of this application, step S150 includes: Step S151: Obtain the amount of stored data for each first data queue at the current time to obtain a second list of stored data, D=(D1,D2,...,D...). p ,...,D q ); where D p Let p be the amount of data stored in the first data queue at the current time. Step S152: Determine the remaining data volume of several first data queues at each data processing time within the target time period to obtain a list of remaining data volumes E=(E1,E2,...,E...). j ,...,E k ); where E j This represents the remaining data volume of several first data queues at the j-th data processing time within the target time period; E j =Z-(∑ q p=1 D p -∑ j t=1 ∑ q p=1 C pt +∑ j t=1 A t Z represents the preset total amount of data in the unexpired data queue.

[0006] In one exemplary embodiment of this application, step S160 further includes: Step S161, if E jIf ≤0, then the unexpired data queues corresponding to the storage data quantities less than MIN(D) in the first storage data quantity list B are determined as the first data queues, and the process returns to step S140; where MIN() is a preset minimum value determination function.

[0007] In one exemplary embodiment of this application, step S120 further includes: Step S121: If the amount of stored data corresponding to each unexpired data queue at the current time is greater than B... 01 If so, a new queue of unexpired data will be created in the data repository; Step S122: Determine the storage priority of the newly created unexpired data queue as the first priority, and determine the storage priority of the remaining unexpired data queues in the data storage repository as the second priority; wherein, the first priority is higher than the second priority.

[0008] In one exemplary embodiment of this application, the predicted data increment corresponding to each data processing moment within a target time period in the data repository is determined through the following steps: Step S131: Obtain the historical data increment corresponding to each data processing moment within the historical time period from the data repository, to obtain the historical data increment list F=(F1,F2,...,F...). j ,...,F k ); where the duration of the historical time period is equal to the duration of the target time period, the end time of the historical time period is the start time of the target time period, and the duration between any two adjacent data processing moments in the historical time period is equal to the duration between any two adjacent data processing moments in the target time period; F j This represents the historical data increment at the j-th data processing moment within the historical time period of the data repository. Step S132: Input the historical data increment list F into the preset data increment prediction model to obtain the predicted data increment corresponding to each data processing time in the data repository output by the data increment prediction model within the target time period; the data increment prediction model is obtained by supervised training on several historical data increments.

[0009] In one exemplary embodiment of this application, the predicted data expiration amount corresponding to the j-th data processing time within the target time period for any first data queue is determined through the following steps: Step S141: Obtain the valid time corresponding to each service data stored in the first data queue to obtain a valid time list G=(G1,G2,...,G...). r ,...,G s ); where r = 1, 2, ..., s; s is the number of service data stored in the first data queue; Gr The valid time corresponding to the r-th service data stored in the first data queue; Step S142: Traverse the list of valid times G. If G r The time corresponding to the j-th data processing moment in the target time period is less than or equal to G. r If the time is greater than the time corresponding to the (j-1)th data processing time of the target time period, then the rth service data stored in the first data queue will be determined as the predicted expiration data corresponding to the jth data processing time. Step S143: Determine the number of predicted due data pairs corresponding to the j-th data processing time stored in the first data queue as the predicted data due amount corresponding to the j-th data processing time in the first data queue within the target time period.

[0010] In one exemplary embodiment of this application, step S200 further includes: Step S210: If the number of unexpired data queues with storage priority of first priority is greater than 1, then the unexpired data queue with the least amount of stored data among the unexpired data queues with storage priority of first priority is determined as the target unexpired data queue. Step S220: Store the service data sent by the user server and the corresponding validity period of the service data into the target unexpired data queue.

[0011] According to another aspect of this application, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, the at least one instruction or the at least one program being loaded and executed by a processor to implement the aforementioned data processing method based on a cache system.

[0012] According to another aspect of this application, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0013] The present invention has at least the following beneficial effects: In the data processing method based on a caching system of the present invention, when the data repository receives service data and the corresponding validity period of the service data sent by the user server, it determines the storage priority of each unexpired data queue based on the predicted data increment of the data repository within the target time period and the remaining data volume of several unexpired data queues of the data repository within the target time period. The service data sent by the user server and the corresponding validity period are then stored in the unexpired data queue with the highest storage priority, so that the storage volume of service data in the unexpired data queue tends to be balanced. Every preset time interval, the user server queries the validity period of each service data in each unexpired data queue. If the validity period of any service data is less than or equal to the current time, the service data is retrieved from the unexpired data queue and stored in the expired data queue of the data repository. Furthermore, the user server obtains each service data in the expired data queue in real time and returns the service termination information corresponding to the service data to the business server corresponding to the service data based on the service data in the expired data queue. By determining the storage priority of several unexpired data queues, the unexpired data queue for storing storage data is selected to ensure that the amount of service data stored in any unexpired data queue does not exceed the preset storage data amount threshold, thereby reducing data processing latency. As the validity period of service data in unexpired data queues with lower storage priority expires, the expired service data will be transferred from the unexpired data queue to the expired data queue. This facilitates the user server's retrieval of expired service data, reduces the overall data storage volume of the unexpired data queue, reduces the number of times the user server calls the unexpired data queue, and further reduces the data processing time. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of a data processing method based on a caching system provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This application proposes a data processing method based on a caching system, such as... Figure 1 As shown, it includes: Step S100: When the data repository receives the service data sent by the user server and the preset validity period corresponding to the service data, the storage priority corresponding to each unexpired data queue is determined according to the predicted data increment of the data repository within the target time period and the remaining data volume of several unexpired data queues of the data repository within the target time period. Service data can be data sent by the user server with a valid time limit, such as user ticket payment service data. Typically, user ticket payment services will set a valid time limit, and users need to make payments within this valid time limit. Once this valid time limit is exceeded, the corresponding ticket payment service will end.

[0018] The unexpired data queue in this application refers to the queue of service data that has not yet expired. Since the more data stored in the data queue, the greater the data latency, when it exceeds a certain threshold, the resulting data latency will increase exponentially. Therefore, in order to reduce data latency, it is necessary to control the amount of data stored in each unexpired data queue so that the amount of data stored and the resulting data latency are kept within a relatively balanced range. Therefore, it is necessary to determine the storage priority of each unexpired data queue.

[0019] Storage priority refers to the priority of storage service data in each unexpired data queue. The higher the storage priority, the greater the storage demand of the corresponding unexpired data queue. When a new service data is received, it is first stored in the unexpired data queue with the higher storage priority.

[0020] The start time of the target time period is the current time, and the duration of the target time period is the preset duration.

[0021] Furthermore, step S100 includes steps S110-S161: Step S110: After the data storage repository receives service data sent by the user server and the preset validity period corresponding to the service data, obtain the amount of stored data corresponding to each unexpired data queue at the current time to obtain a first list of stored data amounts B=(B1,B2,...,B...). i,...,B m ); where i = 1, 2, ..., m; m is the number of data queues that have not yet expired; B i Let i be the amount of data stored in the i-th unexpired data queue at the current time. Step S120: Traverse the list of stored data volumes B. If B 02 ≤B i ≤B 01 Then the i-th unexpired data queue is determined as the first data queue; Among them, B 01 B is the preset first data storage threshold; 02 B is the preset second data storage threshold; 02 <B 01 ; In addition, step S120 also includes steps S121-S122: Step S121: If the amount of stored data corresponding to each unexpired data queue at the current time is greater than B... 01 If so, a new queue of unexpired data will be created in the data repository; Step S122: Determine the storage priority of the newly created unexpired data queue as the first priority, and determine the storage priority of the remaining unexpired data queues in the data storage repository as the second priority; The first priority is higher than the second priority.

[0022] If the amount of stored data in each unexpired data queue at the current time is greater than the preset first storage data threshold, it means that the amount of data stored in all unexpired data queues has exceeded the maximum storage threshold. If newly received service data is stored in these unexpired data queues, it will cause high data latency. Therefore, a new unexpired data queue needs to be established to store the newly received service data.

[0023] Step S130: Obtain the predicted data increment corresponding to each data processing time point within the target time period from the data repository, to obtain the predicted data increment list A=(A1,A2,...,A1). j ,...,A k ); where j=1,2,...,k; k is the number of data processing moments included in the target time period; A j This represents the predicted data increment of the data repository at the j-th data processing time within the target time period; Predicted data increment refers to the predicted increase in data reception and storage in the data repository within a target time period.

[0024] The predicted data increment at each data processing time point within the target time period is determined through steps S131-S132: Step S131: Obtain the historical data increment corresponding to each data processing moment within the historical time period from the data repository, to obtain the historical data increment list F=(F1,F2,...,F...). j ,...,F k ); where the duration of the historical time period is equal to the duration of the target time period, the end time of the historical time period is the start time of the target time period, and the duration between any two adjacent data processing moments in the historical time period is equal to the duration between any two adjacent data processing moments in the target time period; F j This represents the historical data increment at the j-th data processing moment within the historical time period of the data repository. Step S132: Input the historical data increment list F into the preset data increment prediction model to obtain the predicted data increment corresponding to each data processing time in the data repository output by the data increment prediction model within the target time period. The data increment prediction model is obtained by supervised training on samples of several historical data increments. The data increment prediction model can be a neural network model. The training samples used are the actual historical data increments of the other data repositories at each data processing moment within a historical time period and the actual target data increments of the data repository in the target time period after that historical time period. For example, the historical data increments of the first data repository and the second data repository at each data processing moment within any historical time period can be used as input samples, and the target data increments of the first data repository and the second data repository at each data processing moment in the target time period after that historical time period can be used as output labels. The neural network model is then trained in a supervised manner to obtain the data increment prediction model. Alternatively, other training methods can also be used for the sample training of the data increment prediction model.

[0025] Step S140: Obtain the predicted data expiration amount corresponding to each data processing time point of each first data queue within the target time period, to obtain several lists of predicted data expiration amounts C1, C2, ..., C p ,...,C q Where p = 1, 2, ..., q; q is the number of items in the first data queue; C p This is a list of predicted data due dates for the p-th first data queue within the target time period. The predicted data expiration amount is the number of service data that the first data queue predicts will expire within the target time period.

[0026] C p =(Cp1 C p2 ,...,C pj ,...,C pk );C pj This represents the predicted data expiration amount corresponding to the j-th data processing time within the target time period for the p-th first data queue. The predicted data expiration amount corresponding to the j-th data processing time within the target time period for any first data queue is determined through steps S141-S143: Step S141: Obtain the valid time corresponding to each service data stored in the first data queue to obtain a valid time list G=(G1,G2,...,G...). r ,...,G s ); where r = 1, 2, ..., s; s is the number of service data stored in the first data queue; G r The valid time corresponding to the r-th service data stored in the first data queue; Step S142: Traverse the list of valid times G. If G r The time corresponding to the j-th data processing moment in the target time period is less than or equal to G. r If the time is greater than the time corresponding to the (j-1)th data processing time of the target time period, then the rth service data stored in the first data queue will be determined as the predicted expiration data corresponding to the jth data processing time. Step S143: Determine the number of predicted due data pairs corresponding to the j-th data processing time stored in the first data queue as the predicted data due amount corresponding to the j-th data processing time in the first data queue within the target time period.

[0027] Step S150: Based on a list of predicted amounts C1, C2, ..., C... p ,...,C q Based on the predicted data increment list A, determine the remaining data volume of several first data queues at each data processing time within the target time period; Step S150 includes steps S151-S152: Step S151: Obtain the amount of stored data for each first data queue at the current time to obtain a second list of stored data, D=(D1,D2,...,D...). p ,...,D q ); where D p Let p be the amount of data stored in the first data queue at the current time. Step S152: Determine the remaining data volume of several first data queues at each data processing time within the target time period to obtain a list of remaining data volumes E=(E1,E2,...,E...). j ,...,E k ); where E j This represents the remaining data volume of several first data queues at the j-th data processing time within the target time period; E j =Z-(∑ q p=1 D p -∑ j t=1 ∑ q p=1 C pt +∑ j t=1 A t Z represents the preset total amount of data in the unexpired data queue.

[0028] Since each service data in the unexpired data queue has a corresponding validity period, the corresponding service data needs to be cleared from the unexpired data queue when the validity period expires. Therefore, by determining the predicted amount of data expiring and the predicted data increment in each first data queue, the remaining amount of data in each first data queue is determined. Then, the storage priority of the corresponding unexpired data queue is determined by the remaining amount of data, so as to ensure that the storage priority of each first data queue takes into account the dynamic validity period of the service data, making the obtained storage priority more accurate.

[0029] Step S160: If the remaining data volume corresponding to each data processing moment of all first data queues within the target time period is greater than zero, then the storage priority corresponding to each first data queue is determined as the first priority, the storage priority corresponding to the remaining unexpired data queues in the data storage repository is determined as the second priority, and step S200 is executed. Step S161, if E j If ≤0, then the unexpired data queues corresponding to the storage data quantities that are only less than MIN(D) in the first storage data quantity list B are determined as the first data queues, and the process returns to step S140; MIN() is a preset function for determining the minimum value.

[0030] Step S200: Store the service data sent by the user server and the corresponding validity period of the service data in the queue of unexpired data with the highest storage priority; Furthermore, step S200 also includes steps S210-S220: Step S210: If the number of unexpired data queues with storage priority of first priority is greater than 1, then the unexpired data queue with the least amount of stored data among the unexpired data queues with storage priority of first priority is determined as the target unexpired data queue. The queue of unexpired data with the highest storage priority has the greatest storage demand within the target time period. If there is only one queue of unexpired data with the highest storage priority, newly received service data will only be stored in that queue. If there is more than one queue of unexpired data with the highest storage priority, it means that there are multiple queues of unexpired data with the greatest storage demand. In order to further reduce data latency, it is necessary to balance and adjust the storage volume of multiple queues of unexpired data with the highest priority so that the data latency of the queues of unexpired data with the highest priority is kept within a stable range. Therefore, when new service data is received, the service data is first stored in the queue of unexpired data with the lowest storage volume.

[0031] Step S220: Store the service data sent by the user server and the corresponding validity period of the service data into the target unexpired data queue.

[0032] Step S300: Every preset time interval, the user server queries the valid time corresponding to each service data in each unexpired data queue. If the valid time corresponding to any service data is less than or equal to the current time, the service data is taken out from the unexpired data queue and stored in the expired data queue of the data storage repository. The expired data queue is a queue that stores service data whose validity period has expired.

[0033] Step S400: The user server obtains each service data in the expired data queue in real time, and returns the service termination information corresponding to the service data to the business server corresponding to the service data according to the service data in the expired data queue.

[0034] In the data processing method based on a caching system of the present invention, when the data repository receives service data and the corresponding validity period of the service data sent by the user server, it determines the storage priority of each unexpired data queue based on the predicted data increment of the data repository within the target time period and the remaining data volume of several unexpired data queues of the data repository within the target time period. The service data sent by the user server and the corresponding validity period are then stored in the unexpired data queue with the highest storage priority, so that the storage volume of service data in the unexpired data queue tends to be balanced. Every preset time interval, the user server queries the validity period of each service data in each unexpired data queue. If the validity period of any service data is less than or equal to the current time, the service data is retrieved from the unexpired data queue and stored in the expired data queue of the data repository. Furthermore, the user server obtains each service data in the expired data queue in real time and returns the service termination information corresponding to the service data to the business server corresponding to the service data based on the service data in the expired data queue. By determining the storage priority of several unexpired data queues, the unexpired data queue for storing storage data is selected to ensure that the amount of service data stored in any unexpired data queue does not exceed the preset storage data amount threshold, thereby reducing data processing latency. As the validity period of service data in unexpired data queues with lower storage priority expires, the expired service data will be transferred from the unexpired data queue to the expired data queue. This facilitates the user server's retrieval of expired service data, reduces the overall data storage volume of the unexpired data queue, reduces the number of times the user server calls the unexpired data queue, and further reduces the data processing time.

[0035] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0036] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0037] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0038] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0039] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0040] An electronic device according to this embodiment of the invention. The electronic device is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the invention.

[0041] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and buses connecting different system components (including memory and processor).

[0042] The storage device stores program code that can be executed by the processor to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of the present invention.

[0043] The storage may include readable media in the form of volatile storage, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0044] The storage may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more applications, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0045] A bus can represent one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus architectures.

[0046] Electronic devices can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable users to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (such as routers, modems, etc.). This communication can be performed through input / output (I / O) interfaces. Furthermore, electronic devices can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapters.

[0047] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0048] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0049] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0050] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0051] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0052] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0053] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0054] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data processing method based on a caching system, characterized in that, include: Step S100: When the data repository receives service data sent by the user server and the preset valid time corresponding to the service data, the storage priority corresponding to each unexpired data queue is determined according to the predicted data increment corresponding to the data repository within the target time period and the remaining data volume corresponding to several unexpired data queues of the data repository within the target time period; the start time of the target time period is the current time, and the duration of the target time period is the preset duration. Step S200: Store the service data sent by the user server and the validity period corresponding to the service data into the unexpired data queue with the highest storage priority; Step S300: Every preset time interval, the user server queries the validity time corresponding to each service data in each of the unexpired data queues. If the validity time corresponding to any service data is less than or equal to the current time, the service data is taken out from the unexpired data queue and stored in the expired data queue of the data storage repository. Step S400: The user server obtains each service data in the expired data queue in real time, and returns the service termination information corresponding to the service data to the business server corresponding to the service data according to the service data in the expired data queue.

2. The method according to claim 1, characterized in that, Step S100 includes: Step S110: When the data repository receives service data sent by the user server and the preset validity period corresponding to the service data, obtain the amount of stored data corresponding to each of the unexpired data queues at the current time to obtain a first list of stored data amounts B=(B1,B2,...,B...). i ,...,B m ); where i = 1, 2, ..., m; m is the number of the unexpired data queues; B i The amount of stored data corresponding to the i-th unexpired data queue at the current time; Step S120: Traverse the list of stored data volumes B. If B 02 ≤B i ≤B 01 Then, the i-th unexpired data queue is determined as the first data queue; where, B 01 B is the preset first data storage threshold; 02 B is the preset second data storage threshold; 02 <B 01 ; Step S130: Obtain the predicted data increment corresponding to each data processing time point within the target time period from the data repository, to obtain a predicted data increment list A=(A1,A2,...,A1). j ,...,A k ); where j=1,2,...,k; k is the number of data processing moments included within the target time period; A j The predicted data increment of the data repository at the j-th data processing time within the target time period; Step S140: Obtain the predicted data expiration amount corresponding to each data processing time point of each of the first data queues within the target time period, to obtain several lists of predicted data expiration amounts C1, C2, ..., C p ,...,C q Where p = 1, 2, ..., q; q is the number of the first data queue; C p This is a list of predicted data due dates for the p-th first data queue within the target time period; C p =(C p1 C p2 ,...,C pj ,...,C pk );C pj This represents the predicted data expiration amount corresponding to the p-th first data queue at the j-th data processing time within the target time period; Step S150: Based on a list of predicted amounts C1, C2, ..., C... p ,...,C q And with the predicted data increment list A, determine the remaining data volume corresponding to each data processing moment of several first data queues within the target time period; Step S160: If the remaining data volume corresponding to each data processing moment of all the first data queues within the target time period is greater than zero, then the storage priority corresponding to each first data queue is determined as the first priority, and the storage priority corresponding to the remaining unexpired data queues in the data storage repository is determined as the second priority; wherein, the first priority is higher than the second priority.

3. The method according to claim 2, characterized in that, Step S150 includes: Step S151: Obtain the amount of stored data for each of the first data queues at the current time to obtain a second list of stored data amounts D=(D1,D2,...,D...). p ,...,D q ); where D p Let p be the amount of stored data in the first data queue at the current time. Step S152: Determine the remaining data volume of several first data queues at each data processing time within the target time period to obtain a remaining data volume list E=(E1,E2,...,E...). j ,...,E k ); where E j The remaining data volume of several first data queues at the j-th data processing time within the target time period; E j =Z-(∑ q p=1 D p -∑ j t=1 ∑ q p=1 C pt +∑ j t=1 A t Z represents the preset total amount of data in the unexpired data queue.

4. The method according to claim 3, characterized in that, Step S160 further includes: Step S161, if E j If ≤0, then the unexpired data queue corresponding to the storage data volume in the first storage data volume list B that is only less than MIN(D) is determined as the first data queue, and the process returns to step S140; where MIN() is a preset minimum value determination function.

5. The method according to claim 2, characterized in that, Step S120 further includes: Step S121: If the amount of stored data corresponding to each of the unexpired data queues at the current time is greater than B... 01 If so, a new queue of unexpired data will be created in the data repository; Step S122: Determine the storage priority of the newly created unexpired data queue as the first priority, and determine the storage priority of the remaining unexpired data queues in the data storage repository as the second priority; wherein, the first priority is higher than the second priority.

6. The method according to claim 2, characterized in that, The predicted data increment at each data processing moment within the target time period is determined through the following steps: Step S131: Obtain the historical data increment corresponding to each data processing moment within the historical time period of the data repository, so as to obtain the historical data increment list F=(F1,F2,...,F...). j ,...,F k ); wherein the duration of the historical time period is equal to the duration of the target time period, the end time of the historical time period is the start time of the target time period, and the duration between any two adjacent data processing moments in the historical time period is equal to the duration between any two adjacent data processing moments in the target time period; F j The data repository represents the historical data increment corresponding to the j-th data processing moment within the historical time period; Step S132: Input the historical data increment list F into the preset data increment prediction model to obtain the predicted data increment of the data repository at each data processing time within the target time period, as output by the data increment prediction model; the data increment prediction model is obtained by supervised training on samples of several historical data increments.

7. The method according to claim 2, characterized in that, The amount of predicted data due at the j-th data processing time within the target time period for any of the first data queues is determined through the following steps: Step S141: Obtain the valid time corresponding to each service data stored in the first data queue to obtain a valid time list G=(G1,G2,...,G...). r ,...,G s ); where r=1,2,...,s; s is the number of service data stored in the first data queue; G r The effective time corresponding to the r-th service data stored in the first data queue; Step S142: Traverse the valid time list G. If G r The time is less than or equal to the time corresponding to the j-th data processing moment of the target time period, and G r If the time is greater than the (j-1)th data processing time corresponding to the target time period, then the rth service data stored in the first data queue is determined as the predicted expiration data corresponding to the jth data processing time. Step S143: Determine the number of predicted due data pairs corresponding to the j-th data processing time stored in the first data queue as the predicted data due amount corresponding to the j-th data processing time in the first data queue within the target time period.

8. The method according to claim 2, characterized in that, Step S200 further includes: Step S210: If the number of unexpired data queues with storage priority of first priority is greater than 1, then the unexpired data queue with the least amount of stored data among the unexpired data queues with storage priority of first priority is determined as the target unexpired data queue. Step S220: Store the service data sent by the user server and the valid time corresponding to the service data into the target unexpired data queue.

9. A non-transitory computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images