A data processing method, apparatus, electronic device, and storage medium

Abstract

The present application provides a data processing method, apparatus, electronic device, and storage medium, belonging to the technical field of data processing. In the present application, the first storage capacity corresponding to the first redis cluster is obtained in real time, where the first storage capacity increases as the data written into the first redis cluster increases; when the first storage capacity is greater than or equal to the first capacity threshold, the first data to be written corresponding to the first redis cluster is obtained, writing of the first data to be written into the first redis cluster is stopped, and the first data to be written is written into the second redis cluster. Thus, the automatic expansion ability can be provided for the redis cluster with the resources of a set of standby redis clusters, thereby improving the efficiency of expanding the redis storage capacity. Moreover, during the expansion process, there is no need to shut down the redis cluster, so that the stability of the service can be ensured and the user experience can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Redis (Remote Dictionary Server), that is, the remote dictionary service, is an open-source log-type, Key-Value database written in ANSI C language, supporting the network, and can be based on memory or persistent. Since the storage capacity of redis is insufficient, it is easy to cause the redis service to be unavailable. Therefore, when using the redis service, there is often a problem of needing to expand the storage capacity of redis.

[0003] Currently, the main way to expand the storage capacity of redis is manual expansion. Specifically, when it is found that the storage capacity of the redis cluster is insufficient, the redis cluster is shut down, the resources of each node are upgraded, and then the redis cluster is started up.

[0004] However, using the manual expansion method to expand the storage capacity of redis is inefficient. Moreover, since the redis cluster needs to be shut down during each expansion, the service cannot be provided normally during the shutdown period, which affects the user experience. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a data processing method, apparatus, electronic device, and storage medium to solve the problem of low efficiency in expanding and contracting the storage capacity of redis by using the manual expansion and contraction method. The specific technical solutions are as follows:

[0006] In a first aspect, a data processing method is provided. The method includes:

[0007] Obtain the first storage capacity corresponding to the first redis cluster in real time, where the first storage capacity increases as the data written into the first redis cluster increases;

[0008] When the first storage capacity is greater than or equal to the first capacity threshold, obtain the first data to be written corresponding to the first redis cluster, stop writing the first data to be written into the first redis cluster, and write the first data to be written into the second redis cluster.

[0009] In a possible implementation, the method further includes:

[0010] When the first storage capacity is greater than or equal to the first capacity threshold, obtain target data in the first Redis cluster with a data expiration duration less than a preset duration threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration;

[0011] Write the target data to the second Redis cluster, and delete the target data in the first Redis cluster.

[0012] In a possible implementation, the method further includes:

[0013] During the process of writing the data to be written to the second Redis cluster, obtain the second storage capacity corresponding to the first Redis cluster, where the second storage capacity decreases as the data in the first Redis cluster decreases;

[0014] When the second storage capacity is less than or equal to the second capacity threshold, stop writing data to the second Redis cluster, and obtain second data to be written, and write the second data to be written to the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

[0015] In a possible implementation, the method further includes:

[0016] Receive a first access request sent by a first application;

[0017] Determine whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period;

[0018] When the first access request is not a slow command, send the first access request to the first Redis cluster;

[0019] When the first access request is a slow command, ignore the first access request.

[0020] In a possible implementation, the method further includes:

[0021] When the first access request is a slow command, determine the number of second access requests that are currently accessing the first Redis cluster;

[0022] When the number is less than a preset number threshold, send the first access request to the first Redis cluster;

[0023] When the number is greater than or equal to the preset number threshold, ignore the first access request.

[0024] In a second aspect, a data processing device is provided, the device including:

[0025] A first acquisition module, configured to acquire in real time a first storage capacity corresponding to a first Redis cluster, wherein the first storage capacity increases as the data written into the first Redis cluster increases;

[0026] A first writing module, configured to, when the first storage capacity is greater than or equal to a first capacity threshold, acquire first data to be written corresponding to the first Redis cluster, stop writing the first data to be written into the first Redis cluster, and write the first data to be written into a second Redis cluster.

[0027] In a possible implementation manner, the device further includes:

[0028] A second acquisition module, configured to, when the first storage capacity is greater than or equal to the first capacity threshold, acquire target data in the first Redis cluster whose data expiration duration is less than a preset duration threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration;

[0029] A second writing module, configured to write the target data into the second Redis cluster and delete the target data in the first Redis cluster.

[0030] In a possible implementation manner, the device further includes:

[0031] A third acquisition module, configured to, during the process of writing the data to be written into the second Redis cluster, acquire a second storage capacity corresponding to the first Redis cluster, wherein the second storage capacity decreases as the data in the first Redis cluster decreases;

[0032] A third writing module, configured to, when the second storage capacity is less than or equal to a second capacity threshold, stop writing data into the second Redis cluster, and acquire second data to be written and write the second data to be written into the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

[0033] In a possible implementation manner, the method further includes:

[0034] A receiving module, configured to receive a first access request sent by a first application;

[0035] A first determination module, configured to determine whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period;

[0036] A first sending module, configured to send the first access request to the first redis cluster when the first access request is a non-slow command;

[0037] A first ignoring module, configured to ignore the first access request when the first access request is a slow command.

[0038] In a possible implementation manner, the method further includes:

[0039] A second determining module, configured to determine the number of second access requests that are currently accessing the first redis cluster when the first access request is a slow command;

[0040] A second sending module, configured to send the first access request to the first redis cluster when the number is less than a preset number threshold;

[0041] A second ignoring module, configured to ignore the first access request when the number is greater than or equal to the preset number threshold.

[0042] In a third aspect, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0043] The memory is used to store a computer program;

[0044] The processor is configured to implement the method steps of any one of the first aspects when executing the program stored on the memory.

[0045] In a fourth aspect, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, and the computer program implements the method steps of any one of the first aspects when executed by a processor.

[0046] In a fifth aspect, a computer program product containing instructions is provided, which when running on a computer causes the computer to execute the data processing method described above.

[0047] Advantageous effects of the embodiments of the present application:

[0048] The embodiments of the present application provide a data processing method, apparatus, electronic device and storage medium. The present application obtains the first storage capacity corresponding to the first Redis cluster in real time, where the first storage capacity increases as the data written into the first Redis cluster increases; when the first storage capacity is greater than or equal to the first capacity threshold, the first data to be written corresponding to the first Redis cluster is obtained, writing the first data to be written into the first Redis cluster is stopped, and the first data to be written is written into the second Redis cluster. Thus, the automatic expansion ability can be provided for the Redis cluster with the resources of a set of standby Redis clusters, thereby improving the efficiency of expanding the Redis storage capacity. Moreover, during the expansion process, there is no need to shut down the Redis cluster, so the stability of the service can be ensured and the user experience can be improved.

[0049] Of course, it is not necessary for any product or method implementing the present application to achieve all the above advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0051] Figure 1 It is a flowchart of a data processing method provided by an embodiment of the present application;

[0052] Figure 2 It is a flowchart of another data processing method provided by an embodiment of the present application;

[0053] Figure 3 It is a flowchart of another data processing method provided by an embodiment of the present application;

[0054] Figure 4 It is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application;

[0055] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0057] At present, the main way to expand the storage capacity of Redis is manual expansion. Specifically, when it is found that the storage capacity of the Redis cluster is insufficient, the Redis cluster is shut down. After upgrading the resources of each node, the Redis cluster is restarted. However, using the manual expansion method to expand the storage capacity of Redis is inefficient. Moreover, since the Redis cluster needs to be shut down during each expansion, services cannot be provided normally during the shutdown period, affecting the user experience. For this reason, the embodiments of the present application provide a data processing method.

[0058] Next, a data processing method provided by the embodiments of the present application will be described in detail in combination with specific embodiments. As Figure 1 shown, the specific steps are as follows:

[0059] S101, obtain the first storage capacity corresponding to the first Redis cluster in real time, where the first storage capacity increases as the data written into the first Redis cluster increases.

[0060] A data processing method provided by the embodiments of the present application can be applied to a Redis proxy program. Among them, the Redis proxy program can manage multiple sets of Redis clusters at the same time. All application programs accessing the Redis cluster indirectly access the Redis clusters managed by the Redis proxy program through the Redis proxy program.

[0061] In the embodiments of the present application, the first Redis cluster can be any one of the multiple sets of Redis clusters managed by the Redis proxy program at the same time, and the first storage capacity increases as the data written into the first Redis cluster increases. Specifically, the Redis proxy program determines the storage situation of the first Redis cluster by obtaining the first storage capacity corresponding to the first Redis cluster in real time.

[0062] S102, when the first storage capacity is greater than or equal to the first capacity threshold, obtain the first data to be written corresponding to the first Redis cluster, stop writing the first data to be written into the first Redis cluster, and write the first data to be written into the second Redis cluster.

[0063] In the embodiments of the present application, when the first storage capacity is greater than or equal to the first capacity threshold, it means that the storage capacity of the first Redis cluster is insufficient.

[0064] As an implementation manner, the first data to be written can be all the data that was originally to be written into the first Redis cluster. Specifically, all the data that was originally to be written into the first Redis cluster is written into the standby second Redis cluster, thereby alleviating the storage pressure on the first Redis cluster.

[0065] As another implementation manner, the first data to be written can be part of the data that was originally to be written into the first Redis cluster. In this implementation manner, the data that was originally to be written into the first Redis cluster all corresponds to an effective storage duration. When the duration for which the data has been written into the Redis cluster reaches this effective storage duration, it will be automatically cleared. Among them, the effective storage duration corresponding to each data can be different.

[0066] Specifically, the data with an effective storage duration less than a preset duration can be determined as the first data to be written. The specific value of the preset duration can be set or adjusted according to the data volume. For example, after determining the first data to be written according to the initial preset duration, the data volume of the first data to be written is counted. When the data volume is less than the preset data volume threshold, the specific value of the preset duration can be increased. After each adjustment, the data volume can be continuously counted until the data volume is greater than or equal to the preset data volume threshold, or until the preset duration is greater than or equal to the maximum effective storage duration. Among them, the data that is valid for a long time is regarded as data without an effective duration, and the data that is valid for a long time can be set to be written into the first Redis cluster all the time.

[0067] Through this solution, the data with a longer effective storage duration can be continued to be written into the first Redis cluster, and only the data with a shorter effective storage duration is written into the second Redis cluster. Thereby alleviating the storage pressure on the first Redis cluster and at the same time reducing the workload of transferring data back from the second Redis cluster to the first Redis cluster in the future.

[0068] Furthermore, when reading data, the data in the second Redis cluster can be read first. When the pointer (key) does not exist, then the first Redis cluster is read. Since there is less data in the second Redis cluster, the reading time consumption is relatively short, thus improving the reading efficiency.

[0069] In an embodiment of the present application, the first storage capacity corresponding to the first Redis cluster is obtained in real time, where the first storage capacity increases as the data written into the first Redis cluster increases; when the first storage capacity is greater than or equal to the first capacity threshold, the first data to be written corresponding to the first Redis cluster is obtained, writing the first data to be written into the first Redis cluster is stopped, and the first data to be written is written into the second Redis cluster. Thus, the automatic expansion ability is provided to the Redis cluster with the resources of a set of standby Redis clusters, thereby improving the efficiency of expanding the Redis storage capacity. Moreover, during the expansion process, there is no need to shut down the Redis cluster, so the stability of the service can be ensured and the user experience can be improved.

[0070] In another embodiment of the present application, as Figure 2 shown, the method may further include the following steps:

[0071] S201, when the first storage capacity is greater than or equal to the first capacity threshold, obtain the target data in the first Redis cluster whose data expiration duration is less than the preset duration threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration.

[0072] S202, write the target data into the second Redis cluster, and delete the target data in the first Redis cluster.

[0073] The embodiment of the present application provides another expansion method. The data storage duration is the duration when the data is stored in the first Redis cluster, and the data expiration duration is the difference between the data storage duration and the effective storage duration corresponding to the data, that is, the remaining duration until the data is deleted. When the first storage capacity is greater than or equal to the first capacity threshold, it means that the speed of data clearing has not caught up with the speed of data writing. Therefore, in this embodiment, the expansion can be achieved by deleting some data in the first Redis cluster.

[0074] Specifically, the target data in the first Redis cluster whose data expiration duration is less than the preset duration threshold can be obtained, where the data expiration duration is the difference between the data storage duration and the effective duration, that is, the target data is the data in the first Redis cluster that is about to reach the effective duration. Then, the target data is written into the second Redis cluster for backup and the target data is deleted in the first Redis cluster, thereby achieving the expansion of the first Redis cluster.

[0075] Further, a specific duration threshold can be determined according to the data volume in the first Redis cluster. For example, the larger the data volume in the first Redis cluster, the larger the duration threshold, or, the larger the data volume exceeding the capacity threshold in the first Redis cluster, the larger the duration threshold, etc.

[0076] In the embodiment of the present application, when the first storage capacity is greater than or equal to the first capacity threshold, target data in the first Redis cluster with a data expiration duration less than a preset duration threshold is obtained, where the data expiration duration is the difference between the data storage duration and the effective storage duration. The target data is written into the second Redis cluster, and the target data is deleted from the first Redis cluster, thereby realizing automatic expansion of the first Redis cluster. And during the expansion process, there is no need to shut down the Redis cluster, so the stability of the service can be guaranteed and the user experience can be improved.

[0077] In another embodiment of the present application, the method may further include the following steps:

[0078] Step 1, during the process of writing the data to be written into the second Redis cluster, obtain the second storage capacity corresponding to the first Redis cluster, where the second storage capacity decreases as the data in the first Redis cluster decreases;

[0079] Step 2, when the second storage capacity is less than or equal to the second capacity threshold, stop writing data into the second Redis cluster, and obtain second data to be written, and write the second data to be written into the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

[0080] In the embodiment of the present application, when the storage capacity of the first Redis cluster resumes stability, the second Redis cluster can be disconnected and the first Redis cluster can be re-enabled.

[0081] Specifically, during the process of writing the data to be written into the second Redis cluster, obtain the second storage capacity corresponding to the first Redis cluster. Among them, during the process of stopping writing data in the first Redis cluster, as invalid data and expired data are deleted, its corresponding second storage capacity decreases as the data in the first Redis cluster decreases. When the second storage capacity is less than or equal to the second capacity threshold, where the second capacity threshold is less than the first capacity threshold, it means that the storage capacity of the first Redis cluster resumes stability. At this time, writing data into the second Redis cluster can be stopped, and the second data to be written into the second Redis cluster can be written into the first Redis cluster.

[0082] In the embodiments of the present application, when the storage capacity of the first Redis cluster is restored to stability, the second Redis cluster can be automatically disconnected, and the first Redis cluster can be re-enabled. Thereby ensuring that the first Redis cluster can provide stable services, and during the switching process, no manual operation is required, which can improve the switching efficiency and ensure the timeliness of the switching.

[0083] In another embodiment of the present application, the method may further include the following steps: writing the unexpired data written into the second Redis cluster into the first Redis cluster. Thereby ensuring the integrity of the data in the first Redis cluster, and it is convenient to read the data subsequently, that is, only need to read the data in the first Redis cluster.

[0084] In another embodiment of the present application, as Figure 3 shown, the method may further include the following steps:

[0085] S301, receiving a first access request sent by a first application;

[0086] S302, determining whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period;

[0087] S303, when the first access request is not a slow command, sending the first access request to the first Redis cluster;

[0088] S304, when the first access request is a slow command, ignoring the first access request.

[0089] In the embodiments of the present application, the first application is an application that wants to access the first Redis cluster, and it sends a first access request to the Redis proxy program to request access to the first Redis cluster. The slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period.

[0090] Specifically, when the Redis proxy program receives the first access request, it first determines whether the first access request is a slow command. Only when the first access request is not a slow command, the first access request is sent to the first Redis cluster; and when the first access request is a slow command, the first access request is ignored, that is, the first application is prohibited from executing the slow command. Thereby avoiding sending the slow command to the first Redis cluster, and further avoiding the slow processing speed of the first Redis cluster due to responding to the slow command.

[0091] In another embodiment of the present application, the method may further include the following steps:

[0092] Step 1: When the first access request is a slow command, determine the number of second access requests that are currently accessing the first Redis cluster simultaneously.

[0093] Step 2: When the number is less than a preset number threshold, send the first access request to the first Redis cluster.

[0094] Step 3: When the number is greater than or equal to the preset number threshold, ignore the first access request.

[0095] In an embodiment of the present application, when the first access request is a slow command, instead of directly performing an ignore process, it is also possible to determine the number of second access requests that are currently accessing the first Redis cluster simultaneously. When this number is less than the preset number threshold, it means that the number of access requests currently being processed by the first Redis cluster is relatively small and it has the ability to process slow commands. At this time, the first access request can be sent to the first Redis cluster; while when the number is greater than or equal to the preset number threshold, it means that the number of access requests currently being processed by the first Redis cluster is relatively large. If a slow command is processed at this time, the processing speed cannot be guaranteed. At this time, the first access request is ignored. Thus, it is avoided to continue sending slow commands to the first Redis cluster when the number of access requests being processed by the first Redis cluster simultaneously is large, thereby ensuring the processing speed of the first Redis cluster.

[0096] In another embodiment of the present application, the method may further include the following steps:

[0097] During the process of writing the first data to be written into the second Redis cluster, and when the first access request is a slow command, ignore the first access request.

[0098] In an embodiment of the present application, during the process of writing the first data to be written into the second Redis cluster, when executing an access request, it is necessary to query both the first Redis cluster and the second Redis cluster simultaneously. If it takes 5 seconds to execute a slow command once, then the total time for querying the first Redis cluster and the second Redis cluster together will be 10 seconds, thereby increasing the risk of query. Therefore, during the process of writing the first data to be written into the second Redis cluster, when the first access request is a slow command, the first access request is ignored, that is, the slow command is not processed, thereby avoiding the execution time being too long due to processing the slow command, resulting in client exceptions.

[0099] Based on the same technical concept, an embodiment of the present application further provides a data processing device, as Figure 4 shown, the device includes:

[0100] A first acquisition module 401, configured to acquire in real time a first storage capacity corresponding to a first Redis cluster, where the first storage capacity increases as the data written into the first Redis cluster increases;

[0101] A first writing module 402, configured to, when the first storage capacity is greater than or equal to a first capacity threshold, acquire first data to be written corresponding to the first Redis cluster, stop writing the first data to be written into the first Redis cluster, and write the first data to be written into a second Redis cluster.

[0102] In a possible implementation manner, the apparatus further includes:

[0103] A second acquisition module, configured to, when the first storage capacity is greater than or equal to the first capacity threshold, acquire target data in the first Redis cluster whose data expiration duration is less than a preset duration threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration;

[0104] A second writing module, configured to write the target data into the second Redis cluster and delete the target data in the first Redis cluster.

[0105] In a possible implementation manner, the apparatus further includes:

[0106] A third acquisition module, configured to, during the process of writing the data to be written into the second Redis cluster, acquire a second storage capacity corresponding to the first Redis cluster, where the second storage capacity decreases as the data in the first Redis cluster decreases;

[0107] A third writing module, configured to, when the second storage capacity is less than or equal to a second capacity threshold, stop writing data into the second Redis cluster, and acquire second data to be written and write the second data to be written into the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

[0108] In a possible implementation manner, the method further includes:

[0109] A receiving module, configured to receive a first access request sent by a first application;

[0110] A first determination module, configured to determine whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period;

[0111] A first sending module, configured to send the first access request to the first redis cluster when the first access request is not a slow command;

[0112] A first ignoring module, configured to ignore the first access request when the first access request is a slow command.

[0113] In a possible implementation manner, the method further includes:

[0114] A second determining module, configured to determine the number of second access requests that are currently accessing the first redis cluster when the first access request is a slow command;

[0115] A second sending module, configured to send the first access request to the first redis cluster when the number is less than a preset number threshold;

[0116] A second ignoring module, configured to ignore the first access request when the number is greater than or equal to the preset number threshold.

[0117] In the embodiments of the present application, the first storage capacity corresponding to the first redis cluster is obtained in real time, where the first storage capacity increases as the data written into the first redis cluster increases; when the first storage capacity is greater than or equal to the first capacity threshold, the first data to be written corresponding to the first redis cluster is obtained, writing the first data to be written into the first redis cluster is stopped, and the first data to be written is written into the second redis cluster. Thus, the automatic expansion ability can be provided for the redis cluster with the resources of a set of standby redis clusters, thereby improving the efficiency of expanding the storage capacity of the redis. And during the expansion process, there is no need to shut down the redis cluster, so the stability of the service can be guaranteed and the user experience can be improved.

[0118] Based on the same technical concept, the embodiments of the present application further provide an electronic device, as Figure 5 shown, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, where the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114,

[0119] The memory 113 is used to store a computer program;

[0120] The processor 111, when executing the program stored on the memory 113, implements the following steps:

[0121] Obtain the first storage capacity corresponding to the first redis cluster in real time, where the first storage capacity increases as the data written into the first redis cluster increases;

[0122] When the first storage capacity is greater than or equal to the first capacity threshold, obtain the first data to be written corresponding to the first redis cluster, stop writing the first data to be written to the first redis cluster, and write the first data to be written to the second redis cluster.

[0123] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0124] The communication interface is used for communication between the above electronic device and other devices.

[0125] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0126] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0127] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above data processing methods are implemented.

[0128] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute any of the data processing methods in the above embodiments.

[0129] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0130] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0131] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data processing method, characterized in that, The method includes: Obtaining in real time the first storage capacity corresponding to the first Redis cluster, where the first storage capacity increases as the data written to the first Redis cluster increases; When the first storage capacity is greater than or equal to the first capacity threshold, obtaining the first data to be written corresponding to the first Redis cluster, stopping writing the first data to be written to the first Redis cluster, and writing the first data to be written to the second Redis cluster; The method further includes: Receiving a first access request sent by a first application; Determining whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period; When the first access request is not a slow command, sending the first access request to the first Redis cluster; When the first access request is a slow command, ignoring the first access request.

2. The method according to claim 1, characterized in that, The method further includes: When the first storage capacity is greater than or equal to the first capacity threshold, obtaining target data in the first Redis cluster whose data expiration duration is less than a preset duration threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration; Writing the target data to the second Redis cluster and deleting the target data in the first Redis cluster.

3. The method according to claim 1, characterized in that, The method further includes: During the process of writing the data to be written to the second Redis cluster, obtaining the second storage capacity corresponding to the first Redis cluster, where the second storage capacity decreases as the data in the first Redis cluster decreases; When the second storage capacity is less than or equal to the second capacity threshold, stopping writing data to the second Redis cluster, and obtaining second data to be written and writing the second data to be written to the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

4. The method according to claim 1, characterized in that The method further includes: When the first access request is a slow command, determining the number of second access requests currently accessing the first Redis cluster; When the number is less than a preset number threshold, sending the first access request to the first Redis cluster; When the number is greater than or equal to the preset number threshold, ignoring the first access request.

5. A data processing device, characterized in that, The device includes: A first obtaining module, configured to obtain in real time the first storage capacity corresponding to the first Redis cluster, where the first storage capacity increases as the data written to the first Redis cluster increases; A first writing module, configured to, when the first storage capacity is greater than or equal to the first capacity threshold, obtain the first data to be written corresponding to the first Redis cluster, stop writing the first data to be written to the first Redis cluster, and write the first data to be written to the second Redis cluster; A receiving module, configured to receive a first access request sent by a first application; A first determination module, configured to determine whether the first access request is a slow command, where the slow command is an access request whose response time exceeds a preset time threshold when accessing the first Redis cluster within a preset time period; A first sending module, configured to send the first access request to the first Redis cluster when the first access request is not a slow command; A first ignoring module, configured to ignore the first access request when the first access request is a slow command.

6. The device according to claim 5, wherein The device further includes: A second obtaining module, configured to obtain target data in the first Redis cluster whose data expiration duration is less than a preset duration threshold when the first storage capacity is greater than or equal to a first capacity threshold, where the data expiration duration is the difference between the data storage duration and the effective storage duration; A second writing module, configured to write the target data into the second Redis cluster and delete the target data in the first Redis cluster.

7. The device according to claim 5, characterized in that The device further includes: A third obtaining module, configured to obtain a second storage capacity corresponding to the first Redis cluster during the process of writing the data to be written into the second Redis cluster, where the second storage capacity decreases as the data in the first Redis cluster decreases; A third writing module, configured to stop writing data into the second Redis cluster when the second storage capacity is less than or equal to a second capacity threshold, and obtain second data to be written, and write the second data to be written into the first Redis cluster, where the second capacity threshold is less than the first capacity threshold.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store a computer program; The processor, when executing the program stored on the memory, implements the method steps described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method steps described in any one of claims 1-4.

Images