Data Synchronization Method and Apparatus, Storage Medium, and Electronic Device

Abstract

The present application discloses a data synchronization method, apparatus, storage medium, and electronic device, relating to the field of computer technologies, including: receiving a data writing request initiated by a server, where the data writing request is used to request writing target data into a memory; in response to the data writing request, detecting writing parameters of the target data, where the writing parameters are used to indicate the urgency degree of writing the target data into the memory; determining a calling time for the target data to call a synchronization resource according to the writing parameters; calling the synchronization resource according to the calling time, and synchronizing the target data to a standby controller, where the standby controller is used to cache the received data before storing the data into the memory. The technical problems such as low synchronization efficiency of data with a relatively high urgency degree are solved, and the technical effect of improving the synchronization efficiency of data with a relatively high urgency degree is achieved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computers, and more particularly, to a method and apparatus for synchronizing data, a storage medium, and an electronic device. Background Art

[0002] In related technologies, storage devices usually adopt a dual redundant architecture composed of a main controller and a standby controller. The standby controller is used to take over data services and maintain the normal operation of the system in the event of a failure of the main controller. In order to implement the above functions, the standby controller needs to synchronize the data received by the main controller.

[0003] Currently, the way for the main controller to synchronize data to the standby controller is that once the main controller receives data, it immediately synchronizes the data to the standby controller. Since the resources available for synchronizing data are limited, when the main controller receives a large amount of data that needs to be synchronized in a short period of time, the data will be synchronized to the standby controller in the order in which the data is received. In this case, the synchronization efficiency of emergency data (data with a higher degree of urgency for writing to the memory) is low, that is, the emergency data received by the main controller may not be synchronized to the standby controller in time because the main controller is busy synchronizing non-emergency data (data with a lower degree of urgency for writing to the memory). At this time, if the main controller suddenly fails, there will be a risk that the emergency data has not been synchronized to the standby controller, and the standby controller cannot write the emergency data to the memory because it has not received the emergency data, which may result in the loss of important emergency data.

[0004] In view of the technical problems such as the low synchronization efficiency of data with a higher degree of urgency in related technologies, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a method and apparatus for synchronizing data, a storage medium, and an electronic device, so as to at least solve the technical problems such as the low synchronization efficiency of data with a higher degree of urgency in related technologies.

[0006] According to an embodiment of the present application, a method for synchronizing data is provided. A server is respectively connected to a memory through a main controller and a standby controller, and the main controller and the standby controller are connected to each other. The method is applied to the main controller and includes: receiving a data write request initiated by the server, where the data write request is used to request writing target data to the memory; in response to the data write request, detecting write parameters of the target data, where the write parameters are used to indicate the urgency degree of writing the target data to the memory; determining a call timing for the target data to call synchronization resources according to the write parameters; and calling the synchronization resources according to the call timing to synchronize the target data to the standby controller, where the standby controller is used to cache the received data before storing the data in the memory.

[0007] According to another embodiment of the embodiments of the present application, a data synchronization device is further provided. The server is respectively connected to the memory through the main controller and the standby controller, and the main controller and the standby controller are connected to each other. The device is applied to the main controller and includes: a receiving module, configured to receive a data writing request initiated by the server, where the data writing request is used to request to write target data into the memory; a first detection module, configured to respond to the data writing request and detect the writing parameters of the target data, where the writing parameters are used to indicate the urgency of writing the target data into the memory; a determination module, configured to determine the calling timing of the synchronization resource for the target data according to the writing parameters; and a synchronization module, configured to call the synchronization resource according to the calling timing and synchronize the target data to the standby controller, where the standby controller is configured to cache the received data before storing it in the memory.

[0008] The present application further provides an electronic device, including: a memory, configured to store a computer program; and a processor, configured to implement the steps of any of the above data synchronization methods when executing the computer program.

[0009] The present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above data synchronization methods are implemented.

[0010] The present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of any of the above data synchronization methods are implemented.

[0011] Through the present application, when the server initiates a data writing request to the main controller, the main controller will detect the writing parameters of the target data and judge the urgency of writing the target data into the memory; according to the urgency of writing the target data into the memory, the main controller can judge when to call the synchronization resource and synchronize the target data to the standby controller. That is, according to the urgency of writing the target data into the memory, different calling timings are selected to synchronize the target data to the standby controller, realizing the priority synchronization of emergency data (equivalent to data with a higher urgency level), avoiding the delay in processing emergency data, and reducing the risk of loss of emergency data. Therefore, the technical problems such as the low synchronization efficiency of data with a higher urgency level in the related art can be solved, and the technical effect of improving the synchronization efficiency of data with a higher urgency level is achieved. Description of the Drawings

[0012] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0013] Figure 1 It is a hardware structure block diagram of a computer device for a data synchronization method according to an embodiment of the present application;

[0014] Figure 2 It is an overall architecture diagram of an optional storage system according to an embodiment of the present application;

[0015] Figure 3 It is a flowchart of a data synchronization method according to an embodiment of the present application;

[0016] Figure 4 It is a schematic diagram of data transmission of an optional first priority queue according to an embodiment of the present application;

[0017] Figure 5 It is a schematic diagram of an optional communication queue resource allocation according to an embodiment of the present application;

[0018] Figure 6 It is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application Figure 1 ;

[0019] Figure 7 It is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application Figure 2 ;

[0020] Figure 8 It is a schematic diagram of an optional judgment of write parameters according to an embodiment of the present application;

[0021] Figure 9 It is a structure block diagram of a data synchronization device according to an embodiment of the present application;

[0022] Figure 10 It is a schematic diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0024] It should be noted that in the description of this application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0025] To enable those skilled in the art of this technology to better understand the solution of this application, the following further details this application in conjunction with the accompanying drawings and specific embodiments.

[0026] The method embodiments provided in the embodiments of this application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 is a hardware structure block diagram of a computer device for a data synchronization method according to an embodiment of this application. As Figure 1 shown, the server device may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned server device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned server device. For example, the server device may further include more or fewer components than those shown in Figure 1 the figure, or have a different configuration from that shown in Figure 1 the figure.

[0027] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the data synchronization method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the server device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of a server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The nouns involved in the embodiments of this application are explained as follows:

[0030] IO: Input and Output, input and output;

[0031] IOPS: Input / Output Operations Per Second, the number of input / output operations per second;

[0032] RDMA: Remote Direct Memory Access, remote direct memory access;

[0033] FC: Fibre Channel, fibre channel card;

[0034] SAS: Serial Attached SCSI (Small Computer System Interface), serial SCSI technology;

[0035] CA: Cache, cache;

[0036] CACHE POOL: cache pool;

[0037] CPU: Central Processing Unit, central processing unit;

[0038] RAID: Redundant Arrays of Independent Disks, redundant array of independent disks;

[0039] NVME: Non-Volatile Memory Express, non-volatile memory express storage technology;

[0040] Disk: disk.

[0041] In the solution of this application, the storage controller (equivalent to the primary controller and the standby controller) is mainly responsible for storing and managing data on the hard disk, and at the same time provides data access services to the server side. During the process of providing data access (IO) to the server side, the security of data is generally ensured through the services of business modules such as CA, CACHE POOL, RAID, and multiple disks.

[0042] Figure 2 It is an optional overall architecture diagram of a storage system according to an embodiment of this application. As Figure 2 shown, the storage device generally consists of two controllers. The front end is connected to the server through technologies such as FC and RDMA, and the back end is connected to the storage disk (equivalent to the memory) through technologies such as SAS and NVME to form a redundant system.

[0043] When an application on the server generates a write IO request (equivalent to a data write request), the request will first be sent to one of the controllers of the storage device, such as controller A (equivalent to the primary controller), through a front-end card such as FC. Controller A identifies the write IO request and then receives the data. After the data reception is completed, controller A will not immediately send a response indicating that the write IO request is completed to the host. Instead, it will send the write request data to controller B as backup redundancy through the internal network of the storage device, that is, the external cards A and B between controller A and controller B, and then send a response indicating that the write IO request is completed to the host. In this way, if controller A fails, the host (equivalent to the server) can find the originally stored data from controller B.

[0044] In this embodiment, a data synchronization method is provided. The server is respectively connected to the memory through the primary controller and the standby controller, and the primary controller and the standby controller are connected to each other. The method is applied to the primary controller. Figure 3 It is a flowchart of a data synchronization method according to an embodiment of this application. As Figure 3 shown, the process includes the following steps:

[0045] Step S12, receiving a data write request initiated by the server, where the data write request is used to request writing target data into the memory;

[0046] Optionally, in this embodiment, the memory may include physical devices for storing target data, such as hard disk drives (HDDs), solid state drives (SSDs), and random access memories (RAMs), etc.

[0047] Step S14: In response to a data writing request, detect the writing parameters of the target data, where the writing parameters are used to indicate the urgency of writing the target data into the memory.

[0048] Optionally, in this embodiment, the data writing request may carry, but is not limited to, the target data and related parameters of the target data. The target data may include, but is not limited to, various types of data such as database records and log files. The related parameters of the target data may include, but are not limited to, writing parameters, addresses, data volumes, and other related parameters.

[0049] Optionally, in this embodiment, the writing parameters can be used to indicate the urgency of writing the target data into the memory. Different writing parameters can be set for different data according to the specific business requirements of the user. For example, if target data 1 is transaction data with extremely high requirements for real-time and accuracy of data, the writing parameter of target data 1 can be set to a relatively high value. For example, the writing parameter of target data 1 is set to 0.9, indicating a relatively high urgency of writing target data 1 into the memory. If target data 2 is log data with relatively low requirements for data real-time, the writing parameter of target data 2 can be set to a relatively low value. For example, the writing parameter of target data 2 is set to 0.3, indicating a relatively low urgency of writing target data 2 into the memory. The present application does not specifically limit the setting method of the writing parameters.

[0050] Step S16: Determine the calling timing of the target data to call the synchronization resource according to the writing parameters.

[0051] Optionally, in this embodiment, the synchronization resources may include CPU resources, memory resources, and external card link bandwidth resources, etc., which are used to synchronize the target data to the standby controller.

[0052] Optionally, in this embodiment, the calling timing of calling the synchronization resource may include immediately calling the synchronization resource or not immediately calling the synchronization resource.

[0053] Step S18: Call the synchronization resource according to the calling timing, and synchronize the target data to the standby controller, where the standby controller is used to cache the received data before storing it in the memory.

[0054] Optionally, in this embodiment, in the case of a failure of the primary controller, the standby controller can receive the data writing request initiated by the server and write the data into the memory.

[0055] Through the embodiments of the present application, when the server initiates a data writing request to the main controller, the main controller will detect the writing parameters of the target data and judge the urgency of writing the target data into the memory; according to the urgency of writing the target data into the memory, the main controller can judge when to call the synchronization resource to synchronize the target data to the standby controller. That is, according to the urgency of writing the target data into the memory, different calling times are selected to call the synchronization resource to synchronize the target data to the standby controller, realizing the priority synchronization of emergency data, avoiding the delay in processing emergency data, and reducing the risk of loss of emergency data. Therefore, it is possible to solve the technical problems such as low synchronization efficiency of data with a higher urgency level in the related art, and achieve the technical effect of improving the synchronization efficiency of data with a higher urgency level.

[0056] As an optional solution, determining the calling time for the target data to call the synchronization resource according to the writing parameters further includes:

[0057] S21, when it is detected that the writing parameters fall within the parameter range, immediately call the synchronization resource to synchronize the target data to the standby controller;

[0058] S22, when it is detected that the writing parameters do not fall within the parameter range, detect whether the target data meets the synchronization conditions, and when the target data meets the synchronization conditions, call the synchronization resource to synchronize the target data to the standby controller.

[0059] Optionally, in this embodiment, the writing parameters falling within the parameter range can be used to indicate that the urgency of writing the target data into the memory is relatively high, and it is necessary to immediately call the synchronization resource to synchronize the target data to the standby controller.

[0060] Optionally, in this embodiment, the writing parameters not falling within the parameter range can be used to indicate that the urgency of writing the target data into the memory is relatively low, and it is not necessary to immediately call the synchronization resource to synchronize the target data to the standby controller.

[0061] For example, the parameter range can be set to (0.8, 1). When the writing parameters fall within (0.8, 1), it indicates that the urgency of writing the target data into the memory is relatively high, belonging to emergency data, and it is necessary to immediately call the synchronization resource to synchronize the target data to the standby controller; when the writing parameters do not fall within (0.8, 1), it indicates that the urgency of writing the target data into the memory is relatively low, belonging to non-emergency data, and it is not necessary to immediately call the synchronization resource to synchronize the target data to the standby controller.

[0062] Optionally, in this embodiment, the parameter range can be defined according to the specific business requirements of the user, and the present application does not limit this.

[0063] Through the embodiments of the present application, according to the write parameters and the parameter range, the urgency of writing the target data into the memory is judged. When the urgency of writing the target data into the memory is relatively high, the synchronization resource is immediately called to synchronize the target data to the standby controller, ensuring that the urgent data can be quickly synchronized to the standby controller, reducing data latency, and improving data security. When the urgency of writing the target data into the memory is relatively low, the synchronization resource is not immediately called to synchronize the target data to the standby controller, avoiding excessive resource consumption and unnecessary synchronization, and improving resource utilization efficiency.

[0064] As an optional solution, when it is detected that the write parameter falls within the parameter range, immediately calling the synchronization resource to synchronize the target data to the standby controller further includes:

[0065] S31, adding the target data to the first priority queue;

[0066] S32, immediately calling the first synchronization resource for the first priority queue, where the first priority queue is set to synchronize the data in the first priority queue to the standby controller through the first synchronization resource, and the synchronization resource includes the first synchronization resource.

[0067] Optionally, in this embodiment, the data in the first priority queue may be data with a relatively high urgency of being written into the memory.

[0068] Optionally, in this embodiment, a resource pool may be allocated for the first priority queue in the early stage of system initialization. The resources in this resource pool can be used for data synchronization, and the resources in this resource pool include the first synchronization resource.

[0069] As an optional solution, immediately calling the first synchronization resource for the first priority queue further includes:

[0070] S41, detecting the target data volume of the target data in the first priority queue;

[0071] S42, when the target data volume is greater than the target data volume threshold, generating the target quantity of the first unit resource according to the target data volume and the target data volume threshold, where the target data volume threshold is the data volume allowed to be synchronized by a single first unit resource;

[0072] S43, determining the target quantity of the first unit resource as the first synchronization resource.

[0073] Optionally, in this embodiment, the target data volume can be used to indicate the size of the target data. For example, the target data volume of target data 1 can be 10MB, the target data volume of target data 2 can be 20MB, and so on.

[0074] Optionally, in this embodiment, the first unit resource may be used to indicate a unit resource in the resource pool corresponding to the first priority queue, and the resource pool corresponding to the first priority queue may include multiple first unit resources.

[0075] Optionally, in this embodiment, the target data volume threshold may be the data volume allowed to be synchronized by a single first unit resource. For example, if the data volume allowed to be synchronized by a single first unit resource is 10 MB, the target data volume threshold may be set to 10 MB.

[0076] Optionally, in this embodiment, when the target data volume is less than or equal to the target data volume threshold, a single first unit resource is determined as the first synchronization resource. Taking the target data volume as 5 MB and the target data volume threshold as 10 MB as an example, since the target data volume is less than the target data volume threshold, a single first unit resource is determined as the first synchronization resource.

[0077] Figure 4 It is a schematic diagram of data transmission of an optional first priority queue according to an embodiment of the present application. As Figure 4 shown, taking the target data volume as 10 MB and the target data volume threshold as 10 MB as an example. First, it is detected that the target data volume of target data 1 in the first priority queue is 10 MB. At this time, the target data volume is equal to the target data volume threshold, and a single first unit resource is determined as the first synchronization resource. The first unit resource A1 is called from the communication resource pool A, and the target data is synchronized to the standby controller through the external card A.

[0078] Optionally, in this embodiment, when the target data volume is greater than the target data volume threshold, the target number of first unit resources is generated according to the target data volume and the target data volume threshold, including: performing a division operation on the target data volume and the target data volume threshold to obtain a first candidate ratio; when the first candidate ratio is a positive integer, determining the first candidate ratio as the target number of first unit resources; when the first candidate ratio is not a positive integer, determining the result obtained by rounding up the first candidate ratio as the target number of first unit resources.

[0079] For example, when the target data volume is 20 MB and the target data volume threshold is 10 MB, since the target data volume is greater than the target data volume threshold, performing a division operation on the target data volume and the target data volume threshold, the first candidate ratio obtained is 2, and the target number of first unit resources is determined as 2; when the target data volume is 25 MB and the target data volume threshold is 10 MB, since the target data volume is greater than the target data volume threshold, performing a division operation on the target data volume and the target data volume threshold, the first candidate ratio obtained is 2.5, and the result obtained by rounding up the first candidate ratio is 3, and the target number of first unit resources is determined as 3.

[0080] As an alternative solution, when it is detected that the write parameter does not fall within the parameter range, it is detected whether the target data meets the synchronization condition. When the target data meets the synchronization condition, a synchronization resource is called to synchronize the target data to the standby controller. It further includes:

[0081] S52, adding the target data to the second priority queue;

[0082] S53, detecting whether the target data in the second priority queue meets the synchronization condition;

[0083] S54, when it is detected that the target data meets the synchronization condition, a second synchronization resource is called for the second priority queue, where the second priority queue is set to synchronize the data in the second priority queue to the standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

[0084] Optionally, in this embodiment, the data in the second priority queue may be data with a lower urgency level for writing to the memory.

[0085] Optionally, in this embodiment, a resource pool may be allocated for the second priority queue in the early stage of system initialization. The resources in this resource pool can be used for data synchronization, and the resources in this resource pool include the second synchronization resource.

[0086] Optionally, in this embodiment, since the functions and components that the storage device needs to implement are very complex, the CPU, memory, external card link bandwidth, etc. are all precious resources. This solution takes these backgrounds as preconditions, sets two priority queues in the early stage of system initialization, a high-priority queue A (equivalent to the first priority queue) and a low-priority queue B (equivalent to the second priority queue), and pre-allocates a certain number of resources for each queue. This resource is actually a continuous piece of memory with a fixed size. Assume that the size of each resource (equivalent to the first unit resource) in the resource pool of queue A is size_a (the amount of data allowed to be synchronized by a single first unit resource), and the size of each resource in the resource pool of queue B is size_b (the amount of data allowed to be synchronized by a single second unit resource), which is used to carry the data to be sent to other controllers. Figure 5 It is a schematic diagram of an optional communication queue resource allocation according to an embodiment of the present application. As Figure 5 shown, in the early stage of system initialization, a communication resource pool A is allocated for the high-priority queue A. The communication resource pool A includes multiple first unit resources. For example, resources A1, A2... resource A n . A communication resource pool B is allocated for the low-priority queue B. The communication resource pool B includes multiple second unit resources. For example, resources B1, B2... resource B n+m .

[0087] The pre-allocation amounts of the two queue resources can be represented by n and n + m, where n represents the quantity of the first unit of resources in resource pool A, and n + m represents the quantity of the second unit of resources in resource pool B. The quantity of the first unit of resources in the high-priority queue can be set to be lower than the quantity of the second unit of resources in the low-priority queue, and this application does not make specific limitations on this.

[0088] As an optional solution, detecting whether the target data in the second-priority queue meets the synchronization condition further includes at least one of the following:

[0089] S61, detecting whether the waiting duration of the target data waiting to be sent in the second-priority queue is greater than or equal to the first duration threshold; and when the waiting duration is greater than or equal to the first duration threshold, determining that the target data is detected to meet the first synchronization condition, where the synchronization condition includes the first synchronization condition;

[0090] S62, detecting whether the sending interval duration of the second-priority queue where the target data is located is greater than or equal to the second duration threshold, where the sending interval duration is the interval duration from the current moment of the second-priority queue to the last synchronization of data to the standby controller; and when the sending interval duration is greater than or equal to the second duration threshold, determining that the target data is detected to meet the second synchronization condition, where the synchronization condition includes the second synchronization condition;

[0091] S63, detecting whether the reference data volume of all data in the second-priority queue is greater than or equal to the reference data volume threshold; and when the reference data volume is greater than or equal to the reference data volume threshold, determining that the target data is detected to meet the third synchronization condition, where the synchronization condition includes the third synchronization condition.

[0092] Optionally, in this embodiment, the first duration threshold can be but is not limited to indicating the longest waiting duration of the target data waiting to be sent in the second-priority queue. The first duration threshold can be but is not limited to being set by the user according to requirements. For example, the first duration threshold can be set to 500 ms or 600 ms, etc., and this application does not make limitations on this.

[0093] Optionally, in this embodiment, the second duration threshold can be but is not limited to indicating the maximum sending interval duration of the second-priority queue where the target data is located. The second duration threshold can be but is not limited to being set by the user according to requirements. For example, the second duration threshold can be set to 300 ms or 200 ms, etc., and this application does not make limitations on this.

[0094] Optionally, in this embodiment, the reference data volume can be used to indicate the total data volume of all data in the second priority queue, and the reference data volume threshold can be used to indicate the maximum value allowed for the total data volume of all data in the second priority queue. For example, the reference data volume threshold can be set to the data volume allowed to be synchronized by a single second unit resource or the data volume allowed to be synchronized by n second unit resources.

[0095] Through the embodiments of the present application, the timing of calling the synchronization resource is determined according to the waiting duration of the target data waiting to be sent in the second priority queue, the sending interval duration of the second priority queue where the target data is located, and the total data volume of all data in the second priority queue, which can effectively manage the data synchronization in the second priority queue, avoid long-term backlog and delayed processing of data, ensure the reasonable allocation and utilization of system resources, meet the regular update requirements of data, and maintain the stable operation and high efficiency of the system.

[0096] As an optional solution, for calling the second synchronization resource for the second priority queue, it further includes:

[0097] S71, obtaining the candidate data volume of the currently to-be-sent data in the second priority queue, where the currently to-be-sent data includes the target data;

[0098] S72, when the candidate data volume is less than or equal to the candidate data volume threshold, determining a single second unit resource as the second synchronization resource, where the candidate data volume threshold is the data volume allowed to be synchronized by a single second unit resource;

[0099] S73, when the candidate data volume is greater than the candidate data volume threshold, generating the candidate quantity of the second unit resources according to the candidate data volume and the candidate data volume threshold; determining the candidate quantity of the second unit resources as the second synchronization resource.

[0100] Optionally, in this embodiment, generating the candidate quantity of the second unit resources according to the candidate data volume and the candidate data volume threshold includes: performing a division operation on the candidate data volume and the candidate data volume threshold to obtain a second candidate ratio; when the second candidate ratio is a positive integer, determining the second candidate ratio as the target quantity of the second unit resources; when the second candidate ratio is not a positive integer, determining the result of rounding up the second candidate ratio as the target quantity of the second unit resources.

[0101] Optionally, in this embodiment, Figure 6 is a schematic diagram of an optional data sending of the second priority queue according to the embodiments of the present application Figure 1 , such as Figure 6As shown, taking the candidate data volume of the data to be currently sent in the second priority queue as 10 MB and the candidate data volume threshold as 10 MB as an example, first, the candidate data volume of the data to be currently sent in the second priority queue is obtained as 10 MB. It is determined that the candidate data volume of 10 MB is equal to the candidate data volume threshold of 10 MB. A single second unit resource is determined as the second synchronization resource. Resource B1 is called from communication resource pool B, and the current data to be sent is synchronized to the standby controller through external card A.

[0102] Optionally, in this embodiment, Figure 7 is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application Figure 2 , such as Figure 7 As shown, taking the candidate data volume of the data to be currently sent in the second priority queue as 15 MB and the candidate data volume threshold as 10 MB as an example, first, the candidate data volume of the data to be currently sent in the second priority queue is obtained as 15 MB. It is determined that the candidate data volume of 15 MB is greater than the candidate data volume threshold of 10 MB. A division operation is performed on the candidate data volume and the candidate data volume threshold to obtain a second candidate ratio of 1.5. The second candidate ratio is rounded up to 2. The target quantity of the second unit resource is determined as 2. The second unit resource B1 and the second unit resource B2 are called from communication resource pool B, and the current data to be sent is synchronized to the standby controller through external card A.

[0103] As an optional solution, after calling the second synchronization resource for the second priority queue, the method further includes:

[0104] S81, generating transmission parameters for the second priority queue according to the candidate data volume of the data to be currently sent in the second priority queue, the target resource volume of the second synchronization resource, and a preset constant, where the transmission parameters are used to indicate the current utilization rate of the second synchronization resource by the second priority queue, and the preset constant is used to indicate the minimum value allowed for the current utilization rate of the second synchronization resource by the second priority queue;

[0105] S82, adjusting the second duration threshold according to the transmission parameters, the previous second duration threshold used, and the first duration threshold to obtain an adjusted second duration threshold.

[0106] Optionally, in this embodiment, the target resource volume of the second synchronization resource is used to indicate the data volume allowed to be synchronized by the second synchronization resource.

[0107] Optionally, in this embodiment, after each start of the sending process of queue B (equivalent to the second priority queue), that is, after calling the second synchronization resource for the second priority queue, the sending efficiency (equivalent to the transmission parameters) can be statistically calculated, and at the same time, the overall waiting time threshold of the data to be sent in the queue (equivalent to the second duration threshold) is updated.

[0108] As an alternative solution, generate the transmission parameter of the second priority queue according to the candidate data volume of the data to be transmitted by the second priority queue currently, the target resource volume of the second synchronization resource, and a preset constant, including:

[0109] S91, perform a division operation on the target resource volume and the candidate data volume to obtain a target ratio;

[0110] S92, determine the minimum value between the target ratio and the preset constant as the transmission parameter.

[0111] Optionally, in this embodiment, the transmission parameter can be calculated through where ratio represents the transmission parameter, size_B represents the data volume allowed to be synchronized by the second synchronization resource, size represents the candidate data volume of the data to be transmitted by the second priority queue currently, and the preset constant is 2.

[0112] Optionally, in this embodiment, the preset constant can be a constant preset by the user according to requirements. For example, the preset constant can be set to 2. The preset constant can be used to prevent the system from making overly aggressive adjustments to the synchronization mechanism under extremely low load or abnormal conditions, so that the system will not generate excessive jitter. For example, if suddenly the actual transmission volume size is very small, resulting in a very high ratio of size_B / size (equivalent to the target ratio), the system may erroneously think that it is necessary to frequently start the data transmission process, resulting in unnecessary resource consumption and system jitter. By setting the upper limit to 2, the system can moderately accelerate the data processing speed when the resource utilization rate increases, but can remain stable and avoid over-adjustment when the resource utilization rate is low or the data volume fluctuates greatly.

[0113] Optionally, in this embodiment, the purpose of calculating the ratio of size_B / size is to measure whether each data synchronization makes full use of the second synchronization resource. When the ratio of size_B / size is equal to 1, it means that the second synchronization resource is effectively utilized; when the ratio of size_B / size is greater than 1, it means that there is still a certain surplus of the second synchronization resource, and there is room for improvement in the utilization rate of the second synchronization resource.

[0114] For example, the data volume size_B allowed to be synchronized by the second synchronization resource is 10MB, the candidate data volume size of the data to be transmitted by the second priority queue currently is 5MB, and the ratio of size_B / size is 2, indicating that there is still a certain surplus of the second synchronization resource and the utilization rate of the second synchronization resource can be further improved.

[0115] Optionally, in this embodiment, the transmission parameter can be calculated through Calculate the adjusted second duration threshold. Among them, time_thrshold_b represents the adjusted second duration threshold, time_thrshold_B represents the second duration threshold used last time, time_max_b represents the first duration threshold, and ratio represents the sending parameter.

[0116] Optionally, in this embodiment, the overall system response time, that is, the time from when the data of each service module is sent until it is sent completely, can also be counted as a factor to adjust the queue sending time interval threshold (equivalent to the second duration threshold). If it is found that the overall response time is short, it indicates that the current data synchronization efficiency is high. At this time, the queue sending time interval can be appropriately increased to reduce the occupancy of the CPU and other resources and further optimize the system performance. On the contrary, if the overall response time is long, this may mean that there is a delay in the data communication link. The system should reduce the queue sending time interval and increase the data sending frequency to clear the queue as soon as possible and reduce the waiting time of the data in the queue, thereby improving the overall response speed. Adjusting the queue sending time interval through the overall system response time can more sensitively respond to different data communication conditions, ensure the stability and reliability of data synchronization, and reduce the system performance degradation and potential failures caused by data backlog or excessive resource consumption.

[0117] Through the embodiments of the present application, when the utilization rate of the second synchronization resource can be further improved, the utilization rate of the second synchronization resource can be improved by adjusting the second duration threshold. For example, when the amount of data size_B that the second synchronization resource allows to synchronize is 15MB and the candidate data amount size of the data to be sent currently in the second priority queue is 10MB, the ratio of size_B / size is 1.5, indicating that there is still a certain amount of remaining second synchronization resource. The second duration threshold can be increased. This adjustment means that controller A will allow the data to wait longer in the second priority queue until the candidate data amount approaches or reaches the carrying capacity size_B of the second synchronization resource, and then perform data synchronization. By dynamically adjusting the second duration threshold, the efficiency of each data synchronization operation can be maximized, the frequency of data synchronization can be reduced, and the resource waste caused by frequently starting synchronization operations when there are sufficient resources can be avoided.

[0118] As an optional solution, before synchronizing the target data to the standby controller by calling the synchronization resource according to the call timing, the method further includes:

[0119] S101, detect the remaining resource amount of the remaining resources in the resource pool to which the synchronization resource belongs;

[0120] S102, when it is detected that the remaining resource amount is less than or equal to the first resource threshold, add resources to the resource pool;

[0121] S103. When it is detected that the remaining resource amount is greater than the second resource threshold, release some resources in the resource pool.

[0122] Optionally, in this embodiment, both the synchronization resources and the remaining resources are resources that can be used to synchronize target data to the standby controller.

[0123] Optionally, in this embodiment, the first resource threshold can be defined by the user according to requirements. For example, the first resource threshold can be set to 80%.

[0124] Optionally, in this embodiment, the second resource threshold can be defined by the user according to requirements. For example, the second resource threshold can be set to 20%.

[0125] Optionally, in this embodiment, during system initialization, a certain amount of resources is pre-allocated to the resource pools of two queues (the first priority queue and the second priority queue). However, as the storage system undertakes changes in front-end host IO requests (equivalent to data write requests), that is, as the IOPS changes, the resources in the resource pool also need to be dynamically allocated. That is, the resources in the resource pool need to change positively with actual needs such as IOPS, so that the system memory resources can be fully utilized. IOPS can be used to measure the data processing capacity of the system and the performance of the storage system. When the IOPS increases, it means that the number of IO requests that the system needs to process increases, and the demand for resources increases; conversely, when the IOPS decreases, the demand for resources weakens.

[0126] Optionally, in this embodiment, the process of adding resources to the resource pool and releasing some resources in the resource pool can be carried out immediately after a queue completes a round of data transmission.

[0127] Through the embodiments of this application, by detecting the remaining resource amount of the remaining resources in the resource pool in real time and comparing it with the preset first resource threshold and second resource threshold, the system can automatically identify the situations of resource shortage and resource surplus. When resources are in short supply, resources are replenished to the resource pool in a timely manner, ensuring the smooth progress of data synchronization operations and avoiding data synchronization delays that may be caused by insufficient resources. When there is an excess of resources, resource release is initiated to release the resources in the resource pool for other business modules to use, achieving the maximization of resource utilization.

[0128] As an optional solution, when it is detected that the remaining resource amount is less than or equal to the first resource threshold, adding resources to the resource pool further includes:

[0129] S111. Determine the first load parameter as the first ratio between the previously used second duration threshold and the adjusted second duration threshold;

[0130] S112. Generate a first adjustment quantity based on the second ratio of the total resources in the resource pool to the resources possessed by a single second unit of resources, the third ratio between the remaining resources and the total resources in the resource pool, and the first load parameter;

[0131] S113. Add the second unit of resources with the first adjustment quantity to the resource pool.

[0132] Optionally, in this embodiment, when the resources in the resource pool are in urgent need, that is, each time the queue applies for resources from the resource pool, it will query the remaining resources in the pool. Generally, when the resource margin (equivalent to the remaining resources) is lower than 80% of the total amount at initialization in the pool (equivalent to the total resources in the resource pool), additional resources need to be applied to the system. The specific application quantity can be determined through determination. Among them, allocate_count represents the first adjustment quantity, that is, the specific application quantity, n+m represents the ratio of the total resources in the resource pool to the resources possessed by a single second unit of resources, ratio_count represents the ratio between the remaining resources and the total resources in the resource pool, time_thrshold_B represents the second duration threshold used last time, time_thrshold_b represents the adjusted second duration threshold, and time_thrshold_B / time_thrshold_b represents the first load parameter.

[0133] Optionally, in this embodiment of the present application, a third adjustment quantity can be generated based on the ratio of the total resources in the resource pool to the resources possessed by a single first unit of resources, the ratio between the remaining resources and the total resources in the resource pool, and the ratio of the IOPS value processed by the current first-priority queue to the preset IOPS value threshold for the first-priority queue; add the first unit of resources with the third adjustment quantity to the resource pool. By using the ratio of the IOPS value processed by the first-priority queue to the preset IOPS value threshold for the first-priority queue as a factor to add the first unit of resources with the third adjustment quantity to the resource pool, it is ensured that the resource allocation closely matches the real-time business requirements of the first-priority queue. Even in the face of a sudden high-IOPS scenario, the resource allocation can be quickly adjusted, improving the system's ability to handle emergencies and ensuring the security of data and the continuity of services.

[0134] As an optional solution, when it is detected that the remaining resources for synchronizing resources are greater than the second resource threshold, releasing some resources from the resource pool further includes:

[0135] S121. Determine the fourth ratio between the adjusted second duration threshold and the second duration threshold used last time as the second load parameter;

[0136] S122. Generate a second adjustment quantity based on the fifth ratio of the total resource quantity of the resource pool to the resource quantity of a single second unit resource, the sixth ratio between the remaining resource quantity and the total resource quantity of the resource pool, and the second load parameter.

[0137] S123. Release the second unit resources of the second adjustment quantity for the resource pool.

[0138] Optionally, in this embodiment, when the system's busyness level decreases, additional resources in the resource pool need to be released for other service modules to use. At the same time, to ensure system stability, the release of resources should also be stepped and incremented each time. Generally, resource release is only performed when the remaining resource quantity is more than 20% of the initial value and the total resource quantity of the resource pool is more than the initial value. First, the current system load condition can be calculated. Among them, ratio_load represents the second load parameter, time_thrshold_b represents the adjusted second duration threshold, and time_thrshold_B represents the previously used second duration threshold.

[0139] Optionally, in this embodiment, the amount of resource release each time can be calculated through . Among them, n + m represents the ratio of the total resource quantity of the resource pool to the resource quantity of a single second unit resource, ratio_count represents the ratio between the remaining resource quantity and the total resource quantity of the resource pool, and ratio_load represents the second load parameter.

[0140] Optionally, in this embodiment, a fourth adjustment quantity can be generated based on the ratio of the total resource quantity of the resource pool to the resource quantity of a single first unit resource, the ratio between the remaining resource quantity and the total resource quantity of the resource pool, and the ratio of the number of IOPS processed by the current first priority queue to the preset IOPS numerical threshold of the first priority queue; release the first unit resources of the fourth adjustment quantity for the resource pool.

[0141] Optionally, in this embodiment, for a better understanding of the above data synchronization process, the following further describes the above data synchronization process in combination with optional embodiments, but it is not used to limit the technical solutions of the embodiments of the present application.

[0142] Figure 8 It is a schematic diagram of an optional judgment of write parameters according to an embodiment of the present application. As Figure 8As shown in the figure, when the service senses data to be sent to other controller nodes (e.g., controller B), it first determines its message type by itself (equivalent to writing parameters). When it is determined that the message type of the data is used to indicate a relatively high urgency for writing the message to the memory, the data is added to queue A (equivalent to the first priority queue). When it is determined that the message type of the data is used to indicate a relatively low urgency for writing the message to the memory, the data is added to queue B (equivalent to the second priority queue).

[0143] Optionally, in this embodiment, if the data to be sent is of the urgent and non-waiting type (equivalent to data with a relatively high urgency), it is appended to queue A (equivalent to the first priority queue). Queue A senses that there is data to be sent, uses the synchronous sending mechanism, does not stay, immediately applies for a resource from resource pool A, stores the data to be sent in it, and submits it to the external card driver to send to the peer. Specifically, if the size of the data to be sent exceeds the carrying capacity size_a of a single resource (equivalent to the first unit resource), it is split into multiple first unit resources for sending.

[0144] Optionally, in this embodiment, if the data to be sent is of other types (equivalent to data with a relatively low urgency), the following assignment is first made: time_thrshold_b = time_thrshold_B.

[0145] Then the processing steps are as follows:

[0146] a) Append the data to be sent to queue B (equivalent to the second priority queue). Queue B senses the newly appended data to be sent and does not immediately start the data sending mechanism. Instead, it temporarily stores the data in the queue and uses the asynchronous sending mechanism to decouple the message sending process from the message generation process. This can reduce the CPU occupancy and improve the overall response speed of the system.

[0147] b) Queue B polls whether there is data to be sent in the queue that has stayed in the queue for more than the waiting sending time threshold time_max_b (equivalent to the first duration threshold). If so, it immediately applies for resource B1 from resource pool B, packs it with other data in the queue, and submits it to the external card A driver for sending. At the same time, it updates the queue sending data time, denoted as send_time_b.

[0148] c) If there is no data in queue B with a retention time exceeding the waiting sending time threshold time_max_b:

[0149] i. Queue B determines whether the current time is the time threshold time_threshold_b (equivalent to the first duration threshold) from the last time the queue sent data according to the send time send_time_b. If so, it immediately applies for resources from Resource Pool B (equivalent to the second synchronization resource), packs the data to be sent in the queue, and submits it to the external card A for driving and sending. At the same time, it updates the queue data sending time, denoted as send_time_b.

[0150] ii. Queue B polls the data to be sent in the queue and counts the total amount of data to be sent, denoted as total_size_B (equivalent to the reference data volume threshold). If total_size_B is greater than or equal to size_B (equivalent to the reference data volume threshold), it immediately applies for resources from Resource Pool B to pack the data to be sent in the queue and submit it to the external card A for driving and sending. At the same time, it updates the queue data sending time, denoted as send_time_b; and then repeats the above steps again until the condition that total_size_B is greater than or equal to size_B is not met.

[0151] This solution designs a communication mechanism for data synchronization between storage controllers, that is, between clusters. By setting up a first-priority queue and a second-priority queue to distinguish and process urgent and non-urgent data. Urgent data is directly transmitted immediately through the high-priority queue, and non-urgent data in the low-priority queue determines the calling time to call the synchronization resource to synchronize the data to the standby controller according to the waiting time for sending a single message (equivalent to the first duration threshold), the overall waiting time for sending data (equivalent to the second duration threshold), and the maximum value allowed by the total amount of all data in the second-priority queue, reducing data latency, improving data security, and further improving the overall processing efficiency of the IO service.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0153] Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of this application.

[0154] In this embodiment, a data synchronization device is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0155] Figure 9 is a structural block diagram of a data synchronization device according to an embodiment of the present application; as Figure 9 shown, the server is respectively connected to the memory through the main controller and the standby controller, the main controller and the standby controller are connected to each other, and the device is applied to the main controller. The device includes:

[0156] A receiving module 1001, configured to receive a data writing request initiated by the server, where the data writing request is used to request to write target data into the memory;

[0157] A first detection module 1002, configured to respond to the data writing request and detect the writing parameter of the target data, where the writing parameter is used to indicate the urgency of writing the target data into the memory;

[0158] A determination module 1003, configured to determine the calling timing of the synchronization resource for the target data according to the writing parameter;

[0159] A synchronization module 1004, configured to call the synchronization resource according to the calling timing and synchronize the target data to the standby controller, where the standby controller is used to cache the received data before storing it in the memory.

[0160] Through the above embodiments, when the server initiates a data writing request to the main controller, the main controller will detect the writing parameter of the target data and judge the urgency of writing the target data into the memory; according to the urgency of writing the target data into the memory, the main controller can judge when to call the synchronization resource and synchronize the target data to the standby controller. That is, according to the urgency of writing the target data into the memory, different calling timings are selected to synchronize the target data to the standby controller, realizing the priority synchronization of emergency data, avoiding the delay in processing emergency data, and reducing the risk of loss of emergency data. Therefore, it can solve the technical problems such as the low synchronization efficiency of data with a higher urgency in the related art, and achieve the technical effect of improving the synchronization efficiency of data with a higher urgency.

[0161] In an exemplary embodiment, the determination module includes:

[0162] A first calling unit, configured to immediately call the synchronization resource to synchronize the target data to the standby controller when it is detected that the writing parameter falls within the parameter range;

[0163] A second calling unit, configured to detect whether target data meets a synchronization condition when it is detected that a write parameter does not fall within a parameter range, and when the target data meets the synchronization condition, call a synchronization resource to synchronize the target data to a standby controller.

[0164] In an exemplary embodiment, the first calling unit is configured to:

[0165] Add the target data to a first priority queue;

[0166] Immediately call a first synchronization resource for the first priority queue, where the first priority queue is set to synchronize the data in the first priority queue to a standby controller through the first synchronization resource, and the synchronization resource includes the first synchronization resource.

[0167] In an exemplary embodiment, the first calling unit is further configured to:

[0168] Detect a target data volume of the target data in the first priority queue;

[0169] When the target data volume is greater than a target data volume threshold, generate a target quantity of first unit resources according to the target data volume and the target data volume threshold, where the target data volume threshold is the data volume allowed to be synchronized by a single first unit resource;

[0170] Determine the target quantity of first unit resources as the first synchronization resource.

[0171] In an exemplary embodiment, the second calling unit is configured to:

[0172] When it is detected that a write parameter does not fall within a parameter range, detect whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, call a synchronization resource to synchronize the target data to a standby controller, including:

[0173] Add the target data to a second priority queue;

[0174] Detect whether the target data in the second priority queue meets the synchronization condition;

[0175] When it is detected that the target data meets the synchronization condition, call a second synchronization resource for the second priority queue, where the second priority queue is set to synchronize the data in the second priority queue to a standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

[0176] In an exemplary embodiment, the second calling unit is further configured to:

[0177] Detect whether the waiting duration of the target data waiting to be sent in the second priority queue is greater than or equal to the first duration threshold; and in the case where the waiting duration is greater than or equal to the first duration threshold, determine that the target data is detected to meet the first synchronization condition, where the synchronization condition includes the first synchronization condition;

[0178] Detect whether the transmission interval duration of the second priority queue where the target data is located is greater than or equal to the second duration threshold, where the transmission interval duration is the interval duration between the current moment of the second priority queue and the last synchronization of data to the standby controller; and in the case where the transmission interval duration is greater than or equal to the second duration threshold, determine that the target data is detected to meet the second synchronization condition, where the synchronization condition includes the second synchronization condition;

[0179] Detect whether the reference data volume of all data in the second priority queue is greater than or equal to the reference data volume threshold; and in the case where the reference data volume is greater than or equal to the reference data volume threshold, determine that the target data is detected to meet the third synchronization condition, where the synchronization condition includes the third synchronization condition.

[0180] In an exemplary embodiment, the second calling unit is further configured to:

[0181] Obtain the candidate data volume of the data currently waiting to be sent in the second priority queue, where the currently waiting to be sent data includes the target data;

[0182] In the case where the candidate data volume is less than or equal to the candidate data volume threshold, determine a single second unit resource as the second synchronization resource, where the candidate data volume threshold is the data volume allowed to be synchronized by a single second unit resource;

[0183] In the case where the candidate data volume is greater than the candidate data volume threshold, generate the candidate quantity of the second unit resources according to the candidate data volume and the candidate data volume threshold, where the candidate data volume threshold is the data volume allowed to be synchronized by a single second unit resource; determine the candidate quantity of the second unit resources as the second synchronization resource.

[0184] In an exemplary embodiment, the apparatus further includes:

[0185] A generating module, configured to, after calling the second synchronization resource for the second priority queue, generate the transmission parameter of the second priority queue according to the candidate data volume of the data currently waiting to be sent in the second priority queue, the target resource volume of the second synchronization resource, and a preset constant, where the transmission parameter is used to indicate the utilization rate of the second synchronization resource by the second priority queue currently, and the preset constant is used to indicate the minimum value allowed for the utilization rate of the second synchronization resource by the second priority queue currently;

[0186] An adjustment module, configured to adjust a second duration threshold according to a sending parameter, a second duration threshold used last time, and a first duration threshold, so as to obtain an adjusted second duration threshold.

[0187] In an exemplary embodiment, the generation module includes:

[0188] An execution unit, configured to perform a division operation on a target resource amount and a candidate data amount to obtain a target ratio;

[0189] A first determination unit, configured to determine the minimum value between the target ratio and a preset constant as the sending parameter.

[0190] In an exemplary embodiment, the apparatus further includes:

[0191] A second detection module, configured to detect a remaining resource amount of the remaining resources in the resource pool to which the synchronization resource belongs before synchronizing target data to the standby controller according to a call timing;

[0192] An increase module, configured to add resources to the resource pool when it is detected that the remaining resource amount is less than or equal to a first resource threshold;

[0193] A release module, configured to release some resources in the resource pool when it is detected that the remaining resource amount is greater than a second resource threshold.

[0194] In an exemplary embodiment, the increase module includes:

[0195] A second determination unit, configured to determine a first ratio between the second duration threshold used last time and the adjusted second duration threshold as a first load parameter;

[0196] A first generation unit, configured to generate a first adjustment quantity according to a second ratio between the total resource amount of the resource pool and the resource amount of a single second unit resource, a third ratio between the remaining resource amount and the total resource amount of the resource pool, and the first load parameter;

[0197] An increase unit, configured to add a second unit resource with a first adjustment quantity to the resource pool.

[0198] In an exemplary embodiment, the release module includes:

[0199] A third determination unit, configured to determine a fourth ratio between the adjusted second duration threshold and the second duration threshold used last time as a second load parameter;

[0200] A second generation unit, configured to generate a second adjustment quantity according to a fifth ratio between the total resource amount of the resource pool and the resource amount of a single second unit resource, a sixth ratio between the remaining resource amount and the total resource amount of the resource pool, and the second load parameter;

[0201] A release unit for releasing a second quantity of second unit resources for the resource pool.

[0202] It should be noted that the above-mentioned modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: the above-mentioned modules are all located in the same processor; or, the above-mentioned modules are respectively located in different processors in any combination form.

[0203] For the description of the features in the corresponding embodiments of the data synchronization device, reference can be made to the relevant descriptions in the corresponding embodiments of the data synchronization method, which will not be elaborated here one by one.

[0204] An embodiment of the present application also provides an electronic device. Figure 10 It is a schematic diagram of the electronic device according to the embodiment of the present application, as Figure 10 shown. The electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned embodiments of the data synchronization method.

[0205] In an exemplary embodiment, the above-mentioned electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above-mentioned processor, and the input / output device is connected to the above-mentioned processor.

[0206] For the specific examples in this embodiment, reference can be made to the examples described in the above-mentioned embodiments and exemplary embodiments, and this embodiment will not be elaborated here.

[0207] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. Among them, the computer program is configured to execute the steps in any one of the above-mentioned embodiments of the data synchronization method when running.

[0208] In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: USB flash drives, read-only memories (abbreviated as ROM), random access memories (abbreviated as RAM), mobile hard disks, magnetic disks, or optical discs, etc., various media that can store computer programs.

[0209] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the steps of the methods in various embodiments of the present application; the computer program product also includes a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores the computer program. When the computer program is executed by a processor, it implements the steps of the data synchronization method in various embodiments of the present application.

[0210] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0211] The above has introduced in detail a method for synchronizing data provided in this application. Specific examples are used herein to illustrate the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can still be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for synchronizing data, characterized in that The server is respectively connected to the memory through the primary controller and the standby controller, and the primary controller and the standby controller are connected to each other. The method is applied to the primary controller and includes: Receiving a data writing request initiated by the server, where the data writing request is used to request writing target data into the memory; Responding to the data writing request and detecting the writing parameters of the target data, where the writing parameters are used to indicate the urgency of writing the target data into the memory; Determining the calling timing of the synchronization resource for the target data according to the writing parameters; Calling the synchronization resource according to the calling timing and synchronizing the target data to the standby controller, where the standby controller is used to cache the data before storing the received data into the memory; Wherein, determining the calling timing of the synchronization resource for the target data according to the writing parameters includes: when it is detected that the writing parameters fall within the parameter range, immediately calling the synchronization resource to synchronize the target data to the standby controller; when it is detected that the writing parameters do not fall within the parameter range, detecting whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, calling the synchronization resource to synchronize the target data to the standby controller; Wherein, immediately calling the synchronization resource to synchronize the target data to the standby controller includes: when the target data volume of the target data is greater than the target data volume threshold, generating the target quantity of the first unit resource according to the target data volume and the target data volume threshold, where the target data volume threshold is the data volume allowed to be synchronized by a single first unit resource; immediately calling the target quantity of the first unit resource to synchronize the target data to the standby controller, where the synchronization resource includes the target quantity of the first unit resource.

2. The method according to claim 1, wherein When it is detected that the writing parameters fall within the parameter range, immediately calling the synchronization resource to synchronize the target data to the standby controller includes: Adding the target data to the first priority queue; Immediately calling the first synchronization resource for the first priority queue, where the first priority queue is set to synchronize the data in the first priority queue to the standby controller through the first synchronization resource, and the synchronization resource includes the first synchronization resource.

3. The method according to claim 2, wherein The immediately calling the first synchronization resource for the first priority queue includes: Detecting the target data volume of the target data in the first priority queue; When the target data volume is greater than the target data volume threshold, generating the target quantity of the first unit resource according to the target data volume and the target data volume threshold, where the target data volume threshold is the data volume allowed to be synchronized by a single first unit resource; Determining the target quantity of the first unit resource as the first synchronization resource.

4. The method according to claim 1, wherein When it is detected that the writing parameters do not fall within the parameter range, detecting whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, calling the synchronization resource to synchronize the target data to the standby controller includes: Add the target data to the second priority queue; Detect whether the target data in the second priority queue meets the synchronization condition; When it is detected that the target data meets the synchronization condition, call a second synchronization resource for the second priority queue, where the second priority queue is set to synchronize the data in the second priority queue to the standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

5. The method according to claim 4, wherein The detecting whether the target data in the second priority queue meets the synchronization condition includes at least one of the following: Detect whether the waiting duration for the target data to wait for transmission in the second priority queue is greater than or equal to a first duration threshold; and when the waiting duration is greater than or equal to the first duration threshold, determine that it is detected that the target data meets the first synchronization condition, where the synchronization condition includes the first synchronization condition; Detect whether the transmission interval duration of the second priority queue where the target data is located is greater than or equal to a second duration threshold, where the transmission interval duration is the interval duration between the current moment of the second priority queue and the last synchronization of data to the standby controller; and when the transmission interval duration is greater than or equal to the second duration threshold, determine that it is detected that the target data meets the second synchronization condition, where the synchronization condition includes the second synchronization condition; Detect whether the reference data volume of all data in the second priority queue is greater than or a reference data volume threshold; and when the reference data volume is greater than or equal to the reference data volume threshold, determine that it is detected that the target data meets the third synchronization condition, where the synchronization condition includes the third synchronization condition.

6. The method according to claim 4, wherein The calling a second synchronization resource for the second priority queue includes: Obtain the candidate data volume of the data to be sent currently in the second priority queue, where the data to be sent currently includes the target data; When the candidate data volume is less than or equal to the candidate data volume threshold, determine a single second unit resource as the second synchronization resource, where the candidate data volume threshold is the data volume allowed to be synchronized by a single second unit resource; When the candidate data volume is greater than the candidate data volume threshold, generate a candidate quantity of second unit resources according to the candidate data volume and the candidate data volume threshold; determine the candidate quantity of second unit resources as the second synchronization resource.

7. The method according to claim 5, wherein After calling the second synchronization resource for the second priority queue, the method further includes: Generate a transmission parameter for the second priority queue based on the candidate data volume of the data to be transmitted by the second priority queue currently, the target resource volume of the second synchronization resource, and a preset constant, where the transmission parameter is used to indicate the current utilization rate of the second priority queue for the second synchronization resource, and the preset constant is used to indicate the minimum value allowed for the current utilization rate of the second priority queue for the second synchronization resource; Adjust the second duration threshold according to the transmission parameter, the second duration threshold used last time, and the first duration threshold to obtain the adjusted second duration threshold.

8. The method according to claim 7, wherein The generating the transmission parameter for the second priority queue based on the candidate data volume of the data to be transmitted by the second priority queue currently, the target resource volume of the second synchronization resource, and a preset constant includes: Perform a division operation on the target resource volume and the candidate data volume to obtain a target ratio; Determine the minimum value between the target ratio and the preset constant as the transmission parameter.

9. The method according to claim 7, wherein Before invoking the synchronization resource according to the invocation opportunity and synchronizing the target data to the standby controller, the method further includes: Detect the remaining resource volume of the remaining resources in the resource pool to which the synchronization resource belongs; When it is detected that the remaining resource volume is less than or equal to the first resource threshold, add resources to the resource pool; When it is detected that the remaining resource volume is greater than the second resource threshold, release some resources in the resource pool.

10. The method according to claim 9, wherein The adding resources to the resource pool when it is detected that the remaining resource volume is less than or equal to the first resource threshold includes: Determine a first load parameter as the first ratio between the second duration threshold used last time and the adjusted second duration threshold; Generate a first adjustment quantity according to the second ratio between the total resource volume of the resource pool and the resource volume of a single second unit resource, the third ratio between the remaining resource volume and the total resource volume of the resource pool, and the first load parameter; Add the second unit resources of the first adjustment quantity to the resource pool.

11. The method according to claim 9, wherein The releasing some resources in the resource pool when it is detected that the remaining resource volume of the synchronization resource is greater than the second resource threshold includes: Determine a second load parameter as the fourth ratio between the adjusted second duration threshold and the second duration threshold used last time; Generate a second adjustment quantity according to the fifth ratio between the total resource volume of the resource pool and the resource volume of a single second unit resource, the sixth ratio between the remaining resource volume and the total resource volume of the resource pool, and the second load parameter; Release the second unit resources of the second adjustment quantity from the resource pool.

12. An electronic device, characterized in that, including: A memory for storing a computer program; A processor for implementing the steps of the data synchronization method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the data synchronization method according to any one of claims 1 to 11 are implemented.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the data synchronization method according to any one of claims 1 to 11 are implemented.

Images