Data synchronization method and device, storage medium and electronic equipment

Abstract

The invention discloses a data synchronization method and device, a storage medium and electronic equipment, and relates to the technical field of computers, the data synchronization method comprises the steps that a data writing request initiated by a server is received, and the data writing request is used for requesting to write target data into a memory; in response to the data write-in request, write-in parameters of the target data are detected, and the write-in parameters are used for indicating the emergency degree of writing the target data into a memory; determining a calling opportunity of calling the synchronization resource by the target data according to the write-in parameter; the synchronization resource is called according to the calling time, the target data is synchronized to the standby controller, and the standby controller is used for caching the data before the received data is stored in the memory. The technical problem that the synchronization efficiency of the data with the high emergency degree is low is solved, and the technical effect of improving the synchronization efficiency of the data with the high emergency degree is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and more specifically, to a data synchronization method and device, a storage medium, and an electronic device. Background Art

[0002] In the related art, storage devices usually adopt a dual redundant architecture consisting of a main controller and a backup controller. The backup controller is used to take over data services and maintain the normal operation of the system in the event of a failure of the main controller. In order to achieve the above functions, the backup controller needs to synchronize the data received by the main controller.

[0003] Currently, the method for the main controller to synchronize data with the standby controller is: once the main controller receives the data, it will immediately synchronize with the standby controller. Since the resources allowed for synchronizing data are limited, when the main controller receives a large amount of data that needs to be synchronized in a short period of time, it will synchronize the data to the standby controller in the order of the received data. In this case, the efficiency of synchronization of urgent data (data with a higher degree of urgency to write to the memory) is low, that is, the urgent data received by the main controller may not be synchronized to the standby controller in time because the main controller is busy synchronizing non-urgent data (data with a lower degree of urgency to write to the memory). At this time, if the main controller suddenly fails, the urgent data will not be synchronized to the standby controller. Since the standby controller has not received the urgent data, it cannot write the urgent data to the memory, and there may be a risk of losing important urgent data.

[0004] In view of the technical problems in the related technologies, such as low efficiency in synchronizing data with a high degree of urgency, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a data synchronization method and device, a storage medium, and an electronic device to at least solve the technical problems in the related art such as low efficiency of synchronization of data with a high degree of urgency.

[0006] According to an embodiment of the present application, a data synchronization method is provided, wherein a server is connected to a memory via a main controller and a standby controller respectively, and the main controller and the standby controller are connected to each other, and the method is applied to the main controller, and the method includes: receiving a data write request initiated by the server, wherein the data write request is used to request that target data be written to the memory; responding to the data write request, detecting write parameters of the target data, wherein the write parameters are used to indicate the urgency of writing the target data to the memory; determining a calling timing of a synchronization resource for the target data according to the write parameters; calling the synchronization resource according to the calling timing, and synchronizing the target data to the standby controller, wherein the standby controller is used to cache the data before storing the received data in the memory.

[0007] According to another embodiment of the embodiment of the present application, a data synchronization device is also provided, wherein a server is connected to a memory through a main controller and a standby controller respectively, and the main controller and the standby controller are connected to each other, and the device is applied to the main controller, and the device includes: a receiving module, used to receive a data write request initiated by the server, wherein the data write request is used to request to write target data to the memory; a first detection module, used to respond to the data write request and detect the write parameters of the target data, wherein the write parameters are used to indicate the urgency of writing the target data to the memory; a determination module, used to determine the calling timing of the target data calling the synchronization resource according to the write parameters; a synchronization module, used to call the synchronization resource according to the calling timing, and synchronize the target data to the standby controller, wherein the standby controller is used to cache the received data before storing it in the memory.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned data synchronization methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned data synchronization methods are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data synchronization methods when the computer program is executed by a processor.

[0011] Through this application, when the server initiates a data write request to the main controller, the main controller will detect the write parameters of the target data and determine the urgency of writing the target data to the memory; according to the urgency of writing the target data to the memory, the main controller can determine when to call the synchronization resource to synchronize the target data to the standby controller. That is, according to the urgency of writing the target data to the memory, different calling times are selected to synchronize the target data to the standby controller, thereby achieving priority synchronization of urgent data (equivalent to data with a higher degree of urgency), avoiding delays in the processing of urgent data, and reducing the risk of urgent data loss. Therefore, technical problems in related technologies such as low efficiency of synchronization of data with a higher degree of urgency can be solved, and the technical effect of improving the efficiency of synchronization of data with a higher degree of urgency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 It is a hardware structure block diagram of a computer device of a data synchronization method according to an embodiment of the present application;

[0014] Figure 2 is an optional overall architecture diagram of a storage system according to an embodiment of the present application;

[0015] Figure 3 is a flow chart of a data synchronization method according to an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of data transmission of an optional first priority queue according to an embodiment of the present application;

[0017] Figure 5 is a schematic diagram of an optional communication queue resource allocation according to an embodiment of the present application;

[0018] Figure 6 This is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application. Figure 1 ;

[0019] Figure 7 This is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application. Figure 2 ;

[0020] Figure 8 is a schematic diagram of an optional method for determining write parameters according to an embodiment of the present application;

[0021] Fig. 9 is a structural block diagram of a data synchronization device according to an embodiment of the present application;

[0022] Fig.10 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0025] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0026] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a computer device of a data synchronization method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data synchronization method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The nouns involved in the embodiments of the present application are explained as follows:

[0030] IO: Input and Output, input and output;

[0031] IOPS: Input / Output Operations Per Second, the number of input / output operations per second;

[0032] RDMA: Remote Direct Memory Access, remote direct memory access;

[0033] FC: Fibre Channel, fiber optic card;

[0034] SAS: Serial Attached SCSI (Small Computer System Interface), serial SCSI technology;

[0035] CA: Cache;

[0036] CACHE POOL: cache pool;

[0037] CPU: Central Processing Unit, central processing unit;

[0038] RAID: Redundant Arrays of Independent Disks, redundant array of disks;

[0039] NVME: Non-Volatile Memory Express, non-volatile memory fast storage technology;

[0040] Disk: disk.

[0041] In this application, the storage controller (equivalent to the main controller and the backup controller) is mainly responsible for the storage and management of data on the hard disk, and provides data access services to the server. In the process of providing data access (IO) to the server, the security of the data is generally ensured through the services of business modules such as CA, CA POOL, RAID, and multi-disk.

[0042] Figure 2 is an optional overall architecture diagram of a storage system according to an embodiment of the present application, such as Figure 2 As shown, a storage device generally consists of two controllers. The front end is connected to the server through technologies such as FC and RDMA, and the back end is connected to the storage disk (equivalent to the memory) through technologies such as SAS and NVME, forming a redundant system.

[0043] When an application on the server generates a write IO request (equivalent to a data write request), the request will first be sent to one of the controllers of the storage device, such as controller A (equivalent to the main controller) via a front-end card such as FC. Controller A recognizes the write IO request and then receives the data. After receiving the data, controller A does not immediately send a response to the host indicating the completion of the write IO request. Instead, it sends the write request data to controller B as backup redundancy through the internal network of the storage device, i.e., external card A and external card B between controller A and controller B. Then, a response to the completion of the write IO request is sent to the host. In this way, if controller A fails, the host (equivalent to the server) can find the originally stored data from controller B.

[0044] In this embodiment, a data synchronization method is provided. The server is connected to the storage through a main controller and a standby controller respectively. The main controller and the standby controller are connected to each other. The method is applied to the main controller. Figure 3 is a flow chart of a data synchronization method according to an embodiment of the present application. Figure 3 As shown, the process includes the following steps:

[0045] Step S12, receiving a data write request initiated by the server, wherein the data write request is used to request to write target data into a memory;

[0046] Optionally, in this embodiment, the memory may include a physical device for storing target data, such as a hard disk drive (HDD), a solid state drive (SSD), and a random access memory (RAM).

[0047] Step S14, responding to the data write request, detecting a write parameter of the target data, wherein the write parameter is used to indicate the urgency of writing the target data into the memory;

[0048] Optionally, in this embodiment, the data write request may carry, but is not limited to, target data and related parameters of the target data, and the target data may include, but is not limited to, various types of data such as database records and log files. Related parameters of the target data may include, but are not limited to, related parameters such as write parameters, addresses, and data volume of the target data.

[0049] Optionally, in this embodiment, the write parameter can be used to indicate the urgency of writing the target data to the memory, and different write parameters can be set for different data according to the specific business needs of the user. For example, if the target data 1 is transaction data with extremely high requirements for the real-time and accuracy of the data, the write parameter of the target data 1 can be set to a higher value, such as setting the write parameter of the target data 1 to 0.9, indicating that the urgency of writing the target data 1 to the memory is high. The target data 2 is log data with lower requirements for the real-time performance of the data, and the write parameter of the target data 2 can be set to a lower value, such as setting the write parameter of the target data 2 to 0.3, indicating that the urgency of writing the target data 2 to the memory is low. The present application does not impose any specific restrictions on the method for setting the write parameter.

[0050] Step S16, determining the calling timing of the target data calling the synchronization resource according to the write parameters;

[0051] Optionally, in this embodiment, the synchronization resources may include CPU resources, memory resources, external card link bandwidth resources, and other resources used to synchronize the target data to the standby controller.

[0052] Optionally, in this embodiment, the timing of calling the synchronization resource may include immediately calling the synchronization resource or not immediately calling the synchronization resource.

[0053] Step S18, calling the synchronization resource according to the calling timing, and synchronizing the target data to the standby controller, wherein the standby controller is used to cache the received data before storing it in the memory.

[0054] Optionally, in this embodiment, when the main controller fails, the standby controller can receive a data write request initiated by the server and write the data into the memory.

[0055] Through the embodiments of the present application, when the server initiates a data write request to the main controller, the main controller will detect the write parameters of the target data and determine the urgency of writing the target data to the memory; according to the urgency of writing the target data to the memory, the main controller can determine when to call the synchronization resource to synchronize the target data to the standby controller. That is, according to the urgency of writing the target data to the memory, different calling times are selected to call the synchronization resource to synchronize the target data to the standby controller, thereby achieving priority synchronization of urgent data, avoiding delays in urgent data processing, and reducing the risk of urgent data loss. Therefore, technical problems such as low efficiency of synchronization of data with a higher degree of urgency in related technologies can be solved, and the technical effect of improving the efficiency of synchronization of data with a higher degree of urgency can be achieved.

[0056] As an optional solution, determining the calling timing of the target data calling the synchronization resource according to the write parameter also includes:

[0057] S21, when it is detected that the write parameter falls within the parameter range, immediately call the synchronization resource to synchronize the target data to the standby controller;

[0058] S22, when it is detected that the write parameter does not fall within the parameter range, detecting whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, calling the synchronization resource to synchronize the target data to the standby controller.

[0059] Optionally, in this embodiment, the write parameter falling within the parameter range can be used to indicate that the urgency of writing the target data into the memory is high, and the synchronization resource needs to be called immediately to synchronize the target data to the standby controller.

[0060] Optionally, in this embodiment, the fact that the write parameter does not fall within the parameter range can be used to indicate that the urgency of writing the target data into the memory is low, and there is no need to immediately call the synchronization resource to synchronize the target data to the standby controller.

[0061] For example, the parameter range can be set to (0.8, 1). When the write parameter falls within (0.8, 1), it indicates that the urgency of writing the target data to the memory is high, and it is urgent data. It is necessary to immediately call the synchronization resource to synchronize the target data to the standby controller. When the write parameter does not fall within (0.8, 1), it indicates that the urgency of writing the target data to the memory is low, and it is non-urgent data. It is not necessary to immediately call the synchronization resource to synchronize the target data to the standby controller.

[0062] Optionally, in this embodiment, the parameter range can be defined according to the specific business needs of the user, and this application does not impose any restrictions on this.

[0063] Through the embodiment of the present application, the urgency of writing the target data to the memory is determined according to the write parameters and parameter ranges. When the urgency of writing the target data to the memory is high, the synchronization resource is immediately called to synchronize the target data to the standby controller, ensuring that the urgent data can be quickly synchronized to the standby controller, reducing data delays and improving data security. When the urgency of writing the target data to the memory is low, the synchronization resource is not immediately called to synchronize the target data to the standby controller, avoiding excessive consumption of resources and unnecessary synchronization, and improving resource utilization efficiency.

[0064] As an optional solution, when it is detected that the write parameter falls within the parameter range, the synchronization resource is immediately called to synchronize the target data to the standby controller, which also includes:

[0065] S31, adding the target data to the first priority queue;

[0066] S32, immediately call a first synchronization resource for the first priority queue, wherein the first priority queue is configured to synchronize data in the first priority queue to the standby controller through the first synchronization resource, and the synchronization resource includes the first synchronization resource.

[0067] Optionally, in this embodiment, the data in the first priority queue may be data with a higher urgency to be written into the memory.

[0068] Optionally, in this embodiment, a resource pool may be allocated to the first priority queue in the early stage of system initialization. The resources in the resource pool may be used for data synchronization, and the resources in the resource pool include the first synchronization resource.

[0069] As an optional solution, immediately calling the first synchronization resource for the first priority queue also includes:

[0070] S41, detecting the target data volume of the target data in the first priority queue;

[0071] S42, when the target data volume is greater than the target data volume threshold, generating a target quantity of the first unit resource according to the target data volume and the target data volume threshold, wherein the target data volume threshold is the amount of data allowed to be synchronized by a single first unit resource;

[0072] S43: Determine the target number of first unit resources as first synchronization resources.

[0073] Optionally, in this embodiment, the target data volume may be used to indicate the size of the target data. For example, the target data volume of target data 1 may be 10 MB, the target data volume of target data 2 may be 20 MB, and so on.

[0074] Optionally, in this embodiment, the first unit resource may be used to indicate a unit resource in a resource pool corresponding to the first priority queue, and the resource pool corresponding to the first priority queue may include multiple first unit resources.

[0075] Optionally, in this embodiment, the target data volume threshold may be the amount of data allowed to be synchronized by a single first unit resource. For example, if the amount of data allowed to be synchronized by a single first unit resource is 10 MB, the target data volume threshold may be set to 10 MB.

[0076] Optionally, in this embodiment, when the target data volume is less than or equal to the target data volume threshold, a single first unit resource is determined as the first synchronization resource. Taking the target data volume as 5MB and the target data volume threshold as 10MB as an example, the target data volume is less than the target data volume threshold, and the single first unit resource is determined as the first synchronization resource.

[0077] Figure 4 is a schematic diagram of data transmission of an optional first priority queue according to an embodiment of the present application, such as Figure 4 As shown, take the target data volume as 10MB and the target data volume threshold as 10MB as an example. First, it is detected that the target data volume of target data 1 in the first priority queue is 10MB. At this time, the target data volume is equal to the target data volume threshold. The single first unit resource is determined as the first synchronization resource, and the first unit resource A is called from the communication resource pool A. 1 , synchronize the target data to the standby controller through external card A.

[0078] Optionally, in this embodiment, when the target data volume is greater than the target data volume threshold, the target quantity of the first unit resource is generated based on the target data volume and the target data volume threshold, including: performing a division operation on the target data volume and the target data volume threshold to obtain a first candidate ratio; when the first candidate ratio is a positive integer, determining the first candidate ratio as the target quantity of the first unit resource; when the first candidate ratio is not a positive integer, determining the result obtained by rounding up the first candidate ratio as the target quantity of the first unit resource.

[0079] For example, when the target data volume is 20MB and the target data volume threshold is 10MB, the target data volume is greater than the target data volume threshold, and a division operation is performed on the target data volume and the target data volume threshold to obtain a first candidate ratio of 2, and the target quantity of the first unit resource is determined to be 2; when the target data volume is 25MB and the target data volume threshold is 10MB, the target data volume is greater than the target data volume threshold, and a division operation is performed on the target data volume and the target data volume threshold to obtain a first candidate ratio of 2.5, and the result of rounding up the first candidate ratio is 3, and the target quantity of the first unit resource is determined to be 3.

[0080] As an optional solution, when it is detected that the write parameter does not fall within the parameter range, detecting whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, calling the synchronization resource to synchronize the target data to the standby controller, further comprising:

[0081] S52, adding the target data to the second priority queue;

[0082] S53, detecting whether the target data in the second priority queue meets the synchronization condition;

[0083] S54, when it is detected that the target data meets the synchronization condition, calling the second synchronization resource for the second priority queue, wherein the second priority queue is configured to synchronize the data in the second priority queue to the standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

[0084] Optionally, in this embodiment, the data in the second priority queue may be data with a lower urgency to be written to the memory.

[0085] Optionally, in this embodiment, a resource pool may be allocated to the second priority queue in the early stage of system initialization. The resources in the resource pool may be used for data synchronization, and the resources in the resource pool include the second synchronization resources.

[0086] Optionally, in this embodiment, since the functions and components that the storage device needs to implement are very complex, the CPU, memory, external card link bandwidth, etc. are all precious resources. This solution takes these backgrounds as prerequisites and sets two priority queues at the initialization of the system, a high priority queue A (equivalent to the first priority queue) and a low priority queue B (equivalent to the second priority queue), and pre-allocates a certain amount of resources to each queue. The resource is actually a continuous memory of a fixed size. It is assumed that the size of each resource (equivalent to the first unit resource) in the resource pool of queue A is size_a (the amount of data allowed to be synchronized by a single first unit resource), and the size of each resource in the resource pool of queue B is size_b (the amount of data allowed to be synchronized by a single second unit resource), which is used to carry data to be sent to other controllers. Figure 5 is a schematic diagram of an optional communication queue resource allocation according to an embodiment of the present application, such as Figure 5 As shown, in the early stage of system initialization, a communication resource pool A is allocated to a high priority queue A, and the communication resource pool A includes a plurality of first unit resources, for example, resource A 1 、Resource A 2 ... Resource A n . Allocate communication resource pool B to low priority queue B, where communication resource pool B includes multiple second unit resources, for example, resource B 1 、Resource B2 ... Resource B n+m .

[0087] The pre-allocated amounts of the two queue resources can be represented by n and n+m, where n represents the number of first unit resources in resource pool A, and n+m represents the number of second unit resources in resource pool B. The number of first unit resources of the high priority queue can be set to be lower than the number of second unit resources of the low priority queue, which is not specifically limited in the present application.

[0088] As an optional solution, detecting whether the target data in the second priority queue meets the synchronization condition further includes at least one of the following:

[0089] S61, detecting whether the waiting time for the target data to be sent in the second priority queue is greater than or equal to the first time threshold; and when the waiting time is greater than or equal to the first time threshold, determining that the target data meets the first synchronization condition, wherein the synchronization condition includes the first synchronization condition;

[0090] S62, detecting whether the sending interval duration of the second priority queue where the target data is located is greater than or equal to a second duration threshold, wherein the sending interval duration is the interval duration from the current moment of the second priority queue to the last synchronization of data to the standby controller; and when the sending interval duration is greater than or equal to the second duration threshold, determining that the target data is detected to meet the second synchronization condition, wherein the synchronization condition includes the second synchronization condition;

[0091] S63, detect whether the reference data volume of all data in the second priority queue is greater than or equal to the reference data volume threshold; and when the reference data volume is greater than or equal to the reference data volume threshold, determine that the detected target data satisfies the third synchronization condition, wherein the synchronization condition includes the third synchronization condition.

[0092] Optionally, in this embodiment, the first duration threshold may be but is not limited to being used to indicate the maximum waiting time for target data to be sent in the second priority queue. The first duration threshold may be but is not limited to being set by the user according to needs. For example, the first duration threshold may be set to 500ms or 600ms, etc., and this application does not limit this.

[0093] Optionally, in this embodiment, the second duration threshold may be but is not limited to being used to indicate the maximum sending interval duration of the second priority queue where the target data is located. The second duration threshold may be but is not limited to being set by the user according to needs. For example, the second duration threshold may be set to 300ms or 200ms, etc., and this application does not limit this.

[0094] Optionally, in this embodiment, the reference data amount can be used to indicate the total data amount of all data in the second priority queue, and the reference data amount threshold can be used to indicate the maximum value allowed for the total data amount of all data in the second priority queue. For example, the reference data amount threshold can be set to the amount of data allowed to be synchronized by a single second unit resource or the amount of data allowed to be synchronized by n second unit resources.

[0095] Through the embodiments of the present application, the timing of calling the synchronization resource is determined based on the waiting time for the target data to be sent in the second priority queue, the sending interval time of the second priority queue where the target data is located, and the total amount of all data in the second priority queue. This allows for effective management of data synchronization in the second priority queue, avoids long-term backlog and delayed processing of data, ensures the rational allocation and utilization of system resources, and satisfies the need for regular data updates while maintaining stable operation and high efficiency of the system.

[0096] As an optional solution, calling the second synchronization resource for the second priority queue also includes:

[0097] S71, obtaining the candidate data amount of the data to be sent currently in the second priority queue, wherein the data to be sent currently includes the target data;

[0098] S72, when the candidate data amount is less than or equal to the candidate data amount threshold, determining a single second unit resource as a second synchronization resource, wherein the candidate data amount threshold is the amount of data allowed to be synchronized by the single second unit resource;

[0099] S73, when the candidate data amount is greater than the candidate data amount threshold, generate a candidate quantity of the second unit resource according to the candidate data amount and the candidate data amount threshold; and determine the candidate quantity of the second unit resource as the second synchronization resource.

[0100] Optionally, in this embodiment, generating the candidate quantity of the second unit resource based on the candidate data quantity and the candidate data quantity threshold includes: performing a division operation on the candidate data quantity and the candidate data quantity threshold to obtain a second candidate ratio; when the second candidate ratio is a positive integer, determining the second candidate ratio as the target quantity of the second unit resource; when the second candidate ratio is not a positive integer, determining the result obtained by rounding up the second candidate ratio as the target quantity of the second unit resource.

[0101] Optionally, in this embodiment, Figure 6 This is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application. Figure 1 ,like Figure 6As shown, taking the candidate data amount of the second priority queue currently to be sent as 10MB and the candidate data amount threshold as 10MB as an example, firstly, it is obtained that the candidate data amount of the second priority queue currently to be sent is 10MB, and it is determined that the candidate data amount 10MB is equal to the candidate data amount threshold 10MB, and the single second unit resource is determined as the second synchronization resource, and resource B is called from communication resource pool B. 1 , synchronize the current data to be sent to the standby controller through external card A.

[0102] Optionally, in this embodiment, Figure 7 This is a schematic diagram of data transmission of an optional second priority queue according to an embodiment of the present application. Figure 2 ,like Figure 7 As shown, taking the candidate data amount of the second priority queue currently to be sent as 15MB and the candidate data amount threshold as 10MB as an example, firstly, it is obtained that the candidate data amount of the second priority queue currently to be sent is 15MB, and it is judged that the candidate data amount 15MB is greater than the candidate data amount threshold 10MB, and a division operation is performed on the candidate data amount and the candidate data amount threshold, and the second candidate ratio is obtained as 1.5, and the second candidate ratio is rounded up to 2, and the target number of the second unit resource is determined to be 2, and the second unit resource B is called from the communication resource pool B. 1 and the second unit of resource B 2 , synchronize the current data to be sent to the standby controller through external card A.

[0103] As an optional solution, after calling the second synchronization resource for the second priority queue, the method further includes:

[0104] S81, generating a sending parameter of the second priority queue according to the candidate data amount of the data to be sent by the second priority queue, the target resource amount of the second synchronization resource, and a preset constant, wherein the sending parameter is used to indicate the current utilization rate of the second synchronization resource by the second priority queue, and the preset constant is used to indicate the minimum value allowed for the current utilization rate of the second synchronization resource by the second priority queue;

[0105] S82: Adjust the second duration threshold according to the sending parameter, the second duration threshold used last time, and the first duration threshold to obtain an adjusted second duration threshold.

[0106] Optionally, in this embodiment, the target resource amount of the second synchronization resource is used to indicate the amount of data allowed to be synchronized by the second synchronization resource.

[0107] Optionally, in this embodiment, the sending efficiency (equivalent to the sending parameters) can be counted after each time the sending process is started in queue B (equivalent to the second priority queue), that is, after the second synchronization resource is called for the second priority queue, and the overall waiting time threshold for the data to be sent in the queue (equivalent to the second duration threshold) can be updated.

[0108] As an optional solution, the sending parameters of the second priority queue are generated according to the candidate data amount of the data to be sent by the second priority queue, the target resource amount of the second synchronization resource, and the preset constant, including:

[0109] S91, performing a division operation on the target resource amount and the candidate data amount to obtain a target ratio;

[0110] S92: Determine the minimum value between the target ratio and the preset constant as the sending parameter.

[0111] Optionally, in this embodiment, Calculate the sending parameters, where ratio represents the sending parameters, size_B represents the amount of data allowed to be synchronized by the second synchronization resource, size represents the amount of candidate data to be sent by the second priority queue, and the preset constant is 2.

[0112] Optionally, in this embodiment, the preset constant can be a constant preset by the user according to demand. For example, the preset constant can be set to 2. The preset constant can be used to prevent the system from making overly aggressive adjustments to the synchronization mechanism under extremely low load or abnormal conditions, so that the system does not produce excessive jitter. For example, if the actual sending amount size is suddenly very small, resulting in a very high ratio of size_B / size (equivalent to the target ratio), the system may mistakenly believe that the data sending process needs to be started frequently, resulting in unnecessary resource consumption and system jitter. By setting the upper limit to 2, the system can moderately speed up data processing when resource utilization increases, but can remain stable and avoid excessive adjustment when resource utilization is low or the data volume fluctuates greatly.

[0113] Optionally, in this embodiment, the purpose of calculating the ratio of size_B / size is to measure whether the second synchronization resource is fully utilized in each data synchronization. When the ratio of size_B / size is equal to 1, it means that the second synchronization resource is effectively utilized; when the ratio of size_B / size is greater than 1, it means that there is still a certain amount of surplus of the second synchronization resource, and there is still room for improvement in the utilization rate of the second synchronization resource.

[0114] For example, the amount of data size_B allowed to be synchronized by the second synchronization resource is 10MB, the candidate data size of the current data to be sent in the second priority queue is 5MB, and the ratio of size_B / size is 2, indicating that there is still a certain amount of surplus in the second synchronization resource, and the utilization rate of the second synchronization resource can be further improved.

[0115] Optionally, in this embodiment, The adjusted second duration threshold is calculated, wherein time_thrshold_b represents the adjusted second duration threshold, time_thrshold_B represents the second duration threshold used last time, time_max_b represents the first duration threshold, and ratio represents the sending parameter.

[0116] Optionally, in this embodiment, the overall response time of the system, that is, the time from the sending of each business module data to the completion of the sending, can also be counted as a factor to adjust the queue sending time interval threshold (equivalent to the second time threshold). If the overall response time is found to be short, it indicates that the current data synchronization efficiency is high. At this time, the queue sending time interval can be appropriately increased to reduce the occupation of CPU and other resources and further optimize the system performance. On the contrary, if the overall response time is long, this may mean that there is a delay in the data communication link. The system should reduce the queue sending time interval and increase the frequency of data sending to clear the queue as soon as possible, reduce the waiting time of data in the queue, and thus improve the overall response speed. By adjusting the queue sending time interval through the overall response time of the system, it is possible to respond to different data communication conditions more sensitively, ensure the stability and reliability of data synchronization, and reduce the system performance degradation and potential failures caused by data backlog or excessive resource consumption.

[0117] Through the embodiments of the present application, in the case where the utilization rate of the second synchronization resource can be further improved, the utilization rate of the second synchronization resource can be improved by adjusting the second duration threshold. For example, when the amount of data size_B allowed to be synchronized by the second synchronization resource is 15MB, and the amount of candidate data size to be sent by the second priority queue is 10MB, the ratio of size_B / size is 1.5, indicating that there is still a certain amount of surplus in the second synchronization resource, and the second duration threshold can be increased. This adjustment means that controller A will allow data to wait longer in the second priority queue until the amount of candidate data approaches or reaches the carrying capacity size_B of the second synchronization resource, and then perform data synchronization. By dynamically adjusting the second duration threshold, the efficiency of each data synchronization operation can be maximized, the frequency of data synchronization can be reduced, and the waste of resources caused by frequently starting synchronization operations when there are surplus resources can be avoided.

[0118] As an optional solution, before calling the synchronization resource according to the calling timing and synchronizing the target data to the standby controller, the method further includes:

[0119] S101, detecting the remaining amount of remaining resources in the resource pool to which the synchronization resource belongs;

[0120] S102, when it is detected that the remaining amount of resources is less than or equal to a first resource threshold, adding resources to the resource pool;

[0121] S103: When it is detected that the remaining amount of resources is greater than a second resource threshold, release some resources in the resource pool.

[0122] Optionally, in this embodiment, both the synchronization resources and the remaining resources are resources that can be used to synchronize the target data to the standby controller.

[0123] Optionally, in this embodiment, the first resource threshold may be defined by the user according to demand. For example, the first resource threshold may be set to 80%.

[0124] Optionally, in this embodiment, the second resource threshold may be defined by the user according to demand. For example, the second resource threshold may be set to 20%.

[0125] Optionally, in this embodiment, when the system is initialized, a certain amount of resources will be pre-allocated to the resource pool of the two queues (the first priority queue and the second priority queue), but as the storage system undertakes changes in the front-end host IO requests (equivalent to data write requests), that is, as the IOPS changes, the resources in the resource pool also need to be dynamically allocated, that is, the resources in the resource pool need to change in a positive correlation with the actual needs such as IOPS, so that the system memory resources are fully utilized. IOPS can be used to measure the data processing capability of the system and the performance of the storage system. When IOPS increases, it means that the system needs to process more IO requests and the demand for resources increases; conversely, when IOPS decreases, the demand for resources decreases.

[0126] Optionally, in this embodiment, the process of adding resources to the resource pool and releasing some resources in the resource pool can be performed immediately after the queue completes a round of data transmission.

[0127] Through the embodiment of the present application, by real-time detection of the remaining amount of resources in the resource pool and comparing it with the preset first resource threshold and second resource threshold, the system can automatically identify resource shortage and resource surplus conditions. When resources are tight, resources are added to the resource pool in a timely manner to ensure smooth data synchronization operations and avoid data synchronization delays that may be caused by insufficient resources. When resources are in excess, resource release is initiated to release resources in the resource pool for use by other business modules, thereby maximizing resource utilization.

[0128] As an optional solution, when it is detected that the remaining resource amount is less than or equal to the first resource threshold, adding resources to the resource pool further includes:

[0129] S111, determining a first ratio between the second duration threshold used last time and the adjusted second duration threshold as a first load parameter;

[0130] S112, generating a first adjustment quantity according to a second ratio between the total amount of resources in the resource pool and the amount of resources possessed by a single second unit resource, a third ratio between the remaining amount of resources and the total amount of resources in the resource pool, and the first load parameter;

[0131] S113: Add a first adjusted quantity of second unit resources to the resource pool.

[0132] Optionally, in this embodiment, when the resource pool is in urgent need of resources, that is, each time the queue applies for resources from the resource pool, it will query the remaining resources in the pool. Generally, when the resource surplus (equivalent to the remaining resources) is lower than 80% of the total amount in the pool at initialization (equivalent to the total resources in the resource pool), it is necessary to apply for additional resources from the system. The specific application amount can be obtained through Determine. Among them, allocate_count represents the first adjustment quantity, that is, the specific application quantity, n+m represents the ratio of the total resource quantity of the resource pool to the resource quantity of a single second unit resource, ratio_count represents the ratio between the remaining resource quantity and the total resource quantity of the resource pool, time_thrshold_B represents the second duration threshold used last time, time_thrshold_b represents the adjusted second duration threshold, and time_thrshold_B / time_thrshold_b represents the first load parameter.

[0133] Optionally, in an embodiment of the present application, a third adjustment quantity can be generated based on the ratio of the total amount of resources in the resource pool to the amount of resources possessed by a single first unit resource, the ratio between the remaining amount of resources and the total amount of resources in the resource pool, and the ratio between the IOPS value processed by the current first priority queue and the preset IOPS value threshold processed by the first priority queue; the third adjustment quantity of first unit resources is added to the resource pool. By using the ratio of the IOPS value processed by the first priority queue and the preset IOPS value threshold processed by the first priority queue as a factor to add the third adjustment quantity of first unit resources to the resource pool, it is ensured that the allocation of resources is closely matched with the real-time business needs of the first priority queue, and even in the face of sudden high IOPS scenarios, the resource allocation can be quickly adjusted to enhance the system's ability to respond to emergencies and ensure data security and service continuity.

[0134] As an optional solution, when it is detected that the remaining amount of synchronization resources is greater than the second resource threshold, part of the resources in the resource pool are released, further comprising:

[0135] S121, determining a fourth ratio between the adjusted second duration threshold and the second duration threshold used last time as a second load parameter;

[0136] S122, generating a second adjustment quantity according to a fifth ratio of the total amount of resources in the resource pool to the amount of resources possessed by a single second unit resource, a sixth ratio between the remaining amount of resources and the total amount of resources in the resource pool, and a second load parameter;

[0137] S123: Release a second adjusted quantity of second unit resources for the resource pool.

[0138] Optionally, in this embodiment, when the system becomes less busy, it is necessary to release additional resources in the resource pool for use by other business modules. At the same time, to ensure system stability, the release of resources should also be in a step-by-step manner. Generally, resources are released only when the remaining resources exceed 20% of the initial value and the total resources in the resource pool exceed the initial value. First, The current system load is calculated, wherein ratio_load represents the second load parameter, time_thrshold_b represents the adjusted second time threshold, and time_thrshold_B represents the second time threshold used last time.

[0139] Optionally, in this embodiment, the amount of resource released each time can be Calculate. Where n+m represents the ratio of the total amount of resources in the resource pool to the amount of resources possessed by a single second unit resource, ratio_count represents the ratio between the remaining amount of resources and the total amount of resources in the resource pool, and ratio_load represents the second load parameter.

[0140] Optionally, in this embodiment, a fourth adjustment quantity can be generated based on the ratio of the total resources of the resource pool to the resources of a single first unit resource, the ratio of the remaining resources to the total resources of the resource pool, and the ratio of the IOPS value processed by the current first priority queue to the preset IOPS value threshold processed by the first priority queue; and the fourth adjustment quantity of first unit resources is released for the resource pool.

[0141] Optionally, in this embodiment, in order to better understand the process of synchronization of the above data, the synchronization process of the above data is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0142] Figure 8is a schematic diagram of an optional method for determining write parameters according to an embodiment of the present application, such as Figure 8 As shown, when the business senses data to be sent to other controller nodes (for example, controller B), it first determines its message type (equivalent to write parameters) by itself. When it is determined that the message type of the data is used to indicate that the urgency of writing the message to the memory is high, the data is added to queue A (equivalent to the first priority queue). When it is determined that the message type of the data is used to indicate that the urgency of writing the message to the memory is low, the data is added to queue B (equivalent to the second priority queue).

[0143] Optionally, in this embodiment, if the data to be sent is of an urgent and non-waitable type (equivalent to data with a high degree of urgency), it will be added to queue A (equivalent to the first priority queue). Queue A senses that there is data to be sent, uses a synchronous sending mechanism, does not stop, and immediately applies for a resource from resource pool A, stores the data to be sent in it, and submits the external card driver to send it to the other end. In particular, if the size of the data to be sent exceeds the carrying capacity size_a of a single resource (equivalent to the first unit resource), it will be split into multiple first unit resources for sending.

[0144] Optionally, in this embodiment, if the data to be sent is of other types (equivalent to data with a lower degree of urgency), the following assignment is first made: time_thrshold_b = time_thrshold_B.

[0145] Then the processing steps are as follows:

[0146] a) Add the data to be sent to queue B (equivalent to the second priority queue). When queue B senses the newly added data to be sent, it does not immediately start the data sending mechanism, but temporarily stores the data in the queue and uses the asynchronous sending mechanism to decouple the message sending process from the message generation process. This can reduce CPU usage and improve the overall response speed of the system.

[0147] b) Queue B polls the queue to see if there is any data to be sent that has been in the queue for longer than the waiting time threshold time_max_b (equivalent to the first time threshold). If so, it immediately applies for resource pool B. 1 It is packaged with other data in the queue and submitted to the external card A driver for sending. At the same time, the queue data sending time is updated, recorded as send_time_b.

[0148] c) If there is no retention time in queue B that exceeds the waiting time threshold time_max_b:

[0149] i. Queue B determines whether the current time from the last time the queue sent data is within the time threshold time_threshold_b (equivalent to the first duration threshold) based on the sending time send_time_b. If so, it immediately applies for resources from resource pool B (equivalent to the second synchronization resource) to package the waiting data in the queue and submit it to the external card A driver for sending. At the same time, the queue sends data. The time is updated, recorded as send_time_b.

[0150] ii. Queue B polls the data to be sent in the queue and counts the total amount of data sent, which is recorded as total_size_B (equivalent to the reference data volume threshold). If total_size_B is greater than or equal to size_B (equivalent to the reference data volume threshold), it immediately applies for resources from resource pool B to package the data to be sent in the queue and submit it to the external card A driver for sending. At the same time, the queue data sending time is updated, which is recorded as send_time_b. Repeat the above steps again until the condition that total_size_B is greater than or equal to size_B is no longer met;

[0151] This solution designs a communication mechanism for data synchronization between storage controllers, that is, between clusters. It sets up a first priority queue and a second priority queue to distinguish between urgent and non-urgent data. Urgent data is directly transmitted instantly through the high priority queue, while non-urgent data is called in the low priority queue based on the waiting time for a single message to be sent (equivalent to the first duration threshold), the overall waiting time for sending data (equivalent to the second duration threshold), and the maximum value allowed for the total amount of all data in the second priority queue. The synchronization resource is called to synchronize data to the standby controller, which reduces data latency, improves data security, and further improves the overall processing efficiency of IO services.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0153] Based on this understanding, the technical solution of the present application can essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, disk, CD-ROM), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0154] In this embodiment, a data synchronization device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0155] Fig. 9 is a structural block diagram of a data synchronization device according to an embodiment of the present application; Fig. 9 As shown, the server is connected to the storage through the main controller and the standby controller respectively, the main controller and the standby controller are connected to each other, the device is applied to the main controller, and the device includes:

[0156] The receiving module 1001 is used to receive a data writing request initiated by a server, wherein the data writing request is used to request to write target data into a memory;

[0157] A first detection module 1002, configured to respond to a data write request and detect a write parameter of target data, wherein the write parameter is used to indicate the urgency of writing the target data into the memory;

[0158] A determination module 1003 is used to determine a calling time of the target data calling the synchronization resource according to the write parameters;

[0159] The synchronization module 1004 is used to call the synchronization resource according to the calling timing to synchronize the target data to the standby controller, wherein the standby controller is used to cache the received data before storing it in the memory.

[0160] Through the above-mentioned embodiment, when the server initiates a data write request to the main controller, the main controller will detect the write parameters of the target data and determine the urgency of writing the target data to the memory; according to the urgency of writing the target data to the memory, the main controller can determine when to call the synchronization resource to synchronize the target data to the standby controller. That is, according to the urgency of writing the target data to the memory, different calling times are selected to synchronize the target data to the standby controller, thereby achieving priority synchronization of urgent data, avoiding delays in urgent data processing, and reducing the risk of urgent data loss. Therefore, technical problems in related technologies such as low efficiency of synchronization of data with a higher degree of urgency can be solved, and the technical effect of improving the efficiency of synchronization of data with a higher degree of urgency can be achieved.

[0161] In an exemplary embodiment, the determining module includes:

[0162] A first calling unit is used to immediately call the synchronization resource to synchronize the target data to the standby controller when detecting that the write parameter falls within the parameter range;

[0163] The second calling unit is used to detect whether the target data meets the synchronization condition when it is detected that the write parameter does not fall within the parameter range, and to call the synchronization resource to synchronize the target data to the standby controller when the target data meets the synchronization condition.

[0164] In an exemplary embodiment, the first calling unit is used to:

[0165] Add the target data to the first priority queue;

[0166] Immediately call a first synchronization resource for the first priority queue, wherein the first priority queue is configured to synchronize data in the first priority queue to the standby controller through the first synchronization resource, and the synchronization resource includes the first synchronization resource.

[0167] In an exemplary embodiment, the first calling unit is further configured to:

[0168] Detecting a target data volume of target data in a first priority queue;

[0169] When the target data volume is greater than the target data volume threshold, generating a target quantity of the first unit resource according to the target data volume and the target data volume threshold, wherein the target data volume threshold is the amount of data allowed to be synchronized by a single first unit resource;

[0170] A target number of first unit resources is determined as a first synchronization resource.

[0171] In an exemplary embodiment, the second calling unit is used to:

[0172] When it is detected that the write parameter does not fall within the parameter range, whether the target data meets the synchronization condition is detected. When the target data meets the synchronization condition, the synchronization resource is called to synchronize the target data to the standby controller, including:

[0173] Add the target data to the second priority queue;

[0174] Detecting whether the target data in the second priority queue meets the synchronization condition;

[0175] When it is detected that the target data meets the synchronization condition, a second synchronization resource is called for the second priority queue, wherein the second priority queue is configured to synchronize the data in the second priority queue to the standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

[0176] In an exemplary embodiment, the second calling unit is further configured to:

[0177] Detect whether the waiting time for the target data to be sent in the second priority queue is greater than or equal to the first time threshold; and when the waiting time is greater than or equal to the first time threshold, determine that the target data meets the first synchronization condition, wherein the synchronization condition includes the first synchronization condition;

[0178] Detect whether the sending interval duration of the second priority queue where the target data is located is greater than or equal to the second duration threshold, wherein the sending interval duration is the interval duration from the current moment of the second priority queue to the last synchronization of data to the standby controller; and when the sending interval duration is greater than or equal to the second duration threshold, determine that the target data is detected to meet the second synchronization condition, wherein the synchronization condition includes the second synchronization condition;

[0179] Detect whether the reference data volume of all data in the second priority queue is greater than or equal to the reference data volume threshold; and when the reference data volume is greater than or equal to the reference data volume threshold, determine that the target data is detected to meet the third synchronization condition, wherein the synchronization condition includes the third synchronization condition.

[0180] In an exemplary embodiment, the second calling unit is further configured to:

[0181] Obtaining the candidate data amount of the data to be sent currently in the second priority queue, wherein the data to be sent currently includes the target data;

[0182] When the candidate data amount is less than or equal to the candidate data amount threshold, determining the single second unit resource as the second synchronization resource, wherein the candidate data amount threshold is the amount of data allowed to be synchronized by the single second unit resource;

[0183] When the candidate data amount is greater than the candidate data amount threshold, generate a candidate number of second unit resources based on the candidate data amount and the candidate data amount threshold, wherein the candidate data amount threshold is the amount of data allowed to be synchronized by a single second unit resource; and determine the candidate number of second unit resources as the second synchronization resource.

[0184] In an exemplary embodiment, the apparatus further comprises:

[0185] A generation module, configured to generate, after calling the second synchronization resource for the second priority queue, a sending parameter of the second priority queue according to the candidate data amount of data to be sent by the second priority queue, the target resource amount of the second synchronization resource, and a preset constant, wherein the sending parameter is used to indicate the current utilization rate of the second synchronization resource by the second priority queue, and the preset constant is used to indicate the minimum value allowed for the current utilization rate of the second synchronization resource by the second priority queue;

[0186] The adjustment module is used to adjust the second duration threshold according to the sending parameter, the second duration threshold used last time and the first duration threshold to obtain the adjusted second duration threshold.

[0187] In an exemplary embodiment, a generation module includes:

[0188] An execution unit, configured to perform a division operation on a target resource amount and a candidate data amount to obtain a target ratio;

[0189] The first determining unit is used to determine the minimum value between the target ratio and the preset constant as the sending parameter.

[0190] In an exemplary embodiment, the apparatus further comprises:

[0191] A second detection module is used to detect the remaining resource amount of the remaining resources in the resource pool to which the synchronization resource belongs before calling the synchronization resource according to the calling timing and synchronizing the target data to the standby controller;

[0192] An adding module, configured to add resources to the resource pool when it is detected that the remaining amount of resources is less than or equal to a first resource threshold;

[0193] The release module is used to release part of the resources in the resource pool when it is detected that the remaining resource amount is greater than the second resource threshold.

[0194] In an exemplary embodiment, the adding module includes:

[0195] A second determining unit, configured to determine a first ratio between a second duration threshold used last time and the adjusted second duration threshold as a first load parameter;

[0196] A first generating unit is used to generate a first adjustment quantity according to a second ratio between the total amount of resources in the resource pool and the amount of resources possessed by a single second unit resource, a third ratio between the remaining amount of resources and the total amount of resources in the resource pool, and a first load parameter;

[0197] The adding unit is used to add a first adjusted quantity of second unit resources to the resource pool.

[0198] In an exemplary embodiment, the release module includes:

[0199] a third determining unit, configured to determine a fourth ratio between the adjusted second duration threshold and the second duration threshold used last time as a second load parameter;

[0200] A second generating unit, configured to generate a second adjustment quantity according to a fifth ratio between the total amount of resources in the resource pool and the amount of resources possessed by a single second unit resource, a sixth ratio between the remaining amount of resources and the total amount of resources in the resource pool, and a second load parameter;

[0201] The releasing unit is used to release a second adjusted quantity of second unit resources for the resource pool.

[0202] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0203] For the description of the features in the embodiment corresponding to the data synchronization device, reference may be made to the relevant description of the embodiment corresponding to the data synchronization method, which will not be described in detail here.

[0204] An embodiment of the present application further provides an electronic device, Fig.10 is a schematic diagram of an electronic device according to an embodiment of the present application, such as Fig.10 As shown, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above data synchronization method embodiments.

[0205] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0206] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0207] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data synchronization method embodiments when running.

[0208] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0209] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method in each embodiment of the present application; the computer program product also includes a non-volatile computer-readable storage medium, which stores the computer program, and, when executed by a processor, implements the steps of the data synchronization method in each embodiment of the present application.

[0210] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0211] The above is a detailed introduction to a data synchronization method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data synchronization method, characterized in that: The server is connected to the storage via a main controller and a standby controller respectively, the main controller and the standby controller are connected to each other, the method is applied to the main controller, and the method includes: receiving a data writing request initiated by the server, wherein the data writing request is used to request to write target data into the memory; In response to the data write request, detecting a write parameter of the target data, wherein the write parameter is used to indicate the urgency of writing the target data into the memory; Determine the calling time of the target data calling the synchronization resource according to the write parameter; The synchronization resource is called according to the calling timing to synchronize the target data to the standby controller, wherein the standby controller is used to cache the received data before storing it in the memory.

2. The method according to claim 1, characterized in that The step of determining the calling timing of the target data calling the synchronization resource according to the write parameter includes: When it is detected that the write parameter falls within the parameter range, immediately calling the synchronization resource to synchronize the target data to the standby controller; When it is detected that the write parameter does not fall within the parameter range, it is detected whether the target data meets the synchronization condition, and when the target data meets the synchronization condition, the synchronization resource is called to synchronize the target data to the standby controller.

3. The method according to claim 2, characterized in that When it is detected that the write parameter falls within the parameter range, immediately calling the synchronization resource to synchronize the target data to the standby controller includes: Adding the target data to a first priority queue; Immediately call a first synchronization resource for the first priority queue, wherein the first priority queue is configured to synchronize data in the first priority queue to the standby controller through the first synchronization resource, and the synchronization resources include the first synchronization resource.

4. The method according to claim 3, characterized in that Immediately calling a first synchronization resource for the first priority queue includes: Detecting a target data volume of the target data in the first priority queue; In the case where the target data volume is greater than the target data volume threshold, generating a target quantity of the first unit resource according to the target data volume and the target data volume threshold, wherein the target data volume threshold is the amount of data allowed to be synchronized by a single first unit resource; The target number of the first unit resources is determined as the first synchronization resource.

5. The method according to claim 2, characterized in that: The step of detecting whether the target data satisfies a synchronization condition when it is detected that the write parameter does not fall within the parameter range, and calling the synchronization resource to synchronize the target data to the standby controller when the target data satisfies the synchronization condition, includes: adding the target data to a second priority queue; Detecting whether the target data in the second priority queue meets the synchronization condition; When it is detected that the target data meets the synchronization condition, a second synchronization resource is called for the second priority queue, wherein the second priority queue is configured to synchronize the data in the second priority queue to the standby controller through the second synchronization resource, and the synchronization resource includes the second synchronization resource.

6. The method according to claim 5, characterized in that The detecting whether the target data in the second priority queue meets the synchronization condition includes at least one of the following: Detecting whether the waiting time for the target data to be sent in the second priority queue is greater than or equal to a first time threshold; and determining that it is detected that the target data meets a first synchronization condition when the waiting time is greater than or equal to the first time threshold, wherein the synchronization condition includes the first synchronization condition; Detect whether the sending interval duration of the second priority queue where the target data is located is greater than or equal to a second duration threshold, wherein the sending interval duration is the interval duration from the current moment of the second priority queue to the last synchronization of data to the standby controller; and when the sending interval duration is greater than or equal to the second duration threshold, determine that the target data is detected to meet a second synchronization condition, wherein the synchronization condition includes the second synchronization condition; Detect whether the reference data volume of all data in the second priority queue is greater than or equal to the reference data volume threshold; and when the reference data volume is greater than or equal to the reference data volume threshold, determine that the target data is detected to meet the third synchronization condition, wherein the synchronization condition includes the third synchronization condition.

7. The method according to claim 5, characterized in that The calling of the second synchronization resource for the second priority queue includes: Acquire the candidate data amount of the data to be sent currently in the second priority queue, wherein the data to be sent currently includes the target data; In a case where the candidate data amount is less than or equal to a candidate data amount threshold, determining a single second unit resource as the second synchronization resource, wherein the candidate data amount threshold is an amount of data allowed to be synchronized by a single second unit resource; In the case that the candidate data amount is greater than the candidate data amount threshold, a candidate quantity of the second unit resource is generated according to the candidate data amount and the candidate data amount threshold; and the candidate quantity of the second unit resource is determined as the second synchronization resource.

8. The method according to claim 6, characterized in that After calling the second synchronization resource for the second priority queue, the method further includes: Generate a sending parameter of the second priority queue according to the candidate data amount of the data to be sent by the second priority queue, the target resource amount of the second synchronization resource, and a preset constant, wherein the sending parameter is used to indicate the current utilization rate of the second synchronization resource by the second priority queue, and the preset constant is used to indicate the minimum value allowed for the current utilization rate of the second synchronization resource by the second priority queue; The second duration threshold is adjusted according to the sending parameter, the second duration threshold used last time, and the first duration threshold to obtain an adjusted second duration threshold.

9. The method according to claim 8, characterized in that The generating the sending parameter of the second priority queue according to the candidate data amount of the data to be sent by the second priority queue, the target resource amount of the second synchronization resource, and a preset constant includes: Performing a division operation on the target resource amount and the candidate data amount to obtain a target ratio; The minimum value between the target ratio and the preset constant is determined as the sending parameter.

10. The method according to claim 8, characterized in that Before calling the synchronization resource according to the calling timing to synchronize the target data to the standby controller, the method further includes: Detecting the remaining resource amount of the remaining resources in the resource pool to which the synchronization resource belongs; When detecting that the remaining amount of resources is less than or equal to a first resource threshold, adding resources to the resource pool; When it is detected that the remaining resource amount is greater than a second resource threshold, part of the resources in the resource pool are released.

11. The method according to claim 10, characterized in that When detecting that the remaining amount of resources is less than or equal to a first resource threshold, adding resources to the resource pool includes: determining a first ratio between the second duration threshold used last time and the adjusted second duration threshold as a first load parameter; Generate a first adjustment quantity according to a second ratio between the total amount of resources in the resource pool and the amount of resources possessed by a single second unit resource, a third ratio between the remaining amount of resources and the total amount of resources in the resource pool, and the first load parameter; The first adjusted quantity of the second unit resource is increased to the resource pool.

12. The method according to claim 10, characterized in that When detecting that the remaining amount of the synchronization resources is greater than a second resource threshold, releasing part of the resources in the resource pool includes: determining a fourth ratio between the adjusted second duration threshold and the second duration threshold used last time as a second load parameter; Generate a second adjustment quantity according to a fifth ratio of the total amount of resources in the resource pool to the amount of resources possessed by a single second unit resource, a sixth ratio between the remaining amount of resources and the total amount of resources in the resource pool, and the second load parameter; The second adjusted quantity of the second unit resource is released for the resource pool.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data synchronization method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data synchronization method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data synchronization method according to any one of claims 1 to 12 are implemented.

Images