Data transmission methods and electronic devices

Abstract

This application discloses a data transmission method and electronic device, relating to the field of computer technology. By generating a first log and a second log with interleaved polling cycles, the method alternately records the data written by the master end and the corresponding transmission status and synchronizes the logs. Retransmission is triggered only for unsuccessfully transmitted data indicated by the logs. This eliminates the need to periodically generate snapshots and compare snapshot differences, significantly reducing the consumption of storage and computing resources and effectively adapting to business scenarios with high real-time requirements.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data transmission method and electronic device. Background Technology

[0002] The centralized storage active-active system achieves real-time data backup through a remote synchronization mechanism between the master and slave ends. Specifically, after the master end records the newly added data, it sends the data to the slave end to complete the remote synchronization between the master and slave ends.

[0003] In related technologies, data loss during data transmission between the master and slave ends is solved by periodically generating data snapshots and comparing the differences between the snapshots to locate lost data. The lost data is then retransmitted from the master end to the slave end. However, when dealing with large amounts of data, the process of generating snapshots and comparing differences consumes a lot of storage and computing resources, which cannot meet the needs of business scenarios with high real-time requirements. Summary of the Invention

[0004] This application provides a data transmission method and electronic device to at least solve the problem that related technologies cannot adapt to business scenarios with high real-time requirements.

[0005] This application provides a data transmission method, including:

[0006] Generate a first log and a second log with interleaved polling cycles. The polling cycle includes a first period of recording data writing and a second period of recording the transmission status indicating whether the data has been successfully transmitted.

[0007] Based on the first log and the second log, the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end are recorded alternately, and the first log and the second log are synchronized.

[0008] When the transmission status of the first log and the second log indicates that there is second data that was not successfully transmitted, the master end controls the retransmission of the second data to the slave end.

[0009] This application also provides a data transmission device, including:

[0010] The first processing module is used to generate a first log and a second log with interleaved polling cycles. The polling cycle includes a first period of recording data writing and a second period of recording the transmission status indicating whether the data has been successfully transmitted.

[0011] The second processing module is used to alternately record the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end based on the first log and the second log, and to synchronize the data of the first log and the second log.

[0012] The third processing module is used to control the master end to retransmit the second data to the slave end when the transmission status of the first log and the second log indicates that there is second data that was not successfully transmitted.

[0013] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described data transmission methods when executing the computer program.

[0014] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data transmission methods.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data transmission methods.

[0016] This application enables the generation of a first log and a second log with interleaved polling cycles. These logs alternately record the data written by the master end and the corresponding transmission status, and synchronize the logs. Retransmission is triggered only for unsuccessfully transmitted data indicated by the logs. There is no need to periodically generate snapshots and compare snapshot differences, which greatly reduces the consumption of storage and computing resources and effectively adapts to business scenarios with high real-time requirements. Attached Figure Description

[0017] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An architecture diagram of a centralized storage active-active system provided in this application embodiment;

[0019] Figure 2 A flowchart illustrating a data transmission method provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating a method for recording transmission status provided in an embodiment of this application;

[0021] Figure 4 A flowchart illustrating a control method provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application;

[0023] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, other embodiments obtained by those of ordinary skill in the art without creative effort are all within the protection scope of this application.

[0025] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0026] The centralized storage active-active system achieves real-time data backup through a remote synchronization mechanism between the master and slave ends. Specifically, after the master end records the newly added data, it sends the data to the slave end to complete the remote synchronization between the master and slave ends.

[0027] In related technologies, data loss during data transmission between the master and slave ends is solved by periodically generating data snapshots and comparing the differences between the snapshots to locate lost data. The lost data is then retransmitted from the master end to the slave end. However, when dealing with large amounts of data, the process of generating snapshots and comparing differences consumes a lot of storage and computing resources, which cannot meet the needs of business scenarios with high real-time requirements.

[0028] In view of this, this application provides a data transmission method, including: generating a first log and a second log with interleaved polling cycles, wherein the polling cycle includes a first time period for recording data writing and a second time period for recording the transmission status indicating whether the data has been successfully transmitted; based on the first log and the second log, alternately recording the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end, and synchronizing the first log and the second log; when the transmission status of the first log and the second log indicates that there is first data that has not been successfully transmitted, controlling the master end to retransmit the first data to the slave end. In this method, by generating a first log and a second log with interleaved polling cycles, alternately recording the data written by the master end and the corresponding transmission status and synchronizing the logs, retransmission is triggered only for the unsuccessfully transmitted data indicated by the log, eliminating the need to periodically generate snapshots and compare snapshot differences, significantly reducing the consumption of storage and computing resources, effectively adapting to business scenarios with high real-time requirements, and ensuring the integrity and reliability of data synchronization between the master end and the slave end.

[0029] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] This section describes the specific application environment architecture or hardware architecture upon which the data transmission method depends. (References) Figure 1 , Figure 1 An architecture diagram of a centralized storage active-active system provided in this application embodiment includes: a master terminal, a slave terminal, and multiple controllers, wherein:

[0031] The primary and secondary ends are centralized storage nodes deployed in different locations, and the primary and secondary ends communicate with each other.

[0032] Multiple controllers include controller 1, controller 2 to controller N, where N is an integer greater than 2.

[0033] Multiple controllers communicate with the master and slave terminals respectively.

[0034] The master end is used to store newly added data and transmit newly added data to the slave end.

[0035] The auxiliary terminal is used to receive data transmitted by the master terminal and send a signal corresponding to the data to the master terminal. The signal corresponding to the data indicates that the auxiliary terminal has successfully received the data.

[0036] When the primary storage fails, the secondary storage can also be used to store new data, seamlessly taking over the storage needs of the original primary storage's business data and maintaining the service continuity of the centralized storage active-active system.

[0037] Multiple controllers are used to control the transmission of data from the master end to the slave end, and to receive signals corresponding to the data transmitted from the slave end to the master end.

[0038] Figure 2 This is a flowchart illustrating a data transmission method provided in an embodiment of this application, as shown below. Figure 2 As shown, embodiments of this application provide a data transmission method, which is described in detail below:

[0039] S201. Generate a first log and a second log with interleaved polling cycles. The polling cycle includes a first period of recording data writing and a second period of recording the transmission status indicating whether the data has been successfully transmitted.

[0040] In some embodiments, the duration of the polling cycle of the first log is the same as the duration of the polling cycle of the second log, and the start time of the polling cycle of the second log is delayed by half a polling cycle relative to the start time of the polling cycle of the first log.

[0041] The polling cycle is a fixed time interval for logging data, including a first time period and a second time period. The first time period is for recording newly added data written to the master, and the second time period is for recording the transmission status of whether the data was successfully transmitted.

[0042] The following is a detailed explanation of the first and second logs with interleaved polling cycles:

[0043] When the polling cycle duration is T;

[0044] The nth polling cycle of the first log is Correspondingly, the first period is The second period is ;

[0045] The nth polling cycle of the second log is Correspondingly, the first period is The second period is ;

[0046] Where n is an integer greater than or equal to 0.

[0047] The polling cycle duration T can be dynamically configured according to the real-time requirements of the business scenario: for high real-time scenarios (data synchronization latency < 10ms), T can be set between 5ms and 10ms; for scenarios with lower real-time requirements (data synchronization latency of 10ms-20ms is allowed), T can be set between 10ms and 20ms, so as to balance system resource consumption and business response efficiency.

[0048] It should be noted that, This represents a closed interval containing time a and time b. This represents a half-open interval that includes time a but does not include time b. This symbol represents a half-open interval that includes time b but does not include time a. This symbol definition conforms to the conventional expression of time intervals in mathematical and computer technology documents. Unless otherwise specified, this application will not elaborate on the definition of the time interval symbol.

[0049] The following provides an example of the first and second logs with interleaved polling cycles:

[0050] When the polling period is 10ms, the polling period of the first log starts at 0ms, and the polling period of the second log starts at 5ms.

[0051] The first polling cycle of the first log is The corresponding first time period is The corresponding second time period is The second polling cycle of the first log is The corresponding first time period is The corresponding second time period is And so on;

[0052] The first polling cycle of the second log is The corresponding first time period is The corresponding second time period is The second polling cycle of the second log is The corresponding first time period is The corresponding second time period is And so on.

[0053] In terms of hardware architecture, the first log and the second log correspond to two independent circular buffers. The size of each circular buffer is fixed at 2MB, and this fixed value can be dynamically adjusted through the upper-level configuration file to adapt to different application scenarios and performance requirements.

[0054] In some embodiments, generating a first log and a second log with interleaved polling cycles can be understood as creating two independent circular buffers in memory, which serve as the physical carriers of the first log and the second log, respectively.

[0055] The size of the circular buffer can be determined based on the amount of new data generated in the actual business scenario. For example, the size of the circular buffer can be 2MB.

[0056] Because the circular buffer has the storage characteristic of circular overwriting, when the circular buffer is full, the newly generated log data will start from the beginning of the circular buffer and overwrite the oldest data. This avoids the waste of storage resources caused by the infinite expansion of the buffer. It also fits the periodic staggered design of the first log and the second log, further improving the efficiency of data transmission and recording.

[0057] S202. Based on the first log and the second log, alternately record the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end, and synchronize the data of the first log and the second log.

[0058] The data written to the master can be understood as newly added data on the master, or newly stored data on the master.

[0059] Recording the data written to the master can be understood as recording the relevant information of the data written to the master. This relevant information includes the data identifier (data_id), logical block address (LBA), data length, version number, timestamp, and logical unit number (LUNID).

[0060] The data identifier is a unique code that distinguishes different data, ensuring that each data written to the master has a unique identifier and avoiding data confusion.

[0061] The logical block address is the logical storage location number corresponding to the data. The physical storage area of ​​the data in the host can be quickly located through this logical block address.

[0062] Data length refers to the amount of storage space occupied by the data written to the master, usually measured in bytes.

[0063] Version number is used to mark the number of times data has been updated. The version number is incremented when data is modified to prevent older version data from overwriting newer version data.

[0064] The timestamp is the timestamp when the data was written to the master.

[0065] The logical unit number (LUN ID) is used to identify an independent logical storage unit in the master storage node. A storage node can contain multiple logical units, and the specific storage unit to which the data belongs can be determined by the LUN ID.

[0066] For example, when writing data A to the master, relevant information about data A is recorded, including data_id, LBA (Logical Block Address), data length, version number, timestamp, and LUN ID.

[0067] In some embodiments, based on a first log and a second log, the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end are recorded alternately, including:

[0068] For any polling period of the first log;

[0069] During the first time period of the first log's polling cycle, the data added by the master end is recorded in the first log, and the polling cycle of the second log is determined during the second time period. The transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the second log.

[0070] During the second time period of the polling cycle of the first log, the transmission status corresponding to the first data transmitted from the master to the slave is recorded in the first log, and the polling cycle of the second log is determined within the first time period, and the data added by the master is recorded in the second log.

[0071] Determining that the polling period of the second log is within the second time period can be understood as determining that the second log is currently in the second time period of its polling period. Determining that the polling period of the second log is within the first time period can be understood as determining that the second log is currently in the first time period of its polling period.

[0072] In this embodiment of the application, determining the polling period of the second log within the second time period and determining the polling period of the second log within the first time period are to ensure that within the same time period, the data written to the master end is recorded in the first log, and the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the second log, or the data written to the master end is recorded in the second log, and the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the first log.

[0073] For any given polling cycle, when recording the transmission status of the first data transmitted from the master to the slave in the first or second log, the first data includes data added by the master during the first time period of the current polling cycle, and data whose transmission status is pending confirmation and has not been successfully transmitted. Specifically, when recording the transmission status of the first data transmitted from the master to the slave in the first log, the first data includes data added by the master during the first time period of the current polling cycle in the first log, and data whose transmission status is pending confirmation and has not been successfully transmitted at the current moment; when recording the transmission status of the first data transmitted from the master to the slave in the second log, the first data includes data added by the master during the first time period of the second log's polling cycle, and data whose transmission status is pending confirmation and has not been successfully transmitted.

[0074] In some embodiments, the transmission status corresponding to the first data includes: pending confirmation of unsuccessful transmission, confirmation of unsuccessful transmission, and confirmation of successful transmission.

[0075] If the transmission is not confirmed to be successful, it indicates that the master has initiated the transmission of the first data to the slave, but has not received the first signal corresponding to the first data from the slave within the preset timeout period. The first signal is the confirmation signal that the slave has received the first data.

[0076] If the transmission is confirmed to have failed, the master end is instructed not to receive the first signal corresponding to the first data from the slave end within a preset timeout period.

[0077] The system confirms successful transmission and instructs the master end to receive the first signal corresponding to the first data from the slave end within a preset timeout period.

[0078] In some embodiments, the preset timeout duration can be twice the duration of the polling cycle.

[0079] In some embodiments, during the second time period of the first log or the second log, the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded. This can be achieved by determining whether the first signal corresponding to the first data received from the slave end is received within a preset timeout period by executing the first number of times of "determining whether the first signal corresponding to the first data received from the slave end" reaches a preset threshold.

[0080] In some embodiments, data synchronization of the first log and the second log includes:

[0081] For the i-th polling cycle of the first log;

[0082] During the first time period of the i-th polling cycle of the first log, record the data added by the master during the first time period of the (i-1)-th polling cycle of the second log in the first log;

[0083] During the second time period of the i-th polling cycle of the first log, the data added by the master during the first time period of the i-th polling cycle of the first log is recorded in the second log;

[0084] Where i is an integer greater than 1.

[0085] In some embodiments, this applies to the first polling cycle;

[0086] During the first time period of the first polling cycle of the first log, newly added data from the master end is recorded in the first log;

[0087] During the second time period of the first polling cycle of the first log, the transmission status corresponding to the first data transmitted from the master to the slave is recorded in the first log. The first polling cycle of the second log is determined within the first time period, and the data added by the master is recorded in the second log. The data added by the master during the first time period of the first polling cycle of the first log is also recorded in the second log.

[0088] S203. When the transmission status of the first log and the second log indicates that there is first data that was not successfully transmitted, control the master to retransmit the first data to the slave.

[0089] In some embodiments, before controlling the master to retransmit the first data to the slave, the process includes: determining the first data.

[0090] In some embodiments, a mirror cache enables multiple controllers to share master-end data. This means the mirror cache stores data written to the master-end, and multiple controllers can retrieve this data from the mirror cache. Specifically, the mirror cache establishes a real-time data synchronization mechanism with the master-end: after the master-end receives new data and completes storage, it immediately writes the data to the mirror cache. When a controller needs to retrieve data from the master-end (e.g., to determine the first piece of data), it can directly read from the mirror cache. Furthermore, the mirror cache supports simultaneous read operations by multiple controllers without data contention. For example, if controller A needs to retransmit data A, it can quickly retrieve the complete content of data A from the mirror cache without waiting for the master-end, significantly improving the efficiency of the retransmission operation.

[0091] In some embodiments, determining the first data includes:

[0092] When the polling period of the first log is the first time period, determine the relevant information of the first data in the first log;

[0093] When the polling period of the second log is the first time period, determine the relevant information of the first data in the second log;

[0094] Based on the relevant information of the first data, the first data is determined in the image cache.

[0095] In this embodiment, a first log and a second log with interleaved polling cycles are generated. The polling cycle includes a first time period for recording data writing and a second time period for recording the transmission status indicating whether the data has been successfully transmitted. Based on the first log and the second log, the data written to the master end and the corresponding transmission status of the first data transmitted from the master end to the slave end are recorded alternately, and the first log and the second log are synchronized. When the transmission status of the first log and the second log indicates that there is first data that has not been successfully transmitted, the master end is controlled to retransmit the first data to the slave end. In the above method, by generating a first log and a second log with interleaved polling cycles, the data written by the master end and the corresponding transmission status are recorded alternately and the logs are synchronized. Retransmission is triggered only for the unsuccessfully transmitted data indicated by the log, eliminating the need to periodically generate snapshots and compare snapshot differences. This significantly reduces the consumption of storage and computing resources, effectively adapts to business scenarios with high real-time requirements, and ensures the integrity and reliability of data synchronization between the master end and the slave end.

[0096] Hereinafter, based on any of the above embodiments, by... Figure 3 Further explanation is given regarding "the transmission status corresponding to the first data transmitted from the master end to the slave end".

[0097] Figure 3 A flowchart illustrating a method for recording transmission status provided in an embodiment of this application is shown below. Figure 3 As shown, it includes:

[0098] S301, confirming whether the first data transmitted by the auxiliary terminal has been received, and confirming the first number of times the first signal for confirming whether the first data transmitted by the auxiliary terminal has been received.

[0099] Specifically, in this step, it is necessary to confirm whether the first signal of the first data transmitted by the auxiliary end has been received. Here, the first signal specifically refers to the confirmation signal of "first data received successfully" returned by the auxiliary end to the master end after successfully receiving the first data. It is also necessary to determine the cumulative number of times the above confirmation operation is performed (i.e., the first count). As long as the confirmation operation of the first signal of the first data is completed once, the corresponding first count will be automatically incremented by 1.

[0100] S302. If the first number is less than the preset threshold, when the first signal of the first data transmitted by the auxiliary end is received, the transmission status is recorded as confirmed as successful transmission; when the first signal of the first data transmitted by the auxiliary end is not received, the transmission status is recorded as pending confirmation as unsuccessful transmission.

[0101] The preset thresholds are set based on the real-time requirements of the business and the network environment.

[0102] If the first attempt has not yet reached the preset threshold, it means that the process is in the observation period of signal waiting, and the final stage of determining the transmission result has not yet been reached. In this scenario, the determination of the transmission status follows two rules: If the first signal of the first data fed back by the auxiliary end is successfully received during this confirmation process, it means that the first data has been successfully received by the auxiliary end, and the transmission status corresponding to the first data is recorded as confirmed successful transmission; if the first signal is not received during this confirmation, considering that the lack of signal may be due to temporary factors such as network latency or temporary high load of the auxiliary end, rather than the failure of data transmission itself, it is not directly determined as "transmission failure", but is recorded as the intermediate state of unsuccessful transmission pending confirmation, reserving space for subsequent reconfirmation.

[0103] Setting a preset threshold can reduce the probability of accidentally triggering retransmission or failure due to a single confirmation result, thus ensuring the reliability of data transmission status records.

[0104] S303. If the first count is equal to the preset threshold, then when the first signal of the first data transmitted by the auxiliary end is received, the transmission status is recorded as confirmed as successful transmission; when the first signal of the first data transmitted by the auxiliary end is not received, the transmission status is recorded as confirmed as unsuccessful transmission.

[0105] When the preset threshold is reached for the first time, the observation period for signal waiting officially ends. At this point, there is no need to reserve space for subsequent confirmations, and a deterministic judgment must be made based on the current confirmation result. In this scenario, the determination of the transmission status follows two rules: If the controller successfully receives the first signal of the first data from the auxiliary terminal during this confirmation process, the transmission status of the first data is recorded as confirmed as successfully transmitted; if the controller still does not receive the first signal of the first data from the auxiliary terminal during this confirmation, the influence of temporary factors such as network latency and excessive temporary load on the auxiliary terminal is excluded, and it is determined that there is a substantial fault in the transmission of the first data itself (such as loss during data transmission, abnormal storage on the auxiliary terminal, etc.), and the transmission status corresponding to the first data is recorded as confirmed as unsuccessful transmission.

[0106] In some embodiments, recording the transmission status corresponding to the first data transmitted from the master end to the slave end can also be:

[0107] A first signal confirming whether the first data transmitted from the slave device has been received;

[0108] Upon receiving the first signal of the first data transmitted from the auxiliary end, the transmission status is recorded as "transmission confirmed to be successful".

[0109] When the first signal of the first data transmitted by the auxiliary end is not received, the system confirms the first number of times whether the first signal of the first data transmitted by the auxiliary end has been received; when the first number is less than a preset threshold, the system records the transmission status as pending confirmation and unsuccessful transmission; when the first number is equal to the preset threshold, the system records the transmission status as confirmed unsuccessful transmission.

[0110] The following example, using Table 1, illustrates the scenario where the transmission status of recorded data A is "confirmed as unsuccessful" when the preset threshold is 4:

[0111] Table 1

[0112]

[0113] In the second time period of the first polling cycle of the first log, if the first signal confirming that the data A transmitted by the auxiliary end has not been received is confirmed, then the first number of times to confirm whether the first signal of the data A transmitted by the auxiliary end has been received is 1. If the first number is less than 4, then the transmission status corresponding to the data A is recorded as pending confirmation and unsuccessful transmission.

[0114] In the second time period of the first polling cycle of the second log, if the first signal of data A transmitted by the auxiliary end is not received, the first number of times to confirm whether the first signal of data A transmitted by the auxiliary end is received is 2. If the first number is less than 4, the transmission status of data A is recorded as pending confirmation and unsuccessful transmission.

[0115] In the second time period of the second polling cycle of the first log, the first signal confirming that the data A transmitted by the auxiliary end has not been received is confirmed. At this time, the first number of times to confirm whether the first signal of the data A transmitted by the auxiliary end has been received is 3. If the first number is less than 4, the transmission status corresponding to the data A is recorded as pending confirmation and unsuccessful transmission.

[0116] In the second time period of the second polling cycle of the second log, if the first signal of data A transmitted by the auxiliary end is not received, the first number of times to confirm whether the first signal of data A transmitted by the auxiliary end is received is 4. If the first number is less than 4, the transmission status corresponding to data A is recorded as confirmed as successfully transmitted.

[0117] If the transmission status of data A is confirmed as successfully transmitted, then it is not necessary to record the transmission status of data A again in the second time period of the second polling cycle of the first log.

[0118] It should be noted that " / " in Table 1 indicates that there are no record operations related to data A within the corresponding time period of the corresponding polling cycle (this table only describes the transmission status record of data A; non-data A content written to the master in other time periods is not discussed in this table), and does not mean that there are no data records.

[0119] It is understandable that if the log contains the transmission status corresponding to the first data, then recording the transmission status corresponding to the first data is to update the transmission status corresponding to the first data.

[0120] In this embodiment, the system confirms whether a first signal of first data transmitted from the auxiliary terminal has been received, and confirms the first number of times the system confirms whether the first signal of first data transmitted from the auxiliary terminal has been received. If the first number is less than a preset threshold, the transmission status is recorded as "confirmed successful transmission" when the first signal of first data transmitted from the auxiliary terminal is received; and the transmission status is recorded as "pending confirmation of unsuccessful transmission" when the first signal of first data transmitted from the auxiliary terminal is not received. If the first number is equal to the preset threshold, the transmission status is recorded as "confirmed successful transmission" when the first signal of first data transmitted from the auxiliary terminal is received; and the transmission status is recorded as "confirmed unsuccessful transmission" when the first signal of first data transmitted from the auxiliary terminal is not received. In the above method, by first confirming whether the first signal of the first data fed back by the auxiliary end has been received and determining the number of times the confirmation operation is performed, and recording the transmission status in stages in combination with the preset threshold (if the first signal is not received when the preset threshold is not reached, it is recorded as unsuccessful transmission pending confirmation; if the signal is not received when the preset threshold is reached, it is recorded as unsuccessful transmission confirmation), it not only avoids the situation of misjudging transmission failure due to a single confirmation or temporary network fluctuations, ensuring the accuracy of transmission status determination, but also realizes the confirmation and retransmission triggering of millisecond-level transmission results through the flexible configuration of the preset threshold, further improving the real-time performance of data synchronization. At the same time, with the log period staggered design, it takes into account the reliability of data transmission and the high real-time requirements of business scenarios under the premise of low resource consumption.

[0121] Hereinafter, based on any of the above embodiments, by... Figure 4 The following further clarifies the "control master terminal retransmits the second data to the slave terminal".

[0122] Figure 4 A flowchart illustrating a control method provided in an embodiment of this application is shown below. Figure 4 As shown, it includes:

[0123] S401. Obtain the processor utilization, memory usage, and link bandwidth utilization of multiple controllers.

[0124] Processor (Central Processing Unit, CPU) utilization refers to the percentage of effective work done by the processor within a unit of time after removing idle time. Specifically, it is the ratio of the sum of kernel-mode runtime and user-mode runtime to the total runtime. It can be periodically sampled and accurately calculated using the perf tool to ensure the accuracy of the value after removing idle time.

[0125] Memory utilization rate refers to the actual resource usage rate of the image cache. The calculation scope only includes the physical memory space of the stored data, and does not include the reserved memory pre-allocated for the image cache but not yet actually used, so as to avoid interference of pre-allocated space with load judgment.

[0126] Link bandwidth utilization rate refers to the ratio of the actual transmission rate of the data transmission link between the master and slave ends to the maximum rated bandwidth of the link. For example, if the master and slave ends use a 100Gbps transmission link, and the real-time transmission rate is 30Gbps, then the bandwidth utilization rate of the link is 30Gbps÷100Gbps=30%. This indicator directly reflects the busyness of the link.

[0127] S402. Determine the load of each controller based on the processor utilization, memory utilization, and link bandwidth utilization of each controller.

[0128] In some embodiments, for any controller, processor utilization, memory utilization, link bandwidth utilization, and load satisfy the following formula 1:

[0129] Formula 1;

[0130] in, Indicates load, Indicates processor utilization. Weights representing processor utilization Indicates memory usage. Weights representing memory usage Indicates link bandwidth utilization. The weight representing the link bandwidth utilization rate.

[0131] For example, For example, 0.4, For example, 0.3, For example, it is 0.3.

[0132] S403. Among multiple controllers, determine the first controller with the lowest load.

[0133] Specifically, after calculating the load of each controller using Formula 1, the loads of each controller are sorted in ascending order, and the load with the smallest value after sorting is extracted as the minimum load threshold. Then, the loads of each controller are traversed, and controllers whose loads are equal to the minimum load threshold are included in the category of the first controller. If only one controller has a load equal to the minimum load threshold, the number of first controllers is 1; if two or more controllers have loads equal to the minimum load threshold (i.e., the loads are the same), the number of first controllers is greater than 1.

[0134] S404. Based on the first controller, control the master end to retransmit the second data to the slave end.

[0135] In some embodiments, based on the first controller, controlling the master terminal to retransmit the second data to the slave terminal includes:

[0136] When the number of first controllers is equal to 1, the master terminal retransmits the second data to the slave terminal based on the first controller;

[0137] When the number of first controllers is greater than 1, the first controller with the lowest link bandwidth utilization is determined among the multiple first controllers; based on the first controller with the lowest link bandwidth utilization, the master end is controlled to retransmit the second data to the slave end.

[0138] In this embodiment, the processor utilization, memory utilization, and link bandwidth utilization of multiple controllers are obtained. Based on these metrics, the load of each controller is determined. Among the multiple controllers, the controller with the lowest load is identified. Based on the controller with the lowest load, the master end is controlled to retransmit the second data to the slave end. In this method, by collecting multi-dimensional load indicators such as processor utilization, memory utilization, and link bandwidth utilization, the load status of each controller can be accurately determined. The task of retransmitting the second data is assigned to the controller with the lowest load, preventing some controllers from overloading and blocking, avoiding single-point overload failures, and making use of idle resources to improve the overall system resource utilization. Simultaneously, the fully loaded first controller can efficiently handle the retransmission task, reducing waiting latency and ensuring the real-time synchronization of data between the master and slave ends, perfectly adapting to the high requirements of data consistency and transmission timeliness in a dual-active storage system.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0140] Figure 5 This is a schematic diagram of a data transmission device provided in an embodiment of this application. Figure 5 As shown, embodiments of this application also provide a data transmission device 50, wherein:

[0141] The first processing module 501 is used to generate a first log and a second log with interleaved polling cycles. The polling cycle includes a first period of recording data writing and a second period of recording the transmission status indicating whether the data has been successfully transmitted.

[0142] The second processing module 502 is used to alternately record the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end based on the first log and the second log, and to synchronize the data of the first log and the second log.

[0143] The third processing module 503 is used to control the master end to retransmit the second data to the slave end when the transmission status of the first log and the second log indicates that there is second data that was not successfully transmitted.

[0144] For a description of the features in the embodiment corresponding to the data transmission device, please refer to the relevant description in the embodiment corresponding to the data transmission method, which will not be repeated here.

[0145] In some embodiments, the duration of the polling cycle of the first log is the same as the duration of the polling cycle of the second log, and the start time of the polling cycle of the second log is delayed by half a polling cycle relative to the start time of the polling cycle of the first log.

[0146] In some embodiments, for any polling period of the first log; the second processing module 502 is specifically used for:

[0147] During the first period of the polling cycle, the data added by the master end is recorded in the first log, and the polling cycle of the second log is determined during the second period. The transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the second log.

[0148] During the second period of the polling cycle, the transmission status of the first data transmitted from the master to the slave is recorded in the first log, and the polling cycle of the second log is determined within the first period, and the data added by the master is recorded in the second log.

[0149] In some embodiments, the transmission status corresponding to the first data includes: pending confirmation of unsuccessful transmission, confirmation of unsuccessful transmission, and confirmation of successful transmission; the second processing module 502 is specifically used for:

[0150] A first signal to confirm whether the first data transmitted by the auxiliary terminal has been received, and a first number of times to confirm whether the first signal to confirm whether the first data transmitted by the auxiliary terminal has been received;

[0151] If the first number is less than the preset threshold, when the first signal of the first data transmitted by the auxiliary end is received, the transmission status is recorded as confirmed as successful transmission; when the first signal of the first data transmitted by the auxiliary end is not received, the transmission status is recorded as pending confirmation of unsuccessful transmission.

[0152] If the first count equals the preset threshold, then when the first signal of the first data transmitted by the auxiliary end is received, the transmission status is recorded as confirmed as successful transmission; if the first signal of the first data transmitted by the auxiliary end is not received, the transmission status is recorded as confirmed as unsuccessful transmission.

[0153] In some embodiments, the second processing module 502 is specifically used for:

[0154] For the i-th polling cycle of the first log;

[0155] During the first time period of the i-th polling cycle of the first log, record the data added by the master during the first time period of the (i-1)-th polling cycle of the second log in the first log;

[0156] During the second time period of the i-th polling cycle of the first log, the data added by the master during the first time period of the i-th polling cycle of the first log is recorded in the second log, where i is an integer greater than 1.

[0157] In some embodiments, the second processing module 502 is further configured to:

[0158] For the first polling cycle;

[0159] During the first time period of the first polling cycle of the first log, newly added data from the master end is recorded in the first log;

[0160] During the second time period of the first polling cycle of the first log, the transmission status corresponding to the first data transmitted from the master to the slave is recorded in the first log. The first polling cycle of the second log is determined within the first time period, and the data added by the master is recorded in the second log. The data added by the master during the first time period of the first polling cycle of the first log is also recorded in the second log.

[0161] In some embodiments, this applies to any single polling cycle;

[0162] The first log records the transmission status of the first data transmitted from the master to the slave. At this time, the first data includes the data added by the master during the first time period of the first log polling cycle, as well as the data whose transmission status is pending confirmation and has not been successfully transmitted.

[0163] The second log records the transmission status of the first data transmitted from the master to the slave. At this time, the first data includes the data added by the master during the first time period of the second log's polling cycle, as well as the data whose transmission status is pending confirmation and has not been successfully transmitted.

[0164] In some embodiments, the third processing module 503 is specifically used for:

[0165] Obtain the processor utilization, memory usage, and link bandwidth utilization of multiple controllers;

[0166] The load on each controller is determined based on its processor utilization, memory usage, and link bandwidth utilization.

[0167] Among multiple controllers, determine the first controller with the lowest load;

[0168] Based on the first controller, the master terminal is controlled to retransmit the second data to the slave terminal.

[0169] In some embodiments, the third processing module 503 is specifically used for:

[0170] When the number of first controllers is equal to 1, the master terminal retransmits the second data to the slave terminal based on the first controller;

[0171] When the number of first controllers is greater than 1, the first controller with the lowest link bandwidth utilization is determined among the multiple first controllers; based on the first controller with the lowest link bandwidth utilization, the master end is controlled to retransmit the second data to the slave end.

[0172] For a description of the features in the embodiment corresponding to the data transmission device, please refer to the relevant description in the embodiment corresponding to the data transmission method, which will not be repeated here.

[0173] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus.

[0174] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to execute the above-described data transmission method embodiment.

[0175] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0176] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0177] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0178] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0179] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data transmission method embodiments when it is run.

[0180] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0181] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data transmission method embodiments.

[0182] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data transmission method embodiments.

[0183] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of this application.

[0184] The data transmission method and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data transmission method, characterized in that, include: Generate a first log and a second log with interleaved polling cycles, wherein the polling cycle includes a first period of recording data writing and a second period of recording the transmission status indicating whether the data has been successfully transmitted; Based on the first log and the second log, the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end are recorded alternately, and the first log and the second log are synchronized. When the transmission status of the first log and the second log indicates that there is second data that was not successfully transmitted, the master terminal is controlled to retransmit the second data to the slave terminal.

2. The method according to claim 1, characterized in that, The duration of the polling cycle for the first log is the same as the duration of the polling cycle for the second log, and the start time of the polling cycle for the second log is delayed by half a polling cycle relative to the start time of the polling cycle for the first log.

3. The method according to claim 1, characterized in that, For any polling cycle of the first log; the step of alternately recording the data written to the master end and the transmission status corresponding to the first data transmitted from the master end to the slave end based on the first log and the second log includes: During the first time period of the polling cycle, the data newly added by the master end is recorded in the first log, and the polling cycle of the second log is determined during the second time period, and the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the second log. During the second time period of the polling cycle, the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the first log, and the polling cycle of the second log is determined during the first time period, and the data added by the master end is recorded in the second log.

4. The method according to claim 3, characterized in that, The transmission status corresponding to the first data includes: pending confirmation of unsuccessful transmission, confirmed unsuccessful transmission, and confirmed successful transmission; the recording of the transmission status corresponding to the first data transmitted from the master end to the slave end includes: A first signal to confirm whether the first data transmitted by the auxiliary terminal has been received, and a first number of times to confirm whether the first signal to confirm whether the first data transmitted by the auxiliary terminal has been received; If the first number of times is less than a preset threshold, then when the first signal of the first data transmitted by the auxiliary terminal is received, the transmission status is recorded as confirmed as successful transmission; when the first signal of the first data transmitted by the auxiliary terminal is not received, the transmission status is recorded as pending confirmation of unsuccessful transmission. If the first number of times equals a preset threshold, then when the first signal of the first data transmitted by the auxiliary terminal is received, the transmission status is recorded as confirmed as successful transmission; when the first signal of the first data transmitted by the auxiliary terminal is not received, the transmission status is recorded as confirmed as unsuccessful transmission.

5. The method according to claim 1, characterized in that, The data synchronization of the first log and the second log includes: For the i-th polling cycle of the first log; During the first time period of the i-th polling cycle of the first log, the data added by the master terminal during the first time period of the (i-1)-th polling cycle of the second log is recorded in the first log; During the second time period of the i-th polling cycle of the first log, the data added by the master during the first time period of the i-th polling cycle of the first log is recorded in the second log, where i is an integer greater than 1.

6. The method according to claim 5, characterized in that, The method further includes: For the first polling cycle; During the first time period of the first polling cycle of the first log, the data newly added by the master terminal is recorded in the first log; During the second time period of the first polling cycle of the first log, the transmission status corresponding to the first data transmitted from the master end to the slave end is recorded in the first log. The first polling cycle of the second log is determined within the first time period, and the data added by the master end is recorded in the second log. The data added by the master end within the first time period of the first polling cycle of the first log is also recorded in the second log.

7. The method according to any one of claims 1-6, characterized in that, For any given polling cycle; The first log records the transmission status of the first data transmitted from the master end to the slave end. At this time, the first data includes the data added by the master end during the first time period of the first log polling cycle, and the data whose transmission status is pending confirmation and has not been successfully transmitted. The transmission status of the first data transmitted from the master end to the slave end is recorded in the second log. At this time, the first data includes the data added by the master end during the first time period of the second log's polling cycle, as well as the data whose transmission status is pending confirmation and has not been successfully transmitted.

8. The method according to claim 6, characterized in that, The control of the master terminal to retransmit the second data to the slave terminal includes: Obtain the processor utilization, memory usage, and link bandwidth utilization of multiple controllers; The load of each controller is determined based on the processor utilization, memory usage, and link bandwidth utilization of each controller. Among the plurality of controllers, the first controller with the lowest load is determined; Based on the first controller, the master terminal is controlled to retransmit the second data to the slave terminal.

9. The method according to claim 8, characterized in that, The step of controlling the master terminal to retransmit the second data to the slave terminal based on the first controller includes: When the number of the first controllers is equal to 1, the master terminal is controlled to retransmit the second data to the slave terminal based on the first controller; When the number of first controllers is greater than 1, the first controller with the lowest link bandwidth utilization is determined among the plurality of first controllers; based on the first controller with the lowest link bandwidth utilization, the master end is controlled to retransmit the second data to the slave end.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data transmission method as described in any one of claims 1 to 9 when executing the computer program.

Images