Data transmission method, device, storage medium and computer program product

Abstract

The present application discloses a data transmission method, device, storage medium and computer program product, and is in the field of design storage technology. The method includes: determining a first target bitmap to be transmitted; determining a preset length, and dividing the first target bitmap into a plurality of first bitmap segments based on the preset length; judging whether all bit values ​​in the first bitmap segment are the same, if they are the same, compressing the first bitmap segment, and determining a bit value as the compression bit corresponding to the first bitmap segment; if they are not the same, determining the compression bit corresponding to the first bitmap segment based on all bit values ​​in the first bitmap segment; generating a first compression bitmap based on the compression bits corresponding to all first bitmap segments; generating a first mark bitmap based on the compression determination result of each first bitmap segment; and sending the first compression bitmap and the first mark bitmap to the opposite node. The present application improves the transmission efficiency of the bitmap.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and more specifically, to a data transmission method, device, storage medium and computer program product. Background Art

[0002] In storage systems, bitmaps are usually used to mark the execution status of tasks, and bitmaps need to be transmitted between nodes. Due to the large amount of bitmap data, it needs to be transmitted in multiple segments, resulting in a long time consumption on the transmission link and low transmission efficiency.

[0003] Therefore, how to improve the transmission efficiency of the bitmap is a technical problem that those skilled in the art need to solve. Summary of the invention

[0004] The purpose of the present application is to provide a data transmission method, device, storage medium and computer program product, which improve the transmission efficiency of bitmaps.

[0005] To achieve the above object, the present application provides a data transmission method, including:

[0006] determining a first target bitmap to be transmitted;

[0007] Determine a preset length, and divide the first target bitmap into a plurality of first bitmap segments based on the preset length;

[0008] Determine whether all bit values ​​in the first bitmap segment are the same; if they are the same, compress the first bitmap segment and determine one of the bit values ​​as a compression bit corresponding to the first bitmap segment; if they are not the same, determine all bit values ​​in the first bitmap segment as a compression bit corresponding to the first bitmap segment;

[0009] generating a first compressed bitmap based on the compression bits corresponding to all of the first bitmap segments;

[0010] Generate a first mark bitmap according to the compression determination result of each first bitmap segment; wherein, if the compression determination result of the first bitmap segment is that compression is performed, the corresponding mark bit in the first mark bitmap is a first preset value, and if the compression determination result of the first bitmap segment is that compression is not performed, the corresponding mark bit in the first mark bitmap is a second preset value;

[0011] The first compression bitmap and the first marking bitmap are sent to a peer node.

[0012] Wherein, determining whether all bit values ​​in the first bitmap segment are the same includes:

[0013] performing an AND operation and an OR operation on all bit values ​​in the first bitmap segment in sequence;

[0014] If all the AND operation results in the AND operation process and all the OR operation results in the AND or OR operation process are consistent with the first bit value in the first bitmap segment, then it is determined that all the bit values ​​in the first bitmap segment are the same;

[0015] Otherwise, it is determined that all bit values ​​in the first bitmap segment are different.

[0016] Among them, it also includes:

[0017] receiving a second compressed bitmap and a second marked bitmap;

[0018] Determine the first mark bit in the second mark bitmap as the current mark bit, and determine the first bit value in the second compressed bitmap as the current bit value;

[0019] Checking the value of the current flag bit;

[0020] If the current mark bit is the first preset value, a second bitmap segment of the preset length is generated based on the current bit value, and the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the next bit value in the second compressed bitmap is re-determined as the current bit value, and the step of checking the value of the current mark bit is re-entered; wherein all bit values ​​in the second bitmap segment are the current bit value;

[0021] If the current mark bit is the second preset value, a second bitmap segment of the preset length is generated based on the current bit value and subsequent bit values, and the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the bit value of the preset length after the current bit value in the second compressed bitmap is re-determined as the current bit value, and the step of checking the value of the current mark bit is re-entered;

[0022] A second target bitmap is generated based on all of the second bitmap segments.

[0023] Among them, each bit value in the first target bitmap is used to describe the execution information of the target task of the corresponding stripe, and the bit value of the first preset value indicates that the corresponding stripe has executed the target task, and the bit value of the second preset value indicates that the corresponding stripe has not executed the target task.

[0024] Among them, it also includes:

[0025] A ratio of a preset segment size to a preset strip size is calculated as the preset length.

[0026] Among them, it also includes:

[0027] When receiving a search command for a bit value corresponding to a target slice in the first target bitmap, determining a slice index of the target slice based on the search command;

[0028] Calculate a rounded-down value of the ratio of the stripe index to the preset length as a mark index, and determine a target mark bit in the first mark bitmap based on the mark index;

[0029] Determine a target bit index of the target stripe in the first compression bitmap based on the target flag bit;

[0030] A bit value corresponding to the target slice is determined in the first compressed bitmap based on the target bit index.

[0031] Wherein, determining the target bit index of the target stripe in the first compression bitmap based on the target flag bit includes:

[0032] If the target mark bit is the first preset value, determine the first mark bit quantity of the first preset value and the second mark bit quantity of the second preset value in the target interval in the first mark bitmap, calculate the first product of the second mark bit quantity and the preset length, and determine the sum of the first product and the first mark bit quantity as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

[0033] Wherein, determining the target bit index of the target stripe in the first compression bitmap based on the target flag bit includes:

[0034] If the target mark bit is the second preset value, determine the number of third mark bits of the first preset value and the number of fourth mark bits of the second preset value in the target interval in the first mark bitmap, calculate the second product of the fourth mark bit number and the preset length, determine the sum of the second product and the third mark bit number as the starting bit index, calculate the modulus value of the stripe index to the preset length, and determine the sum of the starting bit index and the modulus value as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

[0035] Among them, it also includes:

[0036] If it is necessary to perform a target operation on the first target bitmap, the target operation is performed on the first compressed bitmap to obtain an operation compressed bitmap, and the operation compressed bitmap is used as the compressed bitmap corresponding to the first target bitmap after the target operation is performed on the first target bitmap; wherein the target operation includes bit operations other than shift operations and merge operations.

[0037] Wherein, the data transmission method is applied to a first target node in a multi-controller system, the multi-controller system includes a plurality of controllers, each of the controllers corresponds to a node;

[0038] Accordingly, determining a first target bitmap to be transmitted includes:

[0039] When the first target node as a master node needs to send metadata information to a slave node, determining a bitmap corresponding to the management area of ​​the first target node as a first target bitmap;

[0040] Accordingly, sending the first compression bitmap and the first marking bitmap to the opposite node includes:

[0041] The first compression bitmap and the first marking bitmap are sent to the slave node as metadata information.

[0042] Among them, it also includes:

[0043] When the first target node serves as a slave node, if the master node corresponding to the first target node is a faulty node, the third compression bitmap of the management area of ​​the first target node and the fourth compression bitmap of the management area of ​​the faulty node are merged to obtain a fifth compression bitmap, the third marking bitmap of the management area of ​​the first target node and the fourth marking bitmap of the management area of ​​the faulty node are merged to obtain a fifth marking bitmap, and the management area of ​​the faulty node is taken over;

[0044] The first target node receives, as a slave node, a sixth compression bitmap and a sixth marking bitmap sent by the second target node; wherein the second target node is a master node corresponding to the faulty node when the faulty node is a slave node, and the sixth compression bitmap and the sixth marking bitmap are respectively a compression bitmap and a marking bitmap of a management area of ​​the second target node;

[0045] The first target node, as a master node, sends the fifth compression bitmap and the fifth marking bitmap as metadata information to the corresponding slave node.

[0046] Among them, it also includes:

[0047] If the faulty node is restored to a normal node, returning the fourth compression bitmap and the fourth marking bitmap to the normal node so that the normal node can take over the corresponding management area again;

[0048] The normal node again serves as a slave node to receive the sixth compression bitmap and the sixth marking bitmap sent by the second target node.

[0049] To achieve the above objectives, the present application provides an electronic device, including:

[0050] Memory for storing computer programs;

[0051] A processor is used to implement the steps of the above-mentioned data transmission method when executing the computer program.

[0052] To achieve the above-mentioned purpose, the present application provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned data transmission method are implemented.

[0053] To achieve the above objectives, the present application provides a computer program product, including a computer program, which implements the steps of the above data transmission method when executed by a processor.

[0054] It can be seen from the above scheme that a data transmission method provided by the present application includes: determining a first target bitmap to be transmitted; determining a preset length, and dividing the first target bitmap into multiple first bitmap segments based on the preset length; judging whether all bit values ​​in the first bitmap segment are the same, if they are the same, compressing the first bitmap segment, and determining one of the bit values ​​as the compression bit corresponding to the first bitmap segment; if they are not the same, determining the compression bit corresponding to the first bitmap segment for all bit values ​​in the first bitmap segment; generating a first compression bitmap based on the compression bits corresponding to all the first bitmap segments; generating a first mark bitmap according to the compression judgment result of each of the first bitmap segments; wherein, if the compression judgment result of the first bitmap segment is compression, the corresponding mark bit in the first mark bitmap is a first preset value, and if the compression judgment result of the first bitmap segment is not compression, the corresponding mark bit in the first mark bitmap is a second preset value; sending the first compression bitmap and the first mark bitmap to the opposite node.

[0055] The data transmission method provided by the present application shortens the bitmap length by marking continuous bitmap segments with the same bit value with a single bit value in units of a preset length, and uses a marking bitmap to identify whether the continuous bitmap segment of the preset length has been compressed, so as to distinguish between the continuous compressed part and the discontinuous uncompressed part, so that the bitmap can be compressed selectively. Since the data volume of the compressed bitmap and the marking bitmap is much smaller than the original bitmap to be transmitted, the amount of data transmitted can be reduced, thereby improving the transmission efficiency of the bitmap. The present application also discloses an electronic device, a non-volatile storage medium and a computer program product, which can also achieve the above technical effects.

[0056] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific implementation methods, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:

[0058] Figure 1 It is a schematic diagram of synchronization of metadata information in four control systems;

[0059] Figure 2 is a flow chart of a data transmission method according to an exemplary embodiment;

[0060] Figure 3 is a schematic diagram showing compression of a first target bitmap according to an exemplary embodiment;

[0061] Figure 4 is a schematic diagram showing another compression of a first target bitmap according to an exemplary embodiment;

[0062] Figure 5 The present invention is a flowchart of a method for decompressing a compressed bitmap according to an exemplary embodiment;

[0063] Figure 6 This is a flow chart of a method for searching a bit value corresponding to a stripe according to an exemplary embodiment;

[0064] Figure 7 is a flow chart of another data transmission method according to an exemplary embodiment;

[0065] Figure 8 A schematic diagram of synchronization of metadata in an application embodiment provided by the present application;

[0066] Fig. 9 A schematic diagram of a fault handling in an application embodiment provided by the present application;

[0067] Fig.10 A schematic diagram of metadata synchronization when node3 fails in an application embodiment provided by the present application;

[0068] Fig.11 A schematic diagram of fault recovery in an application embodiment provided by the present application;

[0069] Fig.12The figure is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0071] With the development of storage systems, the demand for storage device processing efficiency and fault tolerance continues to increase, and centralized multi-controller storage systems are increasingly used in various fields. Systems that interact between controllers in a circular mirroring manner, where nodes synchronize metadata information of fault-tolerant tasks in real time, rely on low communication between nodes and high-efficiency read and write processing methods to eliminate inter-controller locks, are gradually becoming an important part of multi-controller storage systems. Taking four control systems as an example, the synchronization of metadata information is as follows: Figure 1 shown.

[0072] In the above-mentioned multi-controller system, in order to ensure the order and accuracy of disk operations of each controller and to realize the diversion of read and write IO (Input / Output) pressure without locking, a strategy of limiting the access area of ​​each controller to the disk is adopted: by dividing the storage area according to a certain logic, each controller only executes read and write operations in the area assigned to them, and only executes background tasks in this part of the area.

[0073] In the multi-controller system in which the nodes above synchronize messages one-way between each other, when a node failure occurs, since some IO processing and metadata information of background tasks of the failed node are backed up on another surviving node being synchronized, this node can restore the unfinished read and write data processing of the failed node and can smoothly take over the business of the failed node, and continue to execute the background tasks of the memory area managed by the failed node based on the synchronized background task metadata information, thereby ensuring that the system's business processing and fault-tolerant recovery tasks are not affected during a failure.

[0074] In the above controller system, a domain (master-slave) system is composed of nodes and peer nodes that back up data to the nodes. The master node performs IO read and write operations in the domain, and the peer node of the master node backs up the execution information of the fault-tolerant tasks of the belonging area executed on the node generated by the master node in this process, including data bitmap information marking the execution status of the fault-tolerant tasks in the form of bitmaps, and marking the location and content of the temporary data that has not been successfully flushed. Since the belonging areas managed by each node do not overlap, in the above one-way two-by-two backup system, the metadata information of each node is only backed up on one node. When this node fails, the peer node in its domain takes over the fault-tolerant tasks. After the business of the failed node management area is taken over, this node will reorganize the domain with other nodes in the domain, and synchronize the metadata information of its own management area and the management area of ​​the newly taken over failed node to the peer node of the new group domain through metadata merging. At the same time, the failed node also saves the synchronized metadata information of the primary write node in the domain composed of it as the peer node and another node. After the failure, the domain is also reorganized, and the primary node in the original domain loses the backup protection of the peer node, so metadata merging is also required. In this process, the metadata information is synchronized to the node that takes over the failed node, and the synchronous closed loop of the circular mirror is re-formed.

[0075] RAID (Redundant Array of Independent Disks) refers to an array composed of a series of hard disks. The fault-tolerant operation and recovery management of the underlying disks are performed in RAID. The RAID that manages the operation of the hard disk array at the software layer is usually called soft RAID. At this level, RAID forms initialization, reconstruction, verification reconstruction, slow disk repair, inspection and other tasks that are separated from the foreground IO and executed in the background during the process of maintaining the consistency of disk information, so as to judge and supervise the occurrence of abnormal situations, as well as correct and recover the data after the abnormal situation. During the execution of these background tasks, RAID records the execution status of these background tasks by managing a piece of memory space. For some background tasks such as initialization and reconstruction, it involves recording whether all stripes on the array are executed, which is generally completed using a bitmap. In the bitmap data, each bit represents the execution status of a stripe.

[0076] Since the merge calculation needs to be performed in an orderly manner, in order to ensure the correctness of the bitmap data during the sending process, the orderly transmission of the bitmap needs to be guaranteed, that is, a bitmap cannot be sent asynchronously and in parallel during the sending process. This also causes the time spent in sending, receiving and merging all bitmaps between nodes to increase significantly according to the length of the bitmap.

[0077] During the execution of RAID background tasks, a bitmap is usually used to mark the execution status of the task. Usually, one bit is used to represent a stripe. Therefore, for the division of the ownership information of each controller and the initialization, reconstruction and other tasks, the identification bitmaps divided by stripes are stored separately. These bitmaps are arranged in sequence and stored in the memory space. At the same time, the merging of bitmaps is performed through bit operations. Therefore, the metadata merging process includes: the sending node sends segment information of a fixed length, the receiving node receives the segment information, the result is calculated through the merging bit operation, and the result is copied to the local memory.

[0078] It can be seen that in the related art, the number of bitmaps is large and occupies a lot of memory space. The transmission of the bitmap needs to be segmented multiple times, which results in longer transmission time on the transmission link and low transmission efficiency. Metadata cannot be operated during bitmap merging, and IO services cannot be processed. The long bitmap merging time leads to service interruption during controller failure, affecting user experience. The bitmap bit identification is in stripes, and it takes a long time to traverse the bitmap to find unexecuted stripes. The process of updating metadata after executing the task is even less efficient.

[0079] Therefore, the present application shortens the bitmap length by marking the bitmaps of the same value of continuous length with a single bit in fixed length units, and uses a secondary bitmap to identify whether the continuous fixed length bits have been compressed to distinguish the continuous compressed part from the discontinuous uncompressed part, so that the bitmap can be selectively compressed and accurately restored from less storage space. The present application can reduce the storage space occupied by the bitmap, save storage resources, reduce the consumption of space resources, reduce the time spent on sending, receiving, calculating and merging in the metadata merging process, improve the efficiency of metadata data transmission, reduce the IO silent time caused by the metadata merging process, and thus reduce the service interruption time after the controller failure occurs in the service processing process.

[0080] The embodiment of the present application discloses a data transmission method, which improves the transmission efficiency of the bitmap.

[0081] See also Figure 2 , a flowchart of a data transmission method according to an exemplary embodiment is shown, such as Figure 2 As shown, including:

[0082] S101: Determine a first target bitmap to be transmitted;

[0083] The execution subject of this embodiment is the target node in the storage system, which forms a domain system with the opposite node, with the target node as the master node and the opposite node as the slave node. In the storage system, the storage area is divided according to a certain logic and allocated to different nodes, that is, different nodes correspond to different management areas, and each node only performs the read and write operations of the corresponding management area and only performs the target tasks occurring in the corresponding management area. In order to realize a highly available storage system, it is necessary to redundantly store the metadata bitmap of the management area of ​​each node, that is, the target node needs to send the first target bitmap of its own management area to the opposite node to realize redundant backup of the first target bitmap. However, since the data volume of the first target bitmap is large, the data transmission method of this embodiment is adopted to compress the first target bitmap into a first compressed bitmap and a first marking bitmap and then transmit them to the opposite node.

[0084] As a feasible implementation, each bit value in the first target bitmap is used to describe the execution information of the target task of the corresponding strip. A bit value of the first preset value indicates that the corresponding strip has executed the target task, and a bit value of the second preset value indicates that the corresponding strip has not executed the target task.

[0085] In a specific implementation, the first target bitmap is used to describe the situation in which each strip in the management area of ​​the target node executes the target task. The first target bitmap includes multiple bits, each bit corresponds to a strip, and the bit value of each bit is used to describe the situation in which the corresponding strip executes the target task. The bit value of the first preset value indicates that the corresponding strip has executed the target task, and the bit value of the second preset value indicates that the corresponding strip has not executed the target task. The first preset value can be 1, and the second preset value can be 0.

[0086] S102: determining a preset length, and dividing the first target bitmap into a plurality of first bitmap segments based on the preset length;

[0087] In a specific implementation, the unit length of the bitmap division is pre-set, and the first target bitmap is divided into a plurality of first bitmap segments based on a preset length, and each first bitmap segment contains bits of a preset length. For example, if the length of the first target bitmap is 24 and the preset length is 8, the first target bitmap is divided into 3 first bitmap segments, and each first bitmap segment contains 8 bits.

[0088] As a feasible implementation, it further includes: calculating the ratio of the preset segment size to the preset stripe size as the preset length. In a specific implementation, the ratio of the preset segment size to the preset stripe size can be used as the preset length, the preset segment size can be set to 32MB, the preset stripe size can be the product of the preset block size and the preset stripe width, the preset block size can be set to 256KB, and the preset stripe width is related to the expected disk acquisition rate, which is the ratio of data to the check data and data.

[0089] S103: Determine whether all bit values ​​in the first bitmap segment are the same; if they are the same, compress the first bitmap segment and determine one of the bit values ​​as a compression bit corresponding to the first bitmap segment; if they are not the same, determine all bit values ​​in the first bitmap segment as a compression bit corresponding to the first bitmap segment;

[0090] In this step, it is determined in turn whether each first bitmap segment can be compressed. If all bit values ​​in the first bitmap segment are the same, it means that the first bitmap segment can be compressed, and the first bitmap segment is compressed to one of the bit values, that is, the compression bit corresponding to the first bitmap segment is set to a bit value in the first bitmap segment. If all bit values ​​in the first bitmap segment are not exactly the same, it means that the first bitmap segment cannot be compressed, and the compression bit corresponding to the first bitmap segment is set to the complete first bitmap segment.

[0091] As a feasible implementation method, determining whether all bit values ​​in the first bitmap segment are the same includes: performing AND operations and OR operations on all bit values ​​in the first bitmap segment in sequence; if all AND operation results in the AND operation process and all OR operation results in the OR operation process are consistent with the first bit value in the first bitmap segment, then determining that all bit values ​​in the first bitmap segment are the same; otherwise, determining that all bit values ​​in the first bitmap segment are not the same.

[0092] In a specific implementation, AND operations are performed on all bit values ​​in the first bit map segment in turn, and multiple AND operation results are obtained during the AND operation process. At the same time, OR operations are performed on all bit values ​​in the first bit map segment in turn, and multiple OR operation results are obtained during the AND operation process. If all the AND operation results and the OR operation results are consistent with the first bit value, then it is determined that all bit values ​​in the first bit map segment are the same, that is, the first bit map segment can be compressed, and the first bit map segment is compressed into the AND operation result, that is, the compression bit corresponding to the first bit map segment is set to the AND operation result; otherwise, it is determined that all bit values ​​in the first bit map segment are not completely the same, and the compression bit corresponding to the first bit map segment is set to the complete first bit map segment.

[0093] S104: Generate a first compressed bitmap based on the compression bits corresponding to all the first bitmap segments;

[0094] In this step, according to the order of the first bitmap segments in the first target bitmap, the compression bits corresponding to the first bitmap segments are concatenated into a first compressed bitmap.

[0095] S105: Generate a first mark bitmap according to the compression determination result of each of the first bitmap segments; wherein, if the compression determination result of the first bitmap segment is that compression is performed, then the corresponding mark bit in the first mark bitmap is a first preset value; if the compression determination result of the first bitmap segment is that compression is not performed, then the corresponding mark bit in the first mark bitmap is a second preset value;

[0096] In this step, a first mark bitmap is generated according to the compression judgment result of each first bitmap segment. The number of bits contained in the first mark bitmap is the number of the first bitmap segments. If the compression judgment result of the first bitmap segment is that compression is performed, then the corresponding mark bit in the first mark bitmap is a first preset value. If the compression judgment result of the first bitmap segment is that compression is not performed, then the corresponding mark bit in the first mark bitmap is a second preset value. The first preset value can be 1, and the second preset value can be 0.

[0097] For example, Figure 3 As shown, the first target bitmap is 1111111100000000011111111, the generated first compression bitmap is 101, and the first mark bitmap is 111. For another example, Figure 4 As shown, the first target bitmap is 1111111111010101011111111, the generated first compression bitmap is 1101010101, and the first mark bitmap is 101.

[0098] S106: Send the first compression bitmap and the first marking bitmap to the opposite node.

[0099] In this step, the target node sends the first compression bitmap and the first marking bitmap to the opposite node.

[0100] On the basis of the above embodiment, as a preferred implementation, after generating the first marking bitmap according to the compression determination result of each of the first bitmap segments, it also includes: deleting the first target bitmap, storing the first compressed bitmap and the first marking bitmap. In a specific implementation, after the target node compresses the first target bitmap, the first target bitmap can be deleted, and only the first compressed bitmap and the first marking bitmap are stored. Since the data volume of the first compressed bitmap and the first marking bitmap is much smaller than the data volume of the first target bitmap, the space occupied by the highly repetitive bitmap can be reduced, saving storage resources.

[0101] The data transmission method provided by the embodiment of the present application shortens the bitmap length by marking the continuous bitmap segments with the same bit value with a single bit value in units of a preset length, and uses a marking bitmap to identify whether the continuous bitmap segment of the preset length has been compressed, so as to distinguish the continuous compressed part from the discontinuous uncompressed part, so that the bitmap can be selectively compressed. Since the data volume of the compressed bitmap and the marking bitmap is much smaller than the original bitmap to be transmitted, the amount of data transmitted can be reduced, thereby improving the transmission efficiency of the bitmap.

[0102] The present application embodiment discloses a method for decompressing a compressed bitmap, specifically:

[0103] See also Figure 5 , a flowchart of a method for decompressing a compressed bitmap according to an exemplary embodiment is shown, such as Figure 5 As shown, including:

[0104] S201: receiving a second compression bitmap and a second marking bitmap;

[0105] The execution subject of this embodiment is the target node in the storage system, and the target node, as a slave node, receives the second compressed bitmap and the second marked bitmap sent by the corresponding master node. The purpose of this embodiment is to decompress the second compressed bitmap and the second marked bitmap into the original second target bitmap.

[0106] S202: Determine the first mark bit in the second mark bitmap as the current mark bit, and determine the first bit value in the second compressed bitmap as the current bit value;

[0107] S203: Check the value of the current flag bit;

[0108] S204: If the current flag bit is the first preset value, generate a second bitmap segment of the preset length based on the current bit value; wherein all bit values ​​in the second bitmap segment are the current bit value;

[0109] S205: Determine whether the current mark bit is the last mark bit of the second mark bitmap; if so, proceed to S210; if not, proceed to S206;

[0110] S206: re-determine the next mark bit in the second mark bitmap as the current mark bit, re-determine the next bit value in the second compressed bitmap as the current bit value, and re-enter S203;

[0111] S207: If the current flag bit is the second preset value, generating a second bitmap segment of the preset length based on the current bit value and subsequent bit values;

[0112] S208: Determine whether the current mark bit is the last mark bit of the second mark bitmap; if so, proceed to S210; if not, proceed to S209;

[0113] S209: re-determine the next mark bit in the second mark bitmap as the current mark bit, re-determine the bit value of the second compressed bitmap that is located after the current bit value by a preset length as the current bit value, and re-enter S203;

[0114] In a specific implementation, according to each mark bit in the second mark bitmap, the second bitmap segment corresponding to each bit value in the second compressed bitmap is decompressed in turn. First, the second mark bitmap is traversed, and each mark bit therein is determined as the current mark bit in turn. If the current mark bit is the first preset value, the current bit value in the second compressed bitmap is expanded to a second bitmap segment of a preset length, and all bit values ​​in the second bitmap segment are current bit values. Then, the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the next bit value in the second compressed bitmap is re-determined as the current bit value, and the subsequent bit values ​​in the second compressed bitmap are decompressed. If the current mark bit is the second preset value, a bitmap segment of a preset length is directly copied from the current bit value in the second compressed bitmap as the second bitmap segment, and then the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the next bit value at the copy end position in the second compressed bitmap is re-determined as the current bit value, and the subsequent bit values ​​in the second compressed bitmap are decompressed until the second compressed bitmap is completely decompressed to obtain multiple second bitmap segments.

[0115] S210: Generate a second target bitmap based on all the second bitmap segments.

[0116] In this step, according to the order of the bit values ​​in the second compressed bitmap, the second bitmap segments corresponding to the bit values ​​are concatenated into a second target bitmap.

[0117] It can be seen that this embodiment can achieve decompression from the compressed bitmap and the marked bitmap to obtain the original bitmap.

[0118] On the basis of the above embodiment, as a preferred implementation, it also includes: if it is necessary to perform a target operation on the first target bitmap, the target operation is performed on the first compressed bitmap to obtain an operation compression bitmap, and the operation compression bitmap is used as the compression bitmap corresponding to the first target bitmap after the target operation is performed on the first target bitmap; wherein, the target operation includes bit operations other than shift operations and merge operations.

[0119] In a specific implementation, if it is necessary to perform a target operation on the first target bitmap, the same target operation can be performed on the first target bitmap to obtain an operation compressed bitmap. After the operation compressed bitmap is decompressed, a bitmap that is exactly the same as the first target bitmap after the target operation can be obtained. That is, compression of the bitmap will not change the result of the target operation between the bitmaps, and the compressed bitmap maintains the same operation characteristics as before compression. The target operation here can be a bit operation other than a shift operation and a merge operation. The merge operation can be understood as a splicing operation on two bitmaps.

[0120] The present application embodiment discloses a method for searching bit values ​​corresponding to a stripe, specifically:

[0121] See also Figure 6 , a flowchart of a method for searching a bit value corresponding to a stripe according to an exemplary embodiment is shown, such as Figure 6 As shown, including:

[0122] S301: When receiving a search command for a bit value corresponding to a target slice in the first target bitmap, determining a slice index of the target slice based on the search command;

[0123] The purpose of this embodiment is to search for the bit value corresponding to the target stripe in the first target bitmap. In this step, the stripe index of the target stripe is determined.

[0124] S302: Calculate a rounded-down value of the ratio of the stripe index to the preset length as a mark index, and determine a target mark bit in the first mark bitmap based on the mark index;

[0125] In this step, the target mark bit corresponding to the first bitmap segment to which the target stripe belongs is determined in the first mark bitmap based on the stripe index and the preset length. First, the floor value of the ratio of the stripe index to the preset length is calculated as the mark index, and then the mark index is used to search for the target mark bit in the first mark bitmap.

[0126] For example, for Figure 4 For example, if the stripe index of the target stripe is 10, the calculated mark index is 1 (corresponding to the second bit value in the mark bitmap), and the corresponding target mark bit is 0. If the stripe index of the target stripe is 20, the calculated mark index is 2 (corresponding to the third bit value in the mark bitmap), and the corresponding target mark bit is 1.

[0127] S303: Determine a target bit index of the target stripe in the first compression bitmap based on the target flag bit;

[0128] In this step, a target bit index of the target stripe in the first compression bitmap is determined based on the target mark and the preset length.

[0129] As a feasible implementation method, the target bit index of the target stripe in the first compression bitmap is determined based on the target mark bit, including: if the target mark bit is the first preset value, determining the first mark bit quantity of the first preset value and the second mark bit quantity of the second preset value in the target interval in the first mark bitmap, calculating the first product of the second mark bit quantity and the preset length, and determining the sum of the first product and the first mark bit quantity as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

[0130] In a specific implementation, if the target mark bit is a first preset value, the target interval is determined to be the mark bit from the first mark bit in the first mark bit map to the mark bit before the target mark bit, and then the number of first mark bits of the first preset value and the number of second mark bits of the second preset value in the target interval are counted, and the first product of the number of second mark bits and the preset length is calculated, and the sum of the first product and the number of first mark bits is determined as the target bit index. It should be noted that when determining the target bit index, the target bit index can be initialized to 0 first, and then the mark bits in the first target interval are processed in sequence. The processing process is: multiply the mark bit by the preset length. If the result is 0, add the current target bit index to the result to update the current target bit index. If the result is not 0, add 1 to the current target bit index to update the current target bit index, until the target mark bit is processed and the final target bit index is obtained.

[0131] As a feasible implementation method, the target bit index of the target stripe in the first compression bitmap is determined based on the target mark bit, including: if the target mark bit is the second preset value, then determining the number of third mark bits of the first preset value and the number of fourth mark bits of the second preset value in the target interval in the first mark bitmap, calculating the second product of the fourth mark bit number and the preset length, determining the sum of the second product and the third mark bit number as the starting bit index, calculating the modulus value of the stripe index to the preset length, and determining the sum of the starting bit index and the modulus value as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

[0132] In a specific implementation, if the target mark bit is the second preset value, the target interval is determined to be the first mark bit in the first mark bitmap to the mark bit before the target mark bit, and then the number of third mark bits of the first preset value and the number of fourth mark bits of the second preset value in the target interval are counted, and the second product of the fourth mark bit number and the preset length is calculated, and the sum of the second product and the third mark bit number is determined as the starting bit index. The modulus value of the stripe index to the preset length is calculated, and the sum of the starting bit index and the modulus value is determined as the target bit index.

[0133] S304: Determine a bit value corresponding to the target stripe in the first compressed bitmap based on the target bit index.

[0134] In this step, the target bit index is used to search for a corresponding bit value in the first compressed bitmap, and the searched bit value is the bit value corresponding to the target stripe.

[0135] For example, for Figure 4 For example, if the stripe index of the target stripe is 10, the calculated mark index is 1, the target interval is 1, the first mark bit number is 1, the second mark bit number is 0, the start bit index is 1, the modulus value of the stripe index to the preset length, that is, the modulus value of 10 to 8 is 2, the calculated target bit index is 1+2=3, and the bit value corresponding to the target stripe is 1. If the stripe index of the target stripe is 20, the mark index is 2, the target interval is 10, the first mark bit number is 1, the second mark bit number is 1, the calculated target bit index is 1+(1×8)=9, and the bit value corresponding to the target stripe is 1.

[0136] It can be seen that by searching the compressed bitmap in the manner of this embodiment, the bit value corresponding to the target stripe can be searched, so that the compressed bitmap is in a usable state, that is, there is no need to decompress it into the original bitmap, and the bit value corresponding to the target stripe can be searched in the compressed bitmap.

[0137] The embodiment of the present application discloses a data transmission method in a multi-controller system, specifically:

[0138] See also Figure 7 , a flowchart of another data transmission method according to an exemplary embodiment is shown, such as Figure 7 As shown, including:

[0139] S401: When the first target node as a master node needs to send metadata information to the slave node, a bitmap corresponding to the management area of ​​the first target node is determined as a first target bitmap;

[0140] This embodiment is applied to a multi-controller system, which includes multiple controllers, each controller corresponds to a node, multiple nodes are connected in a ring, adjacent nodes form a domain system, each node is a master node, and the adjacent nodes connected to it, that is, the slave nodes, synchronize metadata information. This embodiment takes the first target node as an example for explanation, and the bitmap of the first target node management area is the first target bitmap.

[0141] S402: Determine a preset length, and divide the first target bitmap into a plurality of first bitmap segments based on the preset length;

[0142] S403: Determine whether all bit values ​​in the first bitmap segment are the same; if they are the same, compress the first bitmap segment and determine one of the bit values ​​as a compression bit corresponding to the first bitmap segment; if they are not the same, determine all bit values ​​in the first bitmap segment as a compression bit corresponding to the first bitmap segment;

[0143] S404: Generate a first compressed bitmap based on the compression bits corresponding to all the first bitmap segments;

[0144] S405: Generate a first mark bitmap according to the compression determination result of each of the first bitmap segments; wherein, if the compression determination result of the first bitmap segment is that compression is performed, then the corresponding mark bit in the first mark bitmap is a first preset value; if the compression determination result of the first bitmap segment is that compression is not performed, then the corresponding mark bit in the first mark bitmap is a second preset value;

[0145] S406: Send the first compression bitmap and the first marking bitmap as metadata information to the slave node.

[0146] In a specific implementation, the first target node compresses the first target bitmap into a first compression bitmap and a first marking bitmap, and sends them to the corresponding slave node as metadata information.

[0147] On the basis of the above embodiment, as a preferred implementation manner, it also includes: when the first target node acts as a slave node, if the master node corresponding to the first target node is a faulty node, the third compression bitmap of the management area of ​​the first target node and the fourth compression bitmap of the management area of ​​the faulty node are merged to obtain a fifth compression bitmap, the third mark bitmap of the management area of ​​the first target node and the fourth mark bitmap of the management area of ​​the faulty node are merged to obtain a fifth mark bitmap, and the management area of ​​the faulty node is taken over; the first target node acts as a slave node to receive the sixth compression bitmap and the sixth mark bitmap sent by the second target node; wherein the second target node is the master node corresponding to the faulty node when the second target node acts as a slave node, and the sixth compression bitmap and the sixth mark bitmap are respectively the compression bitmap and the mark bitmap of the management area of ​​the second target node; the first target node acts as a master node to send the fifth compression bitmap and the fifth mark bitmap as metadata information to the corresponding slave node.

[0148] In a specific implementation, if the master node corresponding to the first target node fails, that is, the node that synchronizes metadata to the first target node fails, the first target node merges the third compression bitmap of its own management area with the fourth compression bitmap of the faulty node management area to obtain a fifth compression bitmap, and merges the third mark bitmap of its own management area with the fourth mark bitmap of the faulty node management area to obtain a fifth mark bitmap, so that the first target node can take over the management area of ​​the faulty node. At the same time, the first target node, as the master node, synchronizes the fifth compression bitmap and the fifth mark bitmap to the corresponding slave node. In addition, the master node corresponding to the faulty node as a slave node is determined to be the second target node. The second target node, as the master node, synchronizes the sixth compression bitmap and the sixth mark bitmap of its own management area to the first target node.

[0149] On the basis of the above embodiment, as a preferred implementation manner, it also includes: if the faulty node is restored to a normal node, the fourth compression bitmap and the fourth mark bitmap are returned to the normal node so that the normal node can take over the corresponding management area again; wherein the normal node again serves as a slave node to receive the sixth compression bitmap and the sixth mark bitmap sent by the second target node.

[0150] In a specific implementation, if the faulty node is restored to a normal node, the first target node returns the metadata (the fourth compression bitmap and the fourth mark bitmap) of the previous management area of ​​the normal node to the normal node, and the normal node takes over the corresponding management area again. The second target node acts as the master node and synchronizes the sixth compression bitmap and the sixth mark bitmap of its own management area to the normal node.

[0151] It can be seen that this embodiment greatly reduces the space occupied by highly repetitive bitmaps by compressing bitmaps, saves storage resources, greatly shortens the data transmission time in the metadata merging process, greatly improves the efficiency of metadata merging, thereby reducing the interruption time of business processes and improving the user experience of storage devices.

[0152] The following describes an application embodiment provided by the present application. In this embodiment, the storage system is a four-controller system, including four controllers, and the corresponding nodes are node0, node1, node2, and node3, respectively. Figure 8 As shown, the metadata synchronization direction is node0→node1→node2→node3. In addition to storing the metadata of its own management area, each node also stores the metadata synchronized by other nodes. For example, for node0, it stores the metadata of the node0 management area and the metadata of the node3 management area.

[0153] Taking node0 synchronizing metadata to node1 as an example, node0 divides the bitmap of its own management area into multiple bitmap segments based on a preset length, and determines whether all bit values ​​in the bitmap segment are the same. If they are the same, the bitmap segment is compressed, and a bit value is determined as the compression bit corresponding to the bitmap segment; if they are not the same, all bit values ​​in the bitmap segment are determined as the compression bit corresponding to the bitmap segment, and a first compression bitmap is generated based on the compression bits corresponding to all bitmap segments, and a mark bitmap is generated according to the compression judgment result of each bitmap segment, wherein 1 indicates that the corresponding bitmap segment is compressed, and 0 indicates that the corresponding bitmap segment is not compressed. Node0 synchronizes the compressed bitmap and the mark bitmap as metadata to node1.

[0154] That is, the metadata stored in each node is a compression bitmap and a mark bitmap.

[0155] If node3 fails, such as Fig. 9 As shown in the figure, the shadow represents the faulty node. Node0 merges the metadata of its own management area with the metadata of node3's management area, that is, merges the compression bitmap of its own management area with the compression bitmap of node3's management area, and merges the mark bitmap of its own management area with the mark bitmap of node3's management area, so that node0 takes over the management area of ​​the faulty node node3, and synchronizes the merged metadata (merged compression bitmap and mark bitmap) to node1. In addition, node2 synchronizes the metadata (compression bitmap and mark bitmap) of its own management area to node0. The schematic diagram of metadata synchronization at this time is as follows Fig.10 shown.

[0156] If node3 is restored from a faulty node to a normal node, Fig.11 As shown in the figure, node0 returns the metadata (compression bitmap and mark bitmap) of node3's management area to node3, node3 takes over the corresponding management area again, node2 synchronizes the metadata (compression bitmap and mark bitmap) of its own management area to node3, and the metadata synchronization is restored to Figure 8 .

[0157] A data transmission device provided in an embodiment of the present application is introduced below. The data transmission device described below and the data transmission method described above can be cross-referenced. The data transmission device includes:

[0158] A first determining module, used to determine a first target bitmap to be transmitted;

[0159] a division module, configured to determine a preset length, and divide the first target bitmap into a plurality of first bitmap segments based on the preset length;

[0160] a compression module, configured to determine whether all bit values ​​in the first bitmap segment are the same; if they are the same, compress the first bitmap segment and determine one of the bit values ​​as a compression bit corresponding to the first bitmap segment; if they are not the same, determine all bit values ​​in the first bitmap segment as a compression bit corresponding to the first bitmap segment;

[0161] A first generating module, configured to generate a first compressed bitmap based on the compressed bits corresponding to all the first bitmap segments;

[0162] a second generating module, configured to generate a first mark bitmap according to a compression determination result of each of the first bitmap segments; wherein, if the compression determination result of the first bitmap segment is that compression is performed, then the corresponding mark bit in the first mark bitmap is a first preset value; and if the compression determination result of the first bitmap segment is that compression is not performed, then the corresponding mark bit in the first mark bitmap is a second preset value;

[0163] A sending module is used to send the first compression bitmap and the first marking bitmap to a peer node.

[0164] The data transmission device provided in the embodiment of the present application shortens the bitmap length by marking the continuous bitmap segments with the same bit value with a single bit value in units of a preset length, and uses a marking bitmap to identify whether the continuous bitmap segment of the preset length has been compressed, so as to distinguish the continuous compressed part from the discontinuous uncompressed part, so that the bitmap can be selectively compressed. Since the data volume of the compressed bitmap and the marking bitmap is much smaller than the original bitmap to be transmitted, the amount of data transmitted can be reduced, thereby improving the transmission efficiency of the bitmap.

[0165] Based on the above embodiments, as a preferred implementation mode, the compression module is specifically used to: perform AND operations and OR operations on all bit values ​​in the first bitmap segment in sequence; if all AND operation results in the AND operation process and all OR operation results in the OR operation process are consistent with the first bit value in the first bitmap segment, then it is determined that all bit values ​​in the first bitmap segment are the same; otherwise, it is determined that all bit values ​​in the first bitmap segment are not the same.

[0166] Based on the above embodiment, as a preferred implementation, it also includes:

[0167] A receiving module, used for receiving a second compressed bitmap and a second marked bitmap;

[0168] A second determination module, used to determine the first mark bit in the second mark bitmap as the current mark bit, and determine the first bit value in the second compressed bitmap as the current bit value;

[0169] A third determination module, used for checking the value of the current flag bit;

[0170] A decompression module, for generating a second bitmap segment of the preset length based on the current bit value if the current mark bit is the first preset value, and re-determining the next mark bit in the second mark bitmap as the current mark bit, and re-determining the next bit value in the second compressed bitmap as the current bit value, and restarting the workflow of the third determination module; wherein all bit values ​​in the second bitmap segment are the current bit value; if the current mark bit is the second preset value, generating a second bitmap segment of the preset length based on the current bit value and subsequent bit values, and re-determining the next mark bit in the second mark bitmap as the current mark bit, and re-determining the bit value of the preset length after the current bit value in the second compressed bitmap as the current bit value, and restarting the workflow of the third determination module;

[0171] The third generating module is used to generate a second target bitmap based on all the second bitmap segments.

[0172] Based on the above embodiments, as a preferred implementation mode, each bit value in the first target bitmap is used to describe the execution information of the target task of the corresponding strip, and the bit value of the first preset value indicates that the corresponding strip has executed the target task, and the bit value of the second preset value indicates that the corresponding strip has not executed the target task.

[0173] Based on the above embodiment, as a preferred implementation, it also includes:

[0174] The first calculation module is used to calculate the ratio of the preset segment size to the preset strip size as the preset length.

[0175] Based on the above embodiment, as a preferred implementation, it also includes:

[0176] A search module is used to determine the stripe index of the target stripe based on the search command when receiving a search command for the bit value corresponding to the target stripe in the first target bitmap; calculate the floor value of the ratio of the stripe index to the preset length as the mark index, and determine the target mark bit in the first mark bitmap based on the mark index; determine the target bit position index of the target stripe in the first compressed bitmap based on the target mark position; and determine the bit value corresponding to the target stripe in the first compressed bitmap based on the target bit position index.

[0177] On the basis of the above embodiments, as a preferred implementation mode, the search module is specifically used for: if the target mark bit is the first preset value, then determining the first mark bit quantity of the first preset value and the second mark bit quantity of the second preset value in the target interval in the first mark bitmap, calculating the first product of the second mark bit quantity and the preset length, and determining the sum of the first product and the first mark bit quantity as the target bit index; wherein, the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

[0178] On the basis of the above embodiment, as a preferred implementation mode, the search module is specifically used for: if the target mark bit is the second preset value, then determining the number of third mark bits of the first preset value and the number of fourth mark bits of the second preset value in the target interval of the first mark bitmap, calculating the second product of the fourth mark bit number and the preset length, determining the sum of the second product and the third mark bit number as the starting bit index, calculating the modulus value of the stripe index to the preset length, and determining the sum of the starting bit index and the modulus value as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit;

[0179] Based on the above embodiment, as a preferred implementation, it also includes:

[0180] An operation module is used for, if it is necessary to perform a target operation on the first target bitmap, performing the target operation on the first compressed bitmap to obtain an operation compressed bitmap, and using the operation compressed bitmap as the compressed bitmap corresponding to the first target bitmap after the target operation is performed on the first target bitmap; wherein the target operation includes bit operations other than shift operations and merge operations.

[0181] Based on the above embodiment, as a preferred implementation, the data transmission method is applied to a first target node in a multi-controller system, the multi-controller system includes a plurality of controllers, each of the controllers corresponds to a node;

[0182] Accordingly, the first determination module is specifically used to: when the first target node as the master node needs to send metadata information to the slave node, determine the bitmap corresponding to the management area of ​​the first target node as the first target bitmap;

[0183] Correspondingly, the sending module is specifically used to send the first compression bitmap and the first marking bitmap as metadata information to the slave node.

[0184] Based on the above embodiment, as a preferred implementation, it also includes:

[0185] A fault processing module is used for, when the first target node acts as a slave node, if the master node corresponding to the first target node is a faulty node, merging the third compression bitmap of the management area of ​​the first target node and the fourth compression bitmap of the management area of ​​the faulty node to obtain a fifth compression bitmap, merging the third mark bitmap of the management area of ​​the first target node and the fourth mark bitmap of the management area of ​​the faulty node to obtain a fifth mark bitmap, and taking over the management area of ​​the faulty node; the first target node acts as a slave node to receive the sixth compression bitmap and the sixth mark bitmap sent by the second target node; wherein the second target node is the master node corresponding to the faulty node when the second target node acts as a slave node, and the sixth compression bitmap and the sixth mark bitmap are respectively the compression bitmap and the mark bitmap of the management area of ​​the second target node; the first target node acts as a master node to send the fifth compression bitmap and the fifth mark bitmap as metadata information to the corresponding slave node.

[0186] Based on the above embodiment, as a preferred implementation, it also includes:

[0187] A fault recovery module is used to return the fourth compression bitmap and the fourth marking bitmap to the normal node if the faulty node is restored to a normal node, so that the normal node can take over the corresponding management area again; wherein the normal node again serves as a slave node to receive the sixth compression bitmap and the sixth marking bitmap sent by the second target node.

[0188] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0189] Based on the hardware implementation of the above program module, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, Fig.12 FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment. Fig.12 As shown, the electronic equipment includes:

[0190] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0191] The processor 2 is connected to the communication interface 1 to realize information exchange with other devices and is used to execute the data transmission method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 3.

[0192] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.12 Various buses are labeled as bus system 4.

[0193] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0194] It can be understood that the memory 3 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 3 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0195] The method disclosed in the above embodiment of the present application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 2 or the instruction in the form of software. The above processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 3, and the processor 2 reads the program in the memory 3 and completes the steps of the above method in combination with its hardware.

[0196] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented, which will not be repeated here for the sake of brevity.

[0197] In an exemplary embodiment, the embodiment of the present application further provides a non-volatile storage medium storing a computer program, which can be executed by the processor 2 to complete the steps of the aforementioned method.

[0198] In an exemplary embodiment, the present application also provides a computer program product, including a computer program, which is executed by the processor 2 to complete the steps of the aforementioned method.

[0199] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to computer program instructions, and the aforementioned computer program can be stored in a non-volatile storage medium. When the computer program is executed, it executes the steps of the above-mentioned method embodiment. Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product, which is stored in a non-volatile storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, a network device, etc.) to execute all or part of the methods described in each embodiment of the present application.

[0200] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A data transmission method, characterized in that: include: determining a first target bitmap to be transmitted; Determine a preset length, and divide the first target bitmap into a plurality of first bitmap segments based on the preset length; Determine whether all bit values ​​in the first bitmap segment are the same; if they are the same, compress the first bitmap segment and determine one of the bit values ​​as a compression bit corresponding to the first bitmap segment; if they are not the same, determine all bit values ​​in the first bitmap segment as a compression bit corresponding to the first bitmap segment; generating a first compressed bitmap based on the compression bits corresponding to all of the first bitmap segments; Generate a first mark bitmap according to the compression determination result of each first bitmap segment; wherein, if the compression determination result of the first bitmap segment is that compression is performed, the corresponding mark bit in the first mark bitmap is a first preset value, and if the compression determination result of the first bitmap segment is that compression is not performed, the corresponding mark bit in the first mark bitmap is a second preset value; Sending the first compression bitmap and the first marking bitmap to a peer node; Wherein, each bit value in the first target bitmap is used to describe the execution information of the target task of the corresponding stripe; The method further comprises: A ratio of a preset segment size to a preset strip size is calculated as the preset length.

2. The data transmission method according to claim 1, characterized in that: Determining whether all bit values ​​in the first bitmap segment are the same includes: performing an AND operation and an OR operation on all bit values ​​in the first bitmap segment in sequence; If all the AND operation results in the AND operation process and all the OR operation results in the AND or OR operation process are consistent with the first bit value in the first bitmap segment, then it is determined that all the bit values ​​in the first bitmap segment are the same; Otherwise, it is determined that all bit values ​​in the first bitmap segment are different.

3. The data transmission method according to claim 1, characterized in that: Also includes: receiving a second compressed bitmap and a second marked bitmap; Determine the first mark bit in the second mark bitmap as the current mark bit, and determine the first bit value in the second compressed bitmap as the current bit value; Checking the value of the current flag bit; If the current mark bit is the first preset value, a second bitmap segment of the preset length is generated based on the current bit value, and the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the next bit value in the second compressed bitmap is re-determined as the current bit value, and the step of checking the value of the current mark bit is re-entered; wherein all bit values ​​in the second bitmap segment are the current bit value; If the current mark bit is the second preset value, a second bitmap segment of the preset length is generated based on the current bit value and subsequent bit values, and the next mark bit in the second mark bitmap is re-determined as the current mark bit, and the bit value of the preset length after the current bit value in the second compressed bitmap is re-determined as the current bit value, and the step of checking the value of the current mark bit is re-entered; A second target bitmap is generated based on all of the second bitmap segments.

4. The data transmission method according to claim 1, characterized in that: In the first target bitmap, a bit value of the first preset value indicates that the corresponding stripe has executed the target task, and a bit value of the second preset value indicates that the corresponding stripe has not executed the target task.

5. The data transmission method according to claim 4, characterized in that: Also includes: When receiving a search command for a bit value corresponding to a target slice in the first target bitmap, determining a slice index of the target slice based on the search command; Calculate a rounded-down value of the ratio of the stripe index to the preset length as a mark index, and determine a target mark bit in the first mark bitmap based on the mark index; Determine a target bit index of the target stripe in the first compression bitmap based on the target flag bit; A bit value corresponding to the target slice is determined in the first compressed bitmap based on the target bit index.

6. The data transmission method according to claim 5, characterized in that: Determining a target bit index of the target stripe in the first compression bitmap based on the target flag bit includes: If the target mark bit is the first preset value, determine the first mark bit quantity of the first preset value and the second mark bit quantity of the second preset value in the target interval in the first mark bitmap, calculate the first product of the second mark bit quantity and the preset length, and determine the sum of the first product and the first mark bit quantity as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

7. The data transmission method according to claim 5, characterized in that: Determining a target bit index of the target stripe in the first compression bitmap based on the target flag bit includes: If the target mark bit is the second preset value, determine the number of third mark bits of the first preset value and the number of fourth mark bits of the second preset value in the target interval in the first mark bitmap, calculate the second product of the fourth mark bit number and the preset length, determine the sum of the second product and the third mark bit number as the starting bit index, calculate the modulus value of the stripe index to the preset length, and determine the sum of the starting bit index and the modulus value as the target bit index; wherein the target interval is from the first mark bit in the first mark bitmap to the mark bit before the target mark bit.

8. The data transmission method according to claim 1, characterized in that: Also includes: If it is necessary to perform a target operation on the first target bitmap, the target operation is performed on the first compressed bitmap to obtain an operation compressed bitmap, and the operation compressed bitmap is used as the compressed bitmap corresponding to the first target bitmap after the target operation is performed on the first target bitmap; wherein the target operation includes bit operations other than shift operations and merge operations.

9. The data transmission method according to claim 1, characterized in that: The data transmission method is applied to a first target node in a multi-controller system, the multi-controller system comprising a plurality of controllers, each of the controllers corresponding to a node; Accordingly, determining a first target bitmap to be transmitted includes: When the first target node as a master node needs to send metadata information to a slave node, determining a bitmap corresponding to the management area of ​​the first target node as a first target bitmap; Accordingly, sending the first compression bitmap and the first marking bitmap to the opposite node includes: The first compression bitmap and the first marking bitmap are sent to the slave node as metadata information.

10. The data transmission method according to claim 9, characterized in that: Also includes: When the first target node serves as a slave node, if the master node corresponding to the first target node is a faulty node, the third compression bitmap of the management area of ​​the first target node and the fourth compression bitmap of the management area of ​​the faulty node are merged to obtain a fifth compression bitmap, the third marking bitmap of the management area of ​​the first target node and the fourth marking bitmap of the management area of ​​the faulty node are merged to obtain a fifth marking bitmap, and the management area of ​​the faulty node is taken over; The first target node receives, as a slave node, a sixth compression bitmap and a sixth marking bitmap sent by the second target node; wherein the second target node is a master node corresponding to the faulty node when the faulty node is a slave node, and the sixth compression bitmap and the sixth marking bitmap are respectively a compression bitmap and a marking bitmap of a management area of ​​the second target node; The first target node, as a master node, sends the fifth compression bitmap and the fifth marking bitmap as metadata information to the corresponding slave node.

11. The data transmission method according to claim 10, characterized in that: Also includes: If the faulty node is restored to a normal node, returning the fourth compression bitmap and the fourth marking bitmap to the normal node so that the normal node can take over the corresponding management area again; The normal node again serves as a slave node to receive the sixth compression bitmap and the sixth marking bitmap sent by the second target node.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data transmission method according to any one of claims 1 to 11 when executing the computer program.

13. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 11 are implemented.

14. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the steps of the data transmission method according to any one of claims 1 to 11.

Images