Data write method and apparatus for distributed storage system

Abstract

The application discloses a data writing method and device for a distributed storage system, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units, the method executed by the master segment unit comprising: receiving data and a unique identifier from the data driver, the data driver generating the unique identifier for the data; generating a distinguishing identifier for the unique identifier; broadcasting the unique identifier and the distinguishing identifier to each slave segment unit, wherein the slave segment unit receives the data and the unique identifier from the data driver; receiving a response of processing the data from each slave segment unit and sending the response to the data driver and the device. The application realizes high throughput and low delay while maintaining strong consistency of data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data storage, and in particular to a data writing method and device for a distributed storage system, a storage medium, and an electronic device. BACKGROUND

[0002] With the advent of the big data era, the amount of user business data has significantly increased. Distributed storage systems need to meet the requirements of high throughput and low latency of users in addition to ensuring basic features such as high reliability, high availability, and strong consistency.

[0003] A large-scale distributed storage system is often composed of a large number of segments, each segment being composed of segment units from different machines. Each segment unit can be a different device on a different machine, such as a certain segment of storage space on a storage medium. When user data is written to a segment through a data driver, the data driver needs to write data to each segment unit in the segment to achieve data backup. Therefore, the data driver needs to broadcast the requested data to these segment units at the same time.

[0004] Under the concurrent condition of user business data, multiple data drivers will send write data requests to the same segment at the same time. In order to ensure the strong consistency of user data, the segment units need to perform sorting and recording when receiving the data.

[0005] Current data writing methods are divided into two categories:

[0006] I. When a data driver receives a write data request, it generates a unique identifier for the data and binds the data to the unique identifier, and broadcasts it to all group members. Then, it collects the response information of all members, and finally responds to the user request.

[0007] II. When a data driver receives a write data request, it sends it to the master segment unit in the segment. The master segment unit generates a sequence identifier for the data after receiving the data request, and then broadcasts the sequence identifier and the data to other slave segment units. After collecting and processing the responses of other master segment units, the master segment unit sends a response to the data driver. After the data driver receives the normal return information from the master segment unit, it finally responds to the user request.

[0008] The limitation of method I is that only one data driver can perform data writing on the group at the same time, otherwise the strong consistency of data will be destroyed.

[0009] The limitation of method II is that the two network sending times are serially executed, and both times require sending data, resulting in large delay, which is not conducive to the rapid response of user data requests.

[0010] Therefore, the current data writing method is not ideal. SUMMARY

[0011] Embodiments of the present application provide a data writing method and device for a distributed storage system, a storage medium, an electronic device, and a computer program product.

[0012] In a first aspect, embodiments of the present application provide a data writing method for a distributed storage system, for an electronic device, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units, the method performed by the master segment unit comprising:

[0013] receiving data and a unique identifier from the data driver, the data driver generating the unique identifier for the data;

[0014] generating a distinguishing identifier for the unique identifier;

[0015] broadcasting the unique identifier and the distinguishing identifier to each of the slave segment units, wherein the slave segment units receive the data and the unique identifier from the data driver;

[0016] receiving a response from each of the slave segment units for processing the data, and sending the response to the data driver.

[0017] In a possible implementation of the above first aspect, each of the slave segment units indexes the unique identifier, respectively processes the data associated with the distinguishing identifier, and generates the response.

[0018] In a possible implementation of the above first aspect, each of the slave segment units, upon receiving the distinguishing identifier and the data associated with the unique identifier, associates the distinguishing identifier and the data through the unique identifier, thereby processing the data associated with the distinguishing identifier and generating the response.

[0019] In a possible implementation of the above first aspect, the distinguishing identifier is a sequential identifier.

[0020] In a possible implementation of the above first aspect, the unique identifier in each of the segments is unique within a period of time.

[0021] In a second aspect, embodiments of the present application provide a data writing method for a distributed storage system, for an electronic device, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units, the method performed by the slave segment units comprising:

[0022] receiving data and a unique identifier from the data driver, the data driver generating the unique identifier for the data;

[0023] receiving the unique identification and the distinguished identification from the master segment unit, wherein the master segment unit generates the distinguished identification for the unique identification;

[0024] processing the data associated with the distinguished identification respectively with the unique identification as index, and generating a response;

[0025] sending the response to the data driver and the master segment unit.

[0026] In a possible implementation of the second aspect, after receiving the data and the distinguished identification associated with the unique identification, the slave segment unit associates the data with the distinguished identification through the unique identification, thereby processing the data associated with the distinguished identification and generating the response.

[0027] In a possible implementation of the second aspect, the master segment unit sends the response to the data driver.

[0028] In a possible implementation of the second aspect, the distinguished identification is a sequential identification.

[0029] In a possible implementation of the second aspect, the unique identification in each segment is unique within a period of time.

[0030] In a third aspect, an embodiment of the present application provides a data writing device for a distributed storage system, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units, the data writing device comprising:

[0031] a receiving unit configured to receive data and a unique identification from the data driver, wherein the data driver generates the unique identification for the data;

[0032] a generating unit configured to generate a distinguished identification for the unique identification;

[0033] a broadcasting unit configured to broadcast the unique identification and the distinguished identification to each of the slave segment units, wherein the slave segment units receive the data and the unique identification from the data driver;

[0034] a receiving unit configured to receive a response of processing the data from each of the slave segment units, and send the response to the data driver.

[0035] In a fourth aspect, an embodiment of the present application provides a data writing device for a distributed storage system, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units, the data writing device comprising:

[0036] a first receiving unit configured to receive data and a unique identifier from the data driver, the data driver generating the unique identifier for the data;

[0037] a second receiving unit configured to receive the unique identifier and a distinguishing identifier from the master segment unit, wherein the master segment unit generates the distinguishing identifier for the unique identifier;

[0038] a processing unit configured to process the data associated with the distinguishing identifier respectively with the unique identifier as an index, and generate a response;

[0039] a sending unit configured to send the response to the data driver and the master segment unit.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer executable instructions, which are executed by a processor to implement the data writing method for a distributed storage system in the first aspect or the second aspect.

[0041] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, which stores instructions, and the instructions, when executed on a computer, cause the computer to perform the data writing method for a distributed storage system in the first aspect or the second aspect.

[0042] In a seventh aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; wherein the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device performs the data writing method for a distributed storage system in the first aspect or the second aspect.

[0043] In the present application, by setting the unique identifier and the distinguishing identifier, data writing for a segment can be performed by multiple data drivers simultaneously, and strong consistency of data is maintained. Further, the master segment unit does not need to broadcast a data log with a large data volume, but only broadcasts the unique identifier and the distinguishing identifier (metadata information) with a small data volume, so that large delay is not caused. Therefore, the present application realizes high throughput and low delay while maintaining strong consistency of data. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 According to some embodiments of the present application, a block diagram of an electronic device is shown;

[0045] Figure 2 According to some embodiments of the present application, a schematic diagram of a distributed storage system is shown;

[0046] Figure 3According to some embodiments of the present application, a first flow chart of a data writing method of a distributed storage system is shown.

[0047] Figure 4 According to some embodiments of the present application, a second flow chart of a data writing method of a distributed storage system is shown.

[0048] Figure 5 According to some embodiments of the present application, a first structure diagram of a data writing apparatus for a distributed storage system is shown.

[0049] Figure 6 According to some embodiments of the present application, a second structure diagram of a data writing apparatus for a distributed storage system is shown. DETAILED DESCRIPTION

[0050] Illustrative embodiments of the present application include, but are not limited to, a data writing method, an apparatus, a medium, an electronic device, and a computer program product of a distributed storage system.

[0051] Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0052] Figure 1 According to some embodiments of the present application, a block diagram of an electronic device is shown.

[0053] As shown in Figure 1 , the electronic device 100 can include one or more processors 102, a system mainboard 108 connected with at least one of the processors 102, a system memory 104 connected with the system mainboard 108, a non-volatile memory (NVM) 106 connected with the system mainboard 108, and a network interface 110 connected with the system mainboard 108.

[0054] The processor 102 can include one or more single-core or multi-core processors. The processor 102 can include any combination of general-purpose processors (CPUs) and dedicated processors (for example, graphics processors, application processors, baseband processors, etc.). A graphics processing unit (GPU) is a dedicated processor with a higher number of cores and powerful parallel computing capabilities compared with general-purpose processors, and is widely used in computer graphics processing. In embodiments of the present application, the processor 102 can be configured to execute one or more of various embodiments as shown in Figure 3 or Figure 4 .

[0055] In some embodiments, the system mainboard 108 can include any suitable interface controller (not shown in the figure) for connecting the processor 102, the system memory 104, the NVM 106, and the network interface 110. Figure 1The system bus 108 can include any suitable interface or bus that allows for communication between at least one of the processors 102 and / or any suitable device or component in communication with the system bus 108.

[0056] In some embodiments, the system bus 108 can include one or more memory controllers to provide an interface to the system memory 104. The system memory 104 can be used to load and store data and / or instructions 120. In some embodiments, the system memory 104 of the electronic device 100 can include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0057] The non-volatile memory 106 can include one or more tangible, non-transitory computer-readable media for storage of data and / or instructions 120. In some embodiments, the non-volatile memory 106 can include any suitable non-volatile memory, such as flash memory, and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, a digital versatile disc (DVD) drive.

[0058] The non-volatile memory 106 can include a portion of the storage resources installed on the device of the electronic device 100, or it can be accessed by an external device, but not necessarily part of the external device. For example, the non-volatile memory 106 can be accessed over a network via the network interface 110.

[0059] In particular, the system memory 104 and the non-volatile memory 106 can include, respectively, a temporary copy and a permanent copy of the instructions 120. The instructions 120 can include instructions that, when executed by at least one of the processors 102, cause the electronic device 100 to implement a method as shown in Figure 3 or Figure 4 In some embodiments, the instructions 120, hardware, firmware, and / or software components thereof can additionally / alternatively be disposed in the system bus 108, the network interface 110, and / or the processors 102.

[0060] The network interface 110 can include a transceiver to provide a radio interface for the electronic device 100 to communicate with any other suitable device (e.g., a front-end module, an antenna, etc.) over one or more networks. In some embodiments, the network interface 110 can be integrated with other components of the electronic device 100. For example, the network interface 110 can be integrated with at least one of the processors 102, the system memory 104, the non-volatile memory 106, and a firmware device (not shown) having instructions that, when executed by at least one of the processors 102, cause the electronic device 100 to implement a method as shown inFigure 3 or Figure 4 one or more of the various embodiments shown.

[0061] The network interface 110 can further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 110 can be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0062] In one embodiment, at least one of the processors 102 can be packaged together with one or more controllers for the system motherboard 108 to form a system in a package (SiP). In one embodiment, at least one of the processors 102 can be integrated on the same die with one or more controllers for the system motherboard 108 to form a system on a chip (SoC).

[0063] The electronic device 100 can further include an input / output (I / O) device 112 connected to the system motherboard 108. The I / O device 112 can include a user interface to enable a user to interact with the electronic device 100; a peripheral component interface to enable peripheral components to also interact with the electronic device 100. In some embodiments, the electronic device 100 also includes a sensor to determine at least one of environmental conditions and location information related to the electronic device 100.

[0064] In some embodiments, the I / O device 112 can include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light emitting diode flash), a keyboard, and a graphics card. The graphics card is composed of a graphics processor integrated with an I / O interface (e.g., a PCIE interface) that complies with a data transmission protocol specification.

[0065] In some embodiments, the peripheral component interface can include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0066] In some embodiments, the sensor can include, but is not limited to, a gyro sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit can also be part of or interact with the network interface 110 to communicate with components of a positioning network (e.g., Global Positioning System (GPS) satellites).

[0067] It can be understood that the structural schematic of the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0068] The program code can be applied to input instructions to execute the functions described in the present application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purpose of the present application, the system for processing instructions including the processor 102 includes any system having a processor such as a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0069] The program code can be implemented in a high-level programming language or an object-oriented programming language to communicate with the processing system. When necessary, the program code can also be implemented in assembly language or machine language. In fact, the mechanisms described in the present application are not limited to the scope of any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0070] One or more aspects of at least one embodiment can be implemented by instructions stored on a computer-readable storage medium that, when executed by a processor, enable an electronic device to implement methods of embodiments described in the present application.

[0071] For ease of understanding, the related terms and concepts that may be involved in the embodiments of the present application are introduced first as follows.

[0072] Volume: a storage area in a distributed storage system.

[0073] Segment: a basic logical unit that constitutes a volume. A volume has n segments (n≥1), and the number of segments in a volume = size of the volume / size of a segment.

[0074] Segment unit: a basic unit or physical unit that constitutes a segment. There are various segment units, including master segment units and slave segment units. A segment unit corresponds to a certain continuous space on a physical disk, can store user data, and can perform corresponding processing on the data.

[0075] Storage medium: a storage medium for storing data, such as a physical disk.

[0076] Storage node: a storage node can include multiple storage media, and a storage medium can only belong to one storage node.

[0077] It can be understood that the storage node and the physical disk are both physical concepts, the storage node is used for managing a plurality of physical disks (i.e., data disks), and at the same moment, the same physical disk is only managed by a certain storage node. The volume is a logical concept, and different storage block spaces can be divided on a certain physical disk to form different volumes, i.e., segment units of the same volume can be distributed on different physical disks.

[0078] The data writing method of the distributed storage system provided in the present application can be applied to Figure 1 The electronic device 100 shown in the figure is, for example, a computer, a mobile device, a platform device, or the like.

[0079] Figure 2 FIG. 1 is a schematic diagram of a distributed storage system 20 according to an embodiment of the present application. As shown in the figure, the distributed storage system 20 includes a data drive 21 and a segment 22. The segment 22 includes one master segment unit 221 and two slave segment units 222 and 223. Figure 2

[0080] It can be understood that the distributed storage system 20 can include a plurality of data drives and a plurality of volumes (not shown in the figure), each volume including a plurality of segments, Figure 2 The segment 22 in the figure is located in one of the volumes.

[0081] Figure 3 FIG. 2 is a first flowchart of a data writing method for the distributed storage system 20 according to the present application, and the method is executed by an electronic device in which the master segment unit 221 is located. It can be understood that the method executed by the master segment unit in the present application is executed by the electronic device in which the master segment unit is located.

[0082] In step S31, the master segment unit 221 receives data and a unique identifier from the data drive 21, and the data drive 21 generates a unique identifier for the data.

[0083] Specifically, when starting initialization, the data drive 21 will request the information center (not shown in the figure) in the distributed storage system 20 to obtain logical volume information. After obtaining complete volume information, the data drive 21 can normally start and begin to accept user business data.

[0084] In the present example, for example, the current logical volume is in a three-copy backup mode, and the volume contains the segment 22. The data drive 21 will obtain the information of all segment units in the segment 22, i.e., the information of the master segment unit 221 and the slave segment units 222 and 223, according to the segment information of the segment 22 in the logical volume.

[0085] ​When the data drive 21 receives the user's write data request, the data is organized into a data log Dl in fixed size units. Here, the data drive 21 generates a unique identification IDl for the data log Dl. The data drive 21 sends the data log Dl and the unique identification IDl to each of the segment units in the segment 22, i.e., to the master segment unit 221, the slave segment units 222 and 223, respectively.

[0086] In step S32, the master segment unit 221 generates a distinguished identification for the unique identification IDl.

[0087] For example, the distinguished identification is a sequential identification, which can be a sequence of numbers that is incremented in the segment 22. For example, the master segment unit 221 generates a sequential identification SNl for the unique identification IDl.

[0088] It is understood that the distinguished identification can be any identification for distinguishing, without being limited.

[0089] In step S33, the master segment unit 221 broadcasts the unique identification IDl and the sequential identification SNl to each of the slave segment units 222 and 223. It is understood that the master segment unit 221 does not need to broadcast the data log Dl, which has a large data volume, but only broadcasts the unique identification IDl and the sequential identification SNl, which have a small data volume, so that a large delay is not caused.

[0090] It is understood that each of the slave segment units 222 and 223 receives the data log Dl and the unique identification IDl from the data drive 21, and receives the unique identification IDl and the sequential identification SNl from the master segment unit 221.

[0091] In step S34, the master segment unit 221 receives a response to processing the data log Dl from each of the slave segment units 222 and 223, and sends the response to the data drive 21.

[0092] Specifically, each of the slave segment units 222 and 223 indexes the unique identification IDl, processes the data log Dl associated with the sequential identification SNl, and generates a response. Each of the slave segment units 222 and 223, upon receiving the sequential identification SNl and the data log Dl associated with the unique identification IDl, associates the sequential identification SNl and the data log Dl by the unique identification IDl, processes the data log Dl associated with the sequential identification SNl, and generates a response.

[0093] For example, after receiving the data log Dl and the unique identification IDl from the data driver 21 and receiving the unique identification IDl and the sequential identification SNl from the master segment unit 221, the segment unit 222 can determine that the data log Dl and the sequential identification SNl are both associated with the unique identification IDl. At this time, the segment unit 222 receives the sequential identification SNl and the data log Dl associated with the unique identification IDl. Then, the sequential identification SNl and the data log Dl are associated (bound) so that the data log Dl is processed and a response is generated.

[0094] It can be understood that the identification IDl in a segment 22 is unique within a period of time, for example, one year. Therefore, the sequential identification SNl and the data log Dl can be bound one-to-one, so that each data log and sequential identification are consistent in all segment units 222 and 223.

[0095] It can be understood that the sequential identifications in each segment are independent of each other and do not interfere with each other.

[0096] It can be understood that the master segment unit 221 returns a normal response to the data driver 21 after receiving normal responses from the two segment units 222 and 223. The data driver 21 collects and counts the responses returned by the master segment unit 221 and the two segment units 222 and 223, respectively, and then returns the final result of the write data request to the user.

[0097] Figure 4 is a second flowchart of a data write method for a distributed storage system 20 according to the present application, and the method is executed by an electronic device in which the segment unit is located. It can be understood that the method executed by the segment unit in the present application is executed by an electronic device in which the segment unit is located.

[0098] The following will be described by taking the segment unit 222 in the distributed storage system 20 as an example. Figure 2

[0099] In step S41, the segment unit 222 receives data and a unique identification from the data driver 21, and the data driver 21 generates a unique identification for the data.

[0100] Specifically, when starting initialization, the data driver 21 requests a center of information (not shown in the figure) in the distributed storage system 20 to obtain logical volume information. After obtaining complete volume information, the data driver 21 can be normally started and begin to accept user business data.

[0101] In the present example, for example, the current logical volume is in a three-copy backup mode, and the volume contains a segment 22. The data driver 21 can obtain information of all segment units in the segment 22, i.e., information of the master segment unit 221 and the segment units 222 and 223, according to segment information of the segment 22 in the logical volume.​

[0102] When the data driver 21 receives the user's write data request, the data is organized into a data log Dl in a fixed size unit. Here, the data driver 21 generates a unique identification IDl for the data log Dl. The data driver 21 transmits the data log Dl and the unique identification IDl to each of the segment units in the segment 22, i.e., the master segment unit 221, the slave segment units 222 and 223, respectively.

[0103] In step S42, the unique identification and the distinguishing identification are received from the master segment unit 221 by the slave segment unit 222, wherein the master segment unit generates the distinguishing identification for the unique identification.

[0104] For example, the distinguishing identification is a sequential identification, which can be a sequence of numbers that is incremented in the segment 22. For example, the master segment unit 221 generates a sequential identification SNl for the unique identification IDl.

[0105] It can be understood that the distinguishing identification can be any identification for distinguishing without limitation.

[0106] It can be understood that the master segment unit 221 broadcasts the unique identification IDl and the sequential identification SNl to the slave segment units 222 and 223.

[0107] It can be understood that each of the slave segment units 222 and 223 receives the data log Dl and the unique identification IDl from the data driver 21, and receives the unique identification IDl and the sequential identification SNl from the master segment unit 221.

[0108] In step S43, the data associated with the distinguishing identification is processed by the slave segment unit 222 with the unique identification as an index, and a response is generated.

[0109] Specifically, the data log Dl associated with the sequential identification SNl is processed by the slave segment unit 222 with the unique identification IDl as an index, and a response is generated. The slave segment unit 222 associates the sequential identification SNl and the data log Dl by the unique identification IDl after receiving the sequential identification SNl and the data log Dl associated with the unique identification IDl, thereby processing the data log Dl associated with the sequential identification SNl and generating a response.

[0110] For example, after the segment unit 222 receives the data log D1 and the unique identification ID1 from the data driver 21 and receives the unique identification ID1 and the sequential identification SN1 from the master segment unit 221, it can be determined that the data log D1 and the sequential identification SN1 are both associated with the unique identification ID1. At this time, the segment unit 222 receives the sequential identification SN1 and the data log D1 associated with the unique identification ID1. Then, the sequential identification SN1 and the data log D1 are associated (bound) so that the data log D1 is processed and a response is generated.

[0111] In step S44, the response is sent from the segment unit 222 to the data driver 21 and the master segment unit 221.

[0112] Specifically, the response is sent from the segment unit 222 to the data driver 21 and the response is sent to the master segment unit 221.

[0113] It can be understood that the master segment unit 221 returns a normal response to the data driver 21 after receiving the normal responses from the two segment units 222 and 223. The data driver 21 collects and counts the responses returned by the master segment unit 221 and the two segment units 222 and 223 respectively, and then returns the final result of the write data request to the user.

[0114] It can be understood that the identification ID1 in a segment 22 is unique within a period of time, for example, within one year. Therefore, the sequential identification SN1 and the data log D1 can be bound one-to-one, so that each data log and sequential identification is consistent in all segment units 222 and 223.

[0115] It can be understood that the sequential identifications in each segment are independent of each other and do not interfere with each other.

[0116] The method performed by the segment unit 223 is the same as the method performed by the segment unit 222, which will not be described again.

[0117] It can be understood that when data is written to each segment through other data drivers, the method shown in Figure 3 and Figure 4 can be used.

[0118] In the present application, by setting the unique identification and the distinguishing identification, data can be written to the segment through multiple data drivers at the same time, and the strong consistency of the data is maintained. Further, the master segment unit does not need to broadcast a data log with a large amount of data, but only broadcasts a unique identification and a distinguishing identification (metadata information) with a small amount of data, so that a large delay is not generated. Therefore, the present application realizes high throughput and low delay while maintaining the strong consistency of the data.

[0119] The application further provides a data writing device for a distributed storage system, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units. Figure 5 As shown in the figure, the data writing device 50 comprises a receiving unit 51, a generating unit 52, a broadcasting unit 53 and a receiving unit 54. The receiving unit 51 receives data and a unique identifier from the data driver, the data driver generating the unique identifier for the data. The generating unit 52 generates a distinguishing identifier for the unique identifier. The broadcasting unit 53 broadcasts the unique identifier and the distinguishing identifier to each of the slave segment units, wherein the slave segment units receive the data and the unique identifier from the data driver. The receiving unit 54 receives a response of processing the data from each of the slave segment units and sends the response to the data driver.

[0120] It can be understood that the receiving unit 51, the generating unit 52, the broadcasting unit 53 and the receiving unit 54 can be realized by a processor 102 having the functions of these modules or units in the electronic device 100 where the master segment unit is located. The above-mentioned embodiments are method embodiments corresponding to the present embodiment, and the present embodiment can be implemented in cooperation with the above-mentioned embodiments. The related technical details mentioned in the above-mentioned embodiments are still valid in the present embodiment, and are not repeated here to reduce repetition. Correspondingly, the related technical details mentioned in the present embodiment can also be applied to the above-mentioned embodiments.

[0121] The application further provides a data writing device for a distributed storage system, the distributed storage system comprising a plurality of segments and a data driver, each segment comprising a master segment unit and a plurality of slave segment units. Figure 6 As shown in the figure, the data writing device 50 comprises a receiving unit 51, a generating unit 52, a broadcasting unit 53 and a receiving unit 54. The receiving unit 51 receives data and a unique identifier from the data driver, the data driver generating the unique identifier for the data. The generating unit 52 generates a distinguishing identifier for the unique identifier. The broadcasting unit 53 broadcasts the unique identifier and the distinguishing identifier to each of the slave segment units, wherein the slave segment units receive the data and the unique identifier from the data driver. The receiving unit 54 receives a response of processing the data from each of the slave segment units and sends the response to the data driver.

[0122] It can be understood that the first receiving unit 61, the second receiving unit 62, the processing unit 63 and the sending unit 64 can be realized by the processor 102 having the functions of these modules or units in the electronic device 100 where the segment unit is located. The above-mentioned embodiments are method embodiments corresponding to the present embodiment, and the present embodiment can be implemented in cooperation with the above-mentioned embodiments. The related technical details mentioned in the above-mentioned embodiments are still valid in the present embodiment, and are not repeated here to reduce repetition. Correspondingly, the related technical details mentioned in the present embodiment can also be applied to the above-mentioned embodiments.

[0123] The present application also provides a computer program product comprising computer executable instructions for implementing the data writing method for a distributed storage system of the present application by the processor 102.

[0124] The present application also provides a computer readable storage medium, wherein the instructions are stored on the storage medium, and the instructions make the computer execute the data writing method for a distributed storage system of the present application when executed on the computer.

[0125] It should be noted that the relationship terms such as first and second in the examples and descriptions of the present patent are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including one" does not exclude the presence of other identical elements in the processes, methods, articles or devices including the element.

[0126] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that various changes in form and detail can be made therein without departing from the spirit and scope of the present application.

[0127] It is to be understood that the foregoing description is purely illustrative and exemplary of the embodiments of the application and, therefore, not limiting of its scope, for the sole and only purpose of aiding in the understanding of the disclosed embodiments of the application. For while the illustrative embodiments of the application have been described with reference to specific embodiments, it is to be understood that various other adaptations and / or modifications are intended to be considered. For example, it is to be understood that such adaptations and / or modifications can be made in the illustrative embodiments of the application described above, with knowledge obtained in the use of such technology. It is the intent, therefore, to be limited only as indicated by the claims.

[0128] It is to be understood that the embodiments of the present application as described above are merely illustrative and exemplary of the various aspects of the present application, and that the scope of the present application is not to be limited by the specific illustrative embodiments described above. Rather, the scope of the present application is to be defined by the claims set forth below. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. Other embodiments will be apparent to those skilled in the art in view of the detailed description of the application.

[0129] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or apparatus otherwise disclosed in the specification, can be made, except that at least some of such features and / or processes or units are mutually exclusive. Unless explicitly stated otherwise, each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features that serve the same, equivalent or similar purpose.

[0130] Further, those skilled in the art will appreciate that a combination of features of different embodiments can mean within the scope of the application and form a different embodiment. For example, in the claims, any one of the claimed embodiments can be used in any combination.

Claims

1. A data writing method for a distributed storage system, used in an electronic device, characterized in that, The distributed storage system includes multiple segments and a data driver. Each segment includes a master segment unit and multiple slave segment units. The method executed by the master segment unit includes: The data driver receives data and a unique identifier, and the data driver generates the unique identifier for the data. Generate a distinguishing identifier for the unique identifier; The unique identifier and the distinguishing identifier are broadcast to each of the slave segment units, wherein the slave segment units receive the data and the unique identifier from the data driver; Receive a response from each of the slave segment units to process the data, and send the response to the data driver.

2. The data writing method according to claim 1, characterized in that, Each of the sub-segments uses the unique identifier as an index to process the data associated with the distinguishing identifier and generate the response.

3. The data writing method according to claim 2, characterized in that, After receiving the distinguishing identifier associated with the unique identifier and the data, each of the slave units associates the distinguishing identifier and the data through the unique identifier, thereby processing the data associated with the distinguishing identifier and generating the response.

4. The data writing method according to any one of claims 1-3, characterized in that, The distinguishing identifier is a sequence identifier.

5. The data writing method according to claim 1, characterized in that, The unique identifier in each segment is unique for a period of time.

6. A data writing method for a distributed storage system, used in an electronic device, characterized in that, The distributed storage system includes multiple segments and a data driver. Each segment includes a master segment unit and multiple slave segment units. The method executed by the slave segment units includes: The data driver receives data and a unique identifier, and the data driver generates the unique identifier for the data. The main segment unit receives the unique identifier and the distinguishing identifier, wherein the main segment unit generates the distinguishing identifier for the unique identifier; Using the unique identifier as an index, the data associated with the distinguishing identifier are processed respectively, and a response is generated; The response is sent to the data driver and the main segment unit.

7. The data writing method according to claim 6, characterized in that, After receiving the distinguishing identifier associated with the unique identifier and the data, the slave unit associates the distinguishing identifier and the data with the unique identifier, thereby processing the data associated with the distinguishing identifier and generating the response.

8. The data writing method according to claim 6, characterized in that, The main segment unit sends the response to the data driver.

9. The data writing method according to any one of claims 6-8, characterized in that, The distinguishing identifier is a sequence identifier.

10. The data writing method according to claim 6, characterized in that, The unique identifier in each segment is unique for a period of time.

11. A data writing device for a distributed storage system, characterized in that, The distributed storage system includes multiple segments and a data driver, each segment including a master segment unit and multiple slave segment units, and the data writing device includes: The receiving unit receives data and a unique identifier from the data driver, wherein the data driver generates the unique identifier for the data. The generation unit generates a distinguishing identifier for the unique identifier; A broadcasting unit broadcasts the unique identifier and the distinguishing identifier to each of the slave segment units, wherein the slave segment units receive the data and the unique identifier from the data driver; The receiving unit receives a response from each of the slave segment units to process the data, and sends the response to the data driver.

12. A data writing device for a distributed storage system, characterized in that, The distributed storage system includes multiple segments and a data driver, each segment including a master segment unit and multiple slave segment units, and the data writing device includes: The first receiving unit receives data and a unique identifier from the data driver, wherein the data driver generates the unique identifier for the data. The second receiving unit receives the unique identifier and the distinguishing identifier from the main segment unit, wherein the main segment unit generates the distinguishing identifier for the unique identifier; The processing unit uses the unique identifier as an index to process the data associated with the distinguishing identifier and generates a response. The sending unit sends the response to the data driver and the main segment unit.

13. A computer program product comprising computer-executable instructions, characterized in that, The instructions are executed by the processor to implement the data writing method according to any one of claims 1-5 or 6-10.

14. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, cause the computer to perform the data writing method according to any one of claims 1-5 or 6-10.

15. An electronic device, characterized in that, include: One or more processors; One or more memory units; The one or more memories store one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the data writing method according to any one of claims 1-5 or 6-10.

Images